1 Introduction
The threshold concepts framework (Meyer & Land, 2003) has gained much attention within teaching and learning research and is increasingly being anchored in a variety of disciplines. Threshold concepts provide essential impulses for the design of disciplinary curricula and courses, as its central idea consists in linking disciplinary concepts to the individual lifeworld and subjective experiencing of learners. In particular, threshold concepts are seen as way to prepare learners for fundamental epistemic and ontological changes of their perspective on disciplinary phenomena (Meyer & Land, 2005). It is acknowledged across various disciplines that this transformation of perspective not only involves the development of expertise in a specific subject but also has identity-forming properties and can involve a repositioning of learners in relation to a discipline (Land, Cousin, Meyer, & Davies, 2005). Land, Rattray, and Vivian (2014) operationalize this in terms of reconstitutive and discursive properties. Moreover, this transformation may result in learners belonging to a certain community that shares a specific way of understanding and perceiving, which is also observable through the use of professional language. Since this transformative potential is considered to be of high importance for learning, a central issue concerns the design of threshold concepts-oriented assessment. From an educator’s point of view, for instance, it is important to be able to identify learners’ current needs and provide them with focused support in their learning process. Furthermore, gaining evidence on the effect of threshold concepts that have been embedded may also contribute to the further development of teaching formats.
The threshold concepts framework in particular reminds us not only to look at the outcome of learning, but to understand concept transformation as an individual process and therefore also to take learning preconditions as well as learners’ subjective perception into account (Davies & Mangan, 2007). This leads to the question of which characteristics can actually be used to observe
Finally, there is a need for instruments that enable the assessment of individual learning trajectories. While qualitative approaches to assess threshold transitions already exist (Vidal, Smith, & Spetic, 2015; Rose, Leisyte, Haertel, & Terkowsky, 2019), quantitative procedures occur less often. The liminal space, which is considered so important for threshold concepts-based learning, also requires process-oriented assessment, for which appropriate tools must be made available. In this context, Wright and Gilmore (2012, p. 627) ask “how can the dynamics of liminality with respect to these transformations be meaningfully assessed?”
In order to enrich perspectives on assessment in threshold concepts-oriented courses, this chapter presents a systematic overview of approaches appearing in the literature to date. This review forms the basis for raising particular issues for future research on assessment of student learning.
2 Assessment of Student Learning in Threshold Concepts-Oriented Teaching and Learning
Assessment of student learning is a key task for educators and this applies also to threshold concepts-oriented courses. The challenge is to address the specific properties of threshold concepts, which focus on ontological and epistemic shift of the learner’s perception, in learning assessment. Unlike knowledge tests, for example, it is not always possible to evaluate a subjective concept construction in categories such as “true” or “false” (Entwistle, 2005). It is therefore necessary to differentiate assessment purposes and discuss their implications in the context of threshold concepts-oriented learning. One way of structuring assessment activities, and thus systematically considering tools
Firstly, learning precondition considers the individual situation of learners at the beginning of a learning process. Such conditions can include, for example, socio-demographic specifics like age, gender, income etc. and associated with these are certain circumstances that are significant to learners. Prior knowledge or certain learning preferences are also among these conditions with which learners enter a learning arrangement. Preconditions are crucial for learning because they influence the subjective perception of learners and thus also the way in which they encounter the learning subject and identify it as being meaningful for them. It can therefore be assumed that learners with many years of professional experience in a certain field have a different perspective on disciplinary phenomena from students who have just come out of school. Drawing on the phenomenographic approach, Marton and Pang (2008) point out that learning conditions can also lie in different forms of experiencing disciplinary phenomena and can thus go beyond pure knowledge aspects. Hence, assessment could also be of help for learners to activate prior knowledge and to reflect on their learning path in retrospect and thus in turn to create a starting point for their individual learning trajectory. Following this constructivist view, learning conditions are the subjective starting point for learning processes and thus of particular relevance for educators, for example in order to identify concept fragments as starting points of learning.
Secondly, the assessment of learning processes focuses on activities, perceptions, and progress of learners during a learning arrangement. This procedure is also referred to as ‘formative assessment’ or ‘assessment for learning’ (Bennett, 2011). This can include, for example, the extent to which learners succeed in building up knowledge or where they encounter difficulties. In this way, assessment of the learning process fulfils an important feedback function for educators in order to recognize support needs and to be able to intervene in the learning process in a specific way. For learners, this form of assessment can serve a reflective function, reviewing what they have learned so far, dealing with difficulties, and planning further steps. This makes it particularly clear that assessment of learning and teaching method merge, as Rattray (2017) discusses using the example of ipsative self-assessment in threshold transitions. It also becomes clear, that every learning process is highly individual and leads to the circumstance that some students succeed more than others in building up professional expertise (Rattray, 2016; Land, 2019).
Finally, the assessment of learning outcomes is intended to gain information about individual learning progress at a given point in time and to find indications about the development of professional expertise. This form of
The assessment functions listed are not to be understood as distinct, but each sets a specific focus. They also do not follow a specific chronological order, but complement each other. For example, it can be assumed that the assessment of learning preconditions is often included in formative and summative assessments.
By focusing on the subjective handling of disciplinary phenomena, threshold concepts have important implications for the assessment of learning preconditions and learning outcomes. Characteristic for the embedding of threshold concepts in teaching and learning is the linking of disciplinary phenomena with the subjective perspective of learners. Didactically, the threshold concept approach combines the question of ‘what’ (content) with the question of ‘how’ (methods). Moreover, it offers opportunities to ask ‘for what’ (assessment), for example, when preliminal variation reveals which perspectives learners already have or lack on a specific phenomenon. Davies (2006) illustrates in this context how a confrontation with threshold concepts both reveals students’ conception of economic phenomena and may also fulfil a starting point for deeper reflection in their learning process.
Threshold concepts-oriented learning involves considering ontological and epistemic aspects (Land, Meyer, & Baillie, 2010), so that individual perceptions and feelings of learners are in the spotlight. This may refer, for example, to the mental construction of phenomena, such as the separation of a concept into its components, as Ashwin (2008) demonstrates regarding the market concept within economics. However, the focus can also be on beliefs, attitudes, intentions, or role models that learners possess and use to orient themselves and evaluate their relationships (Land & Meyer, 2010). This is a fundamental difference to approaches that, for example, focus on the development of competencies and are intended to prepare students for concrete problem challenges. Threshold concepts do not directly lead to concrete conclusions about which specific competencies (e.g. in cognitive, social or methodological terms) are addressed or in which scenarios they should be applied. Student assessment (especially summative assessment), however, appears often closely mapped to a competency model. In contrast to such approaches, concept-oriented
In addition to concept transition, the path to it also plays a particularly important role within threshold concepts and therefore delivers valuable implications for the assessment of learning processes. The liminal space, which is identified as essential for threshold transitions, provides a dedicated idea of the learner’s journey and can be understood as a further challenge for student assessment. Assuming that learners disengage from old conceptions and enter a fluid state involves them into an active process of pattern exploration. Threshold concept acquisition does not follow a predetermined order, as Hawkins and Edwards (2015, p. 32) note:
threshold concepts are neither experienced one at a time, nor do they represent a fixed, linear route.
Characteristic of this process are oscillations between pre-concepts and threshold concepts (Land & Meyer, 2010; Tucker et al., 2014), where learners may revert to old concepts or initially gain only a partial understanding of threshold concepts. The troublesome effect of threshold concepts – which is one of their key properties (Land et al., 2005) – can paralyze learners, confronting them with ‘monsters of doubt’ (Hawkins & Edwards, 2015, p. 24), which can cause helplessness and resignation. D’Mello, Lehman, Pekrun, and Graesser (2014) consider that learning processes can be characterized by conceptual ‘dead ends’ that lead to confusion and uncertainty. Learners are forced to step back and begin to critically reflect and adapt their problem-solving strategies. To regulate the ‘monsters’ or the sense of uncertainty, learners start to experiment with new patterns. This experimentation can be interpreted from an assessment perspective as indicator of learners’ engagement with the learning subject. The learner’s action in this area between old and new patterns is described as oscillative and recursive, as neither letting go nor adapting happens ad hoc, but in a process of pattern testing (Land et al., 2014). In this context, it is emphasized that confidence and trust in one’s own abilities can help learners to cope with the process of concept adaptation (Rattray, 2016).
Due to the importance of liminal phases to concept transformation, there is a need to include a process perspective in assessment efforts (Land et al., 2005).
3 Assessment Approaches of Student Learning in Threshold Concepts-Oriented Teaching and Learning
Table 2.1 presents an overview that captures existing assessment approaches and structures them in terms of their functions in threshold concepts-oriented teaching and learning. The focus is explicitly on those approaches that relate to assessment in learning contexts and thus directly to students. Therefore, approaches that exclusively aim to derive disciplinary candidate threshold concepts (e.g., in the form of Delphi studies) are not considered. In addition to journal articles, the contributions in the edited books that accompany the threshold concepts conferences are considered. The table lists threshold concepts-oriented assessment approaches and exemplary studies.
Assessment of student learning in threshold concepts-oriented teaching and learning
| Focus of assessment | Tools and analysis |
|---|---|
| Assessment of learning processes | |
| Activities of students during threshold concept acquisition, such as exchanging views with other students, involving educators, applying learning materials, structuring problems and approaches for solving |
|
| Identification and differentiation of forms of troublesomeness |
|
| Localization of liminality and its cognitive, emotional and motivational aspects |
|
| Assessment of learning preconditions and outcomes | |
Differences in threshold conceptions including subjective beliefs
|
|
| Learners’ conception of specific disciplinary phenomena |
|
| Learners’ connection between disciplinary threshold concepts |
|
It is worth noting that the majority of assessment approaches relies on qualitative approaches. With the use of qualitative methods, it is advantageous to be able to draw particularly profound conclusions about how students deal with threshold concepts. Thus, written answers or think-aloud narratives seem to offer a good opportunity to fathom the perception of disciplinary phenomena and to draw conclusions about concept adaptations. This dominance certainly also has to do with subjective positioning towards disciplinary phenomena, which can be very finely branched and encompass different aspects such as cognitions, emotions, and motivational factors. When it comes to uncovering troublesome aspects in liminal phases, interview procedures seem to be popular in the approaches listed above. This overview thus paints a picture
A rare quantitative approach is used by the studies of Strunk et al. (2015), Sender (2017) and Geiger (2022), where liminal transitions are investigated with the help of biofeedback. The assumption is that troublesomeness in liminal phases can be expressed by experienced confusion and stress, which are accompanied by physical symptoms. Heart rate and skin conductance are used here as indicators of changing experiences of stress and interpreted in terms of threshold transitions.
The focus on individual threshold concept characteristics described by Barradell (2013) can also be observed in the studies in Table 2.1. While individual aspects such as integration and troublesomeness are frequently considered, a parallel consideration of several or all of the threshold concept characteristics is mostly absent. This circumstance points to a problem of the threshold concept approach, which underlines the question of the weighting of different
It is noticeable that certain assessment purposes are often merged with each other and aim on preconditions, learning processes and learning outcomes at the same time. The merging is accompanied by the combination of different assessment tools (for instance Shanahan et al., 2008). This meets the call to understand learning preconditions, learning process, and learning outcome as integrative and can help to gain a more complete picture of threshold concepts-oriented courses.
4 Conclusion and Perspectives for Further Research
It becomes clear that assessment is an essential issue in threshold concepts-oriented teaching and learning. The assessment of student learning brings up a number of challenges.
Firstly, educators will require didactic expertise. Educators have to be able to select and use appropriate tools for their courses and need to know how to use specific methods (e.g. structured interviews) to obtain reliable information about learning. Moreover, this expertise also includes being able to utilize assessment evidence and, for example, to provide distinct support for learners or to further develop learning formats.
Secondly, the assessment must fit the learner. For example, one problem seen is that learners may not have the language abilities to respond to exams or interviews in a way that allows useful feedback (Nicola-Richmond, Pépin, Larkin, & Taylor, 2018). Also, parts of information acquisition and processing take place unconsciously, so that learners may not remember relevant details of threshold concept adaptations. Therefore, tools that reveal mechanisms of information perception and processing (such as eye movements, heart rate and skin conductance, Geiger, 2022) can make a helpful contribution here.
A threshold concept-oriented approach requires an assessment framework that shapes learning along the perception and experience of phenomena. Possible conceptual topologies of learners should be explored, reflecting both the development of professional expertise and identity exploration. This can also include beliefs (such as attitudes or self-efficacy) or behavioral dispositions, since a fundamental transformation can also include an individual repositioning towards disciplinary phenomena.
Furthermore, it is important to investigate how assessment can address liminality. Koopmans (2014) sees a fundamental need for research to investigate the ‘black box’ of the learning process, which must go beyond a pure consideration
The individuality attributed to threshold crossings (Land et al., 2014) requires a methodological framework for recording the learning process. If threshold crossings entail a fundamental ontological and epistemic change in the individual perspective, then this should also have an effect on perception, knowledge, and subjective beliefs. Corresponding indicators have therefore to be considered that can validly observe this through the learning process. At the interface of threshold concepts and phenomenography, for instance, procedures for perceptual assessment already exist (Pang, 2010).
Parallel to the further development of assessment tools, it is important to explore external criteria for the study of learner transformation. In the context of the partly heterogeneous expected outcomes of participation in a course, it is necessary to derive suitable indicators that can also map different transformation processes and to connect it to specific configurations of particular courses. Even within the broader threshold concept literature, surprisingly few approaches are dedicated to empirically investigate the interplay between learning arrangements and subjective learning. However, this is precisely where a central promise of the threshold concept approach lies, which opens up a wide scope of further research perspectives.
References
Ashwin, A. (2008). Threshold concept acquisition in economics for the 14–19 age group. In R. Land, J. H. F. Meyer, & J. Smith (Eds.), Threshold concepts within the disciplines (pp. 173–184). Sense. https://doi.org/10.1163/9789460911477_014
Barradell, S. (2013). The identification of threshold concepts: A review of theoretical complexities and methodological challenges. Higher Education, 65(2), 265–276. https://doi.org/10.1007/s10734-012-9542-3
Bennett, R. E. (2011). Formative assessment: a critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5–25. https://doi.org/10.1080/0969594X.2010.513678
Bennion, J., Cannon, B., Hill, B., Nelson, R., & Ricks, M. (2020). Asking the right questions: Using reflective essays for experiential assessment. Journal of Experiential Education, 43(1), 37–54. https://doi.org/10.1177/1053825919880202
Bolinger, A. R., & Brown, K. D. (2015). Entrepreneurial failure as a threshold concept: The effects of student experiences. Journal of Management Education, 39(4), 452–475. https://doi.org/10.1177/1052562914560794
Davies, P., & Mangan, J. (2007). Threshold concepts and the integration of understanding in economics. Studies in Higher Education, 32(6), 711–726. https://doi.org/10.1080/03075070701685148
Davies, P., & Mangan, J. (2010). Assessing progression in students’ economic understanding. The role of threshold concepts. In J. H. F. Meyer, R. Land, & C. Baillie (Eds.), Threshold concepts and transformational learning (pp. 193–206). Sense. https://doi.org/10.1163/9789460912078_013
D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153–170. https://doi.org/10.1016/j.learninstruc.2012.05.003
Entwistle, N. (2005). Learning outcomes and ways of thinking across contrasting disciplines and settings in higher education. Curriculum Journal, 16(1), 67–82. https://doi.org/10.1080/0958517042000336818
Geiger, J.-M. (2022). Schwellenkonzeptorientierte entrepreneurship education – Ein wirtschaftsdidaktischer Ansatz unter Berücksichtigung komplexitätswissenschaftlicher Theorien und Methoden. Springer Gabler. https://doi.org/10.1007/978-3-658-39479-0
Harlow, A., & Peter, M. (2014). Mastering threshold concepts in tertiary education: “I know exactly what you are saying and I can understand it but I’ve got nowhere to hook it”. Waikato Journal of Education, 19(2), 7–23. https://doi.org/10.15663/wje.v19i2.95
Hawkins, B., & Edwards, G. (2015). Managing the monsters of doubt: Liminality, threshold concepts and leadership learning. Management Learning, 46(1), 24–43. https://doi.org/10.1177/1350507613501736
Kiley, M., & Wisker, G. (2009). Threshold concepts in research education and evidence of threshold crossing. Higher Education Research & Development, 28(4), 431–441. https://doi.org/10.1080/07294360903067930
Kinchin, I. M., Cabot, L. B., & Hay, D. B. (2010). Visualizing expertise: Revealing the nature of a threshold concept in the development of an authentic pedagogy for clinical education. In J. H. F. Meyer, R. Land, & C. Baillie (Eds.), Threshold concepts and transformational learning (pp. 81–96). Sense. https://doi.org/10.1163/9789460912078_006
Knight, P. T. (2002). Summative assessment in higher education: Practices in disarray. Studies in Higher Education, 27(3), 275–286. https://doi.org/10.1080/03075070220000662
Koopmans, M. (2014). Change, self-organization and the search for causality in educational research and practice. Complicity: An International Journal of Complexity and Education, 11(1), 20–39. https://doi.org/10.29173/cmplct19523
Land, R. (2019). The Labyrinth within: Threshold concepts, archetype and myth. In J. A. Timmermans & R. Land (Eds.), Threshold concepts on the edge (pp. 3–18). Brill, Sense. https://doi.org/10.1163/9789004419971_001
Land, R., Cousin, G., Meyer, J. H. F., & Davies, P. (2005). Threshold concepts and troublesome knowledge (3): Implications for course design and evaluation. In C. Rust (Ed.), Improving student learning – diversity and inclusivity, Proceedings of the 12th Improving Student Learning Conference (pp. 53–64). Oxford Centre for Staff and Learning Development (OCSLD).
Land, R., & Meyer, J. H. F. (2010). Threshold concepts and troublesome knowledge (5): Dynamics of assessment. In J. H. F. Meyer, R. Land, & C. Baillie (Eds.), Threshold concepts and transformational learning (pp. 61–79). Sense. https://doi.org/10.1163/9789460912078_005
Land, R., Meyer, J. H. F., & Baillie, C. (2010). Threshold concepts and transformational learning. In J. H. F. Meyer, R. Land, & C. Baillie (Eds.), Threshold concepts and transformational learning (pp. ix–xlii). Sense. https://doi.org/10.1163/9789460912078
Land, R., Rattray, J., & Vivian, P. (2014). Learning in the liminal space: A semiotic approach to threshold concepts. Higher Education, 67(2), 199–217. https://doi.org/10.1007/s10734-013-9705-x
Meyer, J. H. F., & Land, R. (2005). Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual framework for teaching and learning. Higher Education, 49(3), 373–388. https://doi.org/10.1007/s10734-004-6779-5
Morgan, D. A. (2019). Learning in liminality. Student experiences of learning during a nursing study abroad journey: A hermeneutic phenomenological research study. Nurse Education Today, 79, 204–209. https://doi.org/10.1016/j.nedt.2019.05.036
Nicola-Richmond, K., Pépin, G., Larkin, H., & Taylor, C. (2018). Threshold concepts in higher education: a synthesis of the literature relating to measurement of threshold crossing. Higher Education Research & Development, 37(1), 101–114. https://doi.org/10.1080/07294360.2017.1339181
Pang, M. F. (2010). Modes of variation in pupil’s apprehension of a threshold concept in economics. In J. H. F. Meyer, R. Land, & C. Baillie (Eds.), Threshold concepts and transformational learning (pp. 365–381). Sense. https://doi.org/10.1163/9789460912078_023
Quinlan, K. M., Male, S., Baillie, C., Stamboulis, A., Fill, J., & Jaffer, Z. (2013). Methodological challenges in researching threshold concepts: A comparative analysis of three projects. Higher Education, 66(5), 585–601. https://doi.org/10.1007/s10734-013-9623-y
Rattray, J. (2016). Affective dimensions of liminality. In R. Land, J. H. F. Meyer, & M. T. Flanagan (Eds.), Threshold concepts in practice (pp. 67–76). Sense. https://doi.org/10.1007/978-94-6300-512-8_6
Rattray, J. (2017). Assessing liminality: The use of ipsative formative assessment during a postgraduate taught induction programme to support the development of criticality. In G. Hughes (Ed.), Ipsative assessment and personal learning gain: Exploring international case studies (pp. 149–171). Palgrave Macmillan UK. https://doi.org/10.1057/978-1-137-56502-0_8
Rose, A.-L., Leisyte, L., Haertel, T., & Terkowsky, C. (2019). Emotions and the liminal space in entrepreneurship education. European Journal of Engineering Education, 44(4), 602–615. https://doi.org/10.1080/03043797.2018.1553937
Scott, J., Peter, M., & Harlow, A. (2012). An electronics threshold-concept inventory. Paper presented at the Proceedings of IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) 2012. https://doi.org/10.1109/TALE.2012.6360366
Sender, T. (2017). Wirtschaftsdidaktische Lerndiagnostik und Komplexität. Lokalisierung liminaler Unsicherheitsphasen im Hinblick auf Schwellenübergänge. Springer Gabler. https://doi.org/10.1007/978-3-658-18947-1
Shanahan, M., Foster, G., & Meyer, J. H. F. (2008). Associations among prior acquisition of threshold concepts, learning dimensions, and examination performance in first-year economics. In R. Land, J. H. F. Meyer, & J. Smith (Eds.), Threshold concepts within the disciplines (pp. 155–172). Sense. https://doi.org/10.1163/9789460911477_013
Tucker, V. M. (2016). Learning experiences and the liminality of expertise. In R. Land, J. H. F. Meyer, & M. T. Flanagan (Eds.), Threshold concepts in practice (pp. 93–106). Sense. https://doi.org/10.1007/978-94-6300-512-8_8
Vidal, N., Smith, R., & Spetic, W. (2015). Designing and teaching business & society courses from a threshold concept approach. Journal of Management Education, 39(4), 497–530. https://doi.org/10.1177/1052562915574595
Wright, A. L., & Gilmore, A. (2012). Threshold concepts and conceptions: Student learning in introductory management courses. Journal of Management Education, 36(5), 614–635. https://doi.org/10.1177/1052562911429446
Yip, J., & Raelin, J. A. (2012). Threshold concepts and modalities for teaching leadership practice. Management Learning, 43(3), 333–354. https://doi.org/10.1177/1350507611422476
Zander, C., Boustedt, J., Eckerdal, A., McCartney, R., Moström, J. E., Ratcliffe, M., & Sanders, K. (2008). Threshold concepts in computer science: A multi-national empirical investigation. In R. Land, J. H. F. Meyer, & J. Smith (Eds.), Threshold concepts within the disciplines (pp. 105–118). Sense. https://doi.org/10.1163/9789460911477_009