Development and analysis of assessments that promote sensemaking in physics

Abstract

Assessments are an integral part of academic environments and can present opportunities for students to make sense of novel contexts using their existing ideas. Assessments also provide insights on students’ learning and the efficacy of the pedagogical practices. Consequently, physics education research shares a storied history of developing research-based assessments (RBAs) that support students' sensemaking. However, contemporary studies have noted several shortcomings of the existing RBAs such as: (i) lack of clarity for instructors in interpreting students’ scores to make modifications to their instruction, (ii) misalignment between the content of the assessments and the local learning goals of the instructors, (iii) scarcity of standardized assessments for undergraduate physics, and (iv) the need to shift the focus of RBAs from "knowing" to "doing" physics. Researchers have also called for probing the contextual factors that influence students' sensemaking in physics. In light of these observations, I present methodological and theoretical approaches to developing and analyzing next-generation RBAs. I introduce the development process of a novel RBA -- the Thermal and Statistical Physics Assessment (TaSPA) -- which focuses on assessing the "doing" aspect of physics along with providing actionable feedback for instructors to modify their courses. Additionally, this assessment allows instructors to choose what they wish to assess, thereby bridging the gap between assessment objectives and the local learning goals of the instructors. I elucidate the leveraging of existing theoretical and design frameworks in the development of TaSPA and how the interplay of these frameworks addresses some of the shortcomings of the contemporary assessments. This diagnostic represents a paradigm shift in how assessments are envisioned and designed by the discipline-based education research community. I also complement the contemporary literature by theoretically exploring assessment task features that increase the likelihood of students sensemaking in physics. I identify the task features by first noting the salient characteristics of the sensemaking process as described in the science education literature. Existing theoretical ideas from cognitive psychology, education, and philosophy of science are then leveraged in unpacking the task features which elicit the characteristics of sensemaking. Furthermore, I leverage Conjecture Mapping -- a framework from design-based research -- to articulate how the proposed task features elicit the desired outcome of sensemaking. I argue that to promote sensemaking, tasks should cue students to unpack the underlying mechanism of a real-world phenomenon by coordinating multiple representations and by physically interpreting mathematical expressions. The proof of concept of this idea is then presented through analysis of students' reasoning about the tasks embodying the proposed features. The analysis is then extended to provide an explicit account of the intertwining between modeling and sensemaking processes. The analyses reveal that particular aspects of modeling and sensemaking processes co-occur. For instance, the priming on the "given'' information from the problem statement constituted the students' engagement with their mental models, and their attempts to resolve inconsistencies in understanding involved the use of external representations. I find that barriers experienced in modeling can inhibit students' sustained sensemaking. Major contributions of this work include: (i) elucidating a methodological approach in developing an RBA that promotes the "doing'' aspect of physics; (ii) demonstrating an agent-based perspective in exploring assessment task features; (iii) operationalizing conjecture mapping in the context of task design in physics; (iv) introducing a methodology extendable to unpack task features which can elicit other valued epistemic practices; and (v) an explicit framework-based unpacking of the association between modeling and sensemaking. This dissertation opens up avenues for future explorations such as extending the proposed methodology in developing RBAs for other upper-division physics courses such as quantum mechanics. The presented theoretical and methodological approaches can also be extended in exploring features of the assessment tasks that promote additional epistemic practices such as argumentation and modeling. Researchers can also explore expanding the proposed list of task features, and the accompanying constraints (if any).