Influence of multimedia hints on conceptual physics problem solving and visual attention

Abstract

Previous research has showed that visual cues can improve learners' problem solving performance on conceptual physics tasks. In this study we investigated the influence of multimedia hints that included visual, textual, and audio modalities, and all possible combinations thereof, on students' problem solving performance and visual attention. The participants (N = 162) were recruited from conceptual physics classes for this study. Each of them participated in an individual interview, which contained four task sets. Each set contained one initial task, six training tasks, one near transfer task and one far transfer task. We used a 2 (visual hint/no visual hint) x 2 (text hint/no text hint) x 2 (audio hint/no audio hint) between participant quasi-experimental design. Participants were randomly assigned into one of the eight conditions and were provided hints for training tasks, corresponding to the assigned condition. Our results showed that problem solving performance on the training tasks was affected by hint modality. Unlike what was predicted by Mayer's modality principle, we found evidence of a reverse modality effect, in which text hints helped participants solve the physics tasks better than audio hints. Then we studied students’ visual attention as they solved these physics tasks. We found the participants preferentially attended to visual hints over text hints when they were presented simultaneously. This effect was unaffected by the inclusion of audio hints. Text hints also imposed less cognitive load than audio hints, as measured by fixation durations. And presenting visual hints caused more cognitive load while fixating expert-like interest areas than during the time intervals before and after hints. A theoretical model is proposed to explain both problem solving performance and visual attention. According to the model, because visual hints integrated the functions of selection, organization, and integration, this caused a relatively heavy cognitive load yet improved problem solving performance. Furthermore, text hints were a better resource for complex linguistic information than transient audio hints. We also discuss limitations of the current study, which may have led to results contrary to Mayer's modality principle in some respects, but consistent with it in others.