Exploring multimodal learning

I have long been interested in the way educators can use a variety of learning styles to augment their teaching and promote students' learning. In particular, I have advocated using audiobooks since my early teaching days. Recently, I have started exploring using iPad book apps as an integrated multimedia reading experience. So I was particularly interested in reading a research report published by Cisco, done by the Metiri Group:

Metiri Group (2008). Multimodal learning through media: What the research says. Retrieved from http://www.cisco.com/web/strategy/docs/education/Multimodal-Learning-Through-Media.pdf

As cognitive neuroscientists and education researchers have come to understand the way that the human brain processes information, it has become clear that we process visual information using different channels in the brain than linguistic information. In their review of the research on multimodal learning, the Metiri Group (2008) notes,

Recent technological advances through functional Magnetic Resonance Imaging (fMRI) scans confirm a dual coding system through which visuals and text/auditory input are processed in separate channels, presenting the potential for simultaneous augmentation of learning. The bottom line is that students using well-designed combinations of visuals and text learn more than students who only use text. (p. 3)

This combination of inputs through multimodal learning can be achieved in many ways. A lecture augmented by clear visuals and printed notes helps students see and hear the information. Likewise, a well-designed book app or CD-ROM storybook makes clear use of this type of multimodal input. Multimedia books like book apps can combine bright illustrations and appealing animation, narration both of the text and of the character’s dialog, written text for this narrative and dialog, and interactive features that bring the reader to touching the screen to activate dialog and movement.

Several principles of multimedia learning and design can be applied to the understanding of effective book apps. The Metiri Group (2008) summarizes several principles of multimedia design that affect learning. Students learn better when words and pictures are combined near each other and simultaneously - not several pages later. They learn better when content is focused and concise; input from different channels should be related but not redundant or duplicated. Individuals will benefit from design effects in different ways, with particular effect being noticed by low-knowledge or high-spatial learners. And finally, narration combined with animation supports understanding better than animation and text only, without narration.

It is of particular interest that much of the research examined by the Metiri Group (2008) focused on older students and higher order thinking, but it applies equally well to multimedia digital books for beginning readers. These readers need multiple inputs to support their efforts at making meaning from books. As Grimshaw, Dungworth and McKnight (2007) note, multimodal features such as narration, word pronunciation, sound effects and animation can help remove the effort from decoding individual words and help children focus on making meaning from texts. The question at hand is whether these these multimedia elements can really enhance understanding of stories or nonfiction, or whether they are essentially entertainment. How does reading these multimedia texts differ at its essence from reading traditional print books?