Exploring multimodal learning

I have long been interested in the way educators can use a variety of learning styles to augment their teaching and promote students' learning. In particular, I have advocated using audiobooks since my early teaching days. Recently, I have started exploring using iPad book apps as an integrated multimedia reading experience. So I was particularly interested in reading a research report published by Cisco, done by the Metiri Group:

Metiri Group (2008). Multimodal learning through media: What the research says. Retrieved from http://www.cisco.com/web/strategy/docs/education/Multimodal-Learning-Through-Media.pdf

As cognitive neuroscientists and education researchers have come to understand the way that the human brain processes information, it has become clear that we process visual information using different channels in the brain than linguistic information. In their review of the research on multimodal learning, the Metiri Group (2008) notes,

Recent technological advances through functional Magnetic Resonance Imaging (fMRI) scans confirm a dual coding system through which visuals and text/auditory input are processed in separate channels, presenting the potential for simultaneous augmentation of learning. The bottom line is that students using well-designed combinations of visuals and text learn more than students who only use text. (p. 3)

This combination of inputs through multimodal learning can be achieved in many ways. A lecture augmented by clear visuals and printed notes helps students see and hear the information. Likewise, a well-designed book app or CD-ROM storybook makes clear use of this type of multimodal input. Multimedia books like book apps can combine bright illustrations and appealing animation, narration both of the text and of the character’s dialog, written text for this narrative and dialog, and interactive features that bring the reader to touching the screen to activate dialog and movement.

Several principles of multimedia learning and design can be applied to the understanding of effective book apps. The Metiri Group (2008) summarizes several principles of multimedia design that affect learning. Students learn better when words and pictures are combined near each other and simultaneously - not several pages later. They learn better when content is focused and concise; input from different channels should be related but not redundant or duplicated. Individuals will benefit from design effects in different ways, with particular effect being noticed by low-knowledge or high-spatial learners. And finally, narration combined with animation supports understanding better than animation and text only, without narration.

It is of particular interest that much of the research examined by the Metiri Group (2008) focused on older students and higher order thinking, but it applies equally well to multimedia digital books for beginning readers. These readers need multiple inputs to support their efforts at making meaning from books. As Grimshaw, Dungworth and McKnight (2007) note, multimodal features such as narration, word pronunciation, sound effects and animation can help remove the effort from decoding individual words and help children focus on making meaning from texts. The question at hand is whether these these multimedia elements can really enhance understanding of stories or nonfiction, or whether they are essentially entertainment. How does reading these multimedia texts differ at its essence from reading traditional print books?

Considering my own perspective

I so appreciate the chance to connect with classmates and hear their own perspectives on the reading and learning we are doing. We are starting to converse more fluidly about our reading in our small group. But I am also reading other classmates' blogs. I particularly appreciated the perspective shared by Allyson B. in her blog post on MLIS Musings.

My class on research methods is challenging me quite a lot right now, with a heavy dose of social science research outside of my comfort zone. I am so rooted in my own particular perspective as a school teacher and school librarian, that I find it very difficult to imagine doing sociological or ethnographical research of the sort described in our reading. One researcher pretended to be a kindergartner for an extended period to try to understand the social dynamics of these young children. Another researcher is trying to understand the ways teens interact on internet chat sites, and so is lurking on sites, trying to be accepted as a teen.

Allyson talked about how researchers bring their own perspectives to research, and this can influence the way we see data around us. For example, Allyson is very excited about the prospect of convincing school leaders that students should be able to use their own personal devices to further their learning. But this enthusiasm might influence the way she goes about her research. As she notes, a thorough, well-balanced literature review can help the researcher approach the subject more neutrally. She was interested in an article that noted that increased expenditures in technology have not necessarily meant increased student performance, if they were also combined with decreased staffing or poor teaching.

I also am prone to suffering from similar enthusiasm with technology. I'm very excited by the opportunities for reading iPad book apps that combine many types of learning: print, audio, video, and interactive features. I love watching students explore these different types of book apps. But I need to think critically about what they bring to students' learning, and not just embrace them as a new shiny toy. I am hoping that my literature review will encompass studies of CD-ROMS and older forms of multimedia learning.

Bogie, A. (2012). "Research journal - reflecting on personal bias." MLIS Musings. Blog post on October 20, 2012. Retrieved from http://bogiemlismusings.blogspot.com/2012/10/research-journal-reflecting-on-personal.html

Thinking about my research question

I am wrestling with my research topic for my literature review and research proposal for my Research Methodologies class at SJSU. I'm fascinated by the way that iPad book apps combine many types of reading and learning. I would like to delve into understanding how children respond to digital reading, especially multimedia, multimodal types of reading. But this research is just beginning to emerge. So for my initial literature review, I've had to broaden the scope of my topic to investigating digital reading more broadly.

I'd like to keep in mind the key questions that Dr. Josepch Stetar (2009), of Seton Hall University, posed in his Slideshare "Developing Research Questions".

Criteria for identifying research problem:

1. Must be interesting enough to hold my interest 
2. Must be within my range of competence. 
3. Must be a manageable size 
4. Must have some basis in theory—ie, topics found in the lit review 
5. Must have the potential to make an original contribution—can’t be a duplicate of something that’s already been done. 
6. Must be based on obtainable data—research plan must be viable and practical. 
7. Must permit me to demonstrate my independent mastery of both subject and research method

Clearly, I have a long way to go. At the moment, I am interested in seeing how I would extend on the research started by the Cooney Foundation (Chiong, et al., 2012), but extending it to school-aged children. The Cooney Foundation research focuses on children ages 3-6 and how they read book apps with their parents. I'm interested in how children ages 6-10 read book apps, the ways that they absorb information, interact with the digital elements, and comprehend the story/information. I really want to know whether book apps appeal to children because they seem so much like games and movies, or if they are drawing children into a book-like experience.

I wonder if Slater would argue that my topic is still too broad, that I need to focus it down more to just one or two variables. I suppose the Cooney research from 2012 (Chiong, et al., 2012) did this by comparing comprehension from a print, ebook and enhanced book app for similar books. I can see doing this with the Magic School Bus: Oceans book, which has a print and excellent iPad book app - perfect for a 6 to 9 year old.

The dilemma that I'm facing is that there is very little research out there right now on this specific topic. So to meet the literature review element of my assignment (15 or more scholarly articles), I need to look more broadly at digital reading. One of the aspects I will examine is how other researchers have structured their investigations into students digital reading, say with Kindles.

I'm curious if theoretical articles that define and identify issues within this area are a part of the literature survey. They are not other studies, per se, but they lay an important foundation, especially in this developing field.

Ahh, much to ponder. But it's certainly holding my interest!

Chiong, C., Ree, J., Takeuchi, L. and Erickson, I. (2012). Comparing parent-child co-reading on print, basic and enhanced e-book platforms. The Joan Ganz Cooney Center. Spring 2012. Retrieved from http://joanganzcooneycenter.org/Reports-35.html.

Stetar, J. (2009). Developing Research Questions. Slideshare. Retrieved from http://www.slideshare.net/mikeparent/developing-research-question.

Toontastic: a pilot study examining creative play with an iPad app

My kids have loved playing with Toontastic, an iPad app that kids use to create their own animated stories. We only played with it in the early days of the app, and it's grown by leaps and bounds. I was very interested to read a new pilot study by Launchpad Learning, the company behind Toontastic. Andy Russell, one of the co-founders of Launchpad, and Alicia Chang, a Cognitive and Developmental Psychologist working with Launchpad Toys to research how kids are learning through cartoon creation, wrote about their study on the Joan Gantz Cooney Center Blog.

It's interesting to see the nonprofit Cooney Research Center is writing about a company's research, but I think it shows how seriously Launchpad Learning takes its focus on educational creative play in the digital age. Launchpad Learning developed a two-pronged study, both examining over two million cartoon created on Toontastic by over 600,000 kids, and examining semi-structured play sessions in a fourth grade classroom.

In the semi-structured play session, the fourth graders were divided into small groups of two or three students and asked to create a story about peer relationships over the course of one week. The study resulted in several observations, notably:

• Students used recurring characters more as the week progressed, suggesting that they developed a sense of cohesive story arcs and character development over time.
• As the students spent more time on their stories, they explored more emotional highs and lows, adding more depth and color to their stories.
• Students used more descriptive language and distinct character voices as time progressed.

All of these results reflect what I've observed with this app. It is not a quick and dirty app, but one that works best if kids have more time to develop their stories. If kids just have 30 or 40 minutes to work with it, they'll certainly have fun manipulating the puppets and recording their voices. But it's hard work making a cohesive story! It takes time, experimenting, and creative thinking.

Russell and Chang conclude by connecting the key components they've studied to the standards set out in the Common Core State Standards Initiative. I completely agree with that. I'd love to figure out a way to have our students be able to use this app. But a huge barrier in my experience is having the time for students to plan their stories, experiment with the app, and then craft their final products. Time is precious in a classroom, and there is never enough of it.

Russell, A. and Chang, A. (October 4, 2012). "Pilot study: Creative play with Toontastic." The Joan Gantz Cooney Center at Sesame Workshop, Blog. Retrieved from http://joanganzcooneycenter.org/Cooney-Center-Blog-272.html

For more information, see:

http://launchpadtoys.com/blog/2012/09/creative-learning-with-toontastic-at-home-and-in-the-classroom/
http://launchpadtoys.com/content/launchpad-leaders/

Action research - using a wiki to collaborate and gather information

I do find the idea of action research the most intriguing and appealing of the different approaches to research I've studied so far. I know that other qualitative and quantitative approaches are important, but I cannot imagine going down those routes myself.

Shannon Burger and Mary Ann McFarland, from Southwest Middle School,Parkway School District in suburban St. Louis, Missouri, had very interesting comments on their approach to action research. As Burger and McFarland (2009) write, "Action research can be used for the analysis and reflection of everyday problems in the practitioners specific situation versus traditional research, which aims to formulate general statements that are broadly applicable." As a teacher librarian, I think action research is part of my daily life on an informal basis. I'm always trying to put together my thoughts on what works and what doesn't. The problem is that I don't often have - or take - the time to write down my observations and solidify or extend my thinking on them. So these in-the-moment observations can get lost in the hustle-bustle of daily life.

Burger and McFarland describe the way they used their wiki to collaborate on their action research. The wiki gave them a shared space to record their observations, gather linked resources, and comment on each other's work. As they note, by sharing information on the wiki, they were then able to use their time in face-to-face meetings much more effectively to problem solve and brainstorm.

I have used PBworks, the wiki that Burger and McFarland used, for several projects both for work and for SJSU classes. But I had not thought of these wikis quite so formally as a way to gather and share data on an action research. This is a very interesting suggestion to keep in mind.

Burger, S. and McFarland, M.A. (2009). "Action research and wikis: an effective collaboration." Library Media Connection. 28(2): 38-40. Retrieved from http://libaccess.sjlibrary.org/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=44773682&site=ehost-live