Multimodal Text Development
Information Guide for Teachers
What Is a Multimodal Text?
A picture book
"...in which the textual and visual elements are arranged on individual pages that contribute to an overall set of bound pages." (What are multimodal texts? section, para. 2)
"...in which elements such as sound effects, oral language, written language, music, and still or moving images are combined." (What are multimodal texts? section, para. 2)
A live ballet performance
"...in which gesture, music, and space are the main elements." (What are multimodal texts? section, para. 2)
A picture book
Affordances & Constraints of Multimodal Texts
Jones and Hafner (2012) explained that affordances of multimodal texts are quite apparent. Audio and video may be embedded, unlike in traditional printed texts. Creativity is fostered with the ease of use of the various modes listed above and the abundant availability of online resources. The use of images in multimodal texts has a more direct effect on viewers because images are spatial/simultaneous meaning that all of the information in an image is seen by the viewer at the exact same moment with the spacing of the different elements all in relationship to one another. However, this same affordance can also be considered a constraint due to the image being able to relate several messages at the same time challenging the viewer to have to discern between the possible meanings of the image and integrate the information accordingly.
Designing the Visual Layout
Jones and Hafner (2012) discussed how designers of a multimodal text must make specific choices about the visual layout of their text including the size and location of elements. Most multimodal texts have similar structures and are divided into three general regions.
Given information, or information that the viewer is presumed to already know, is located at the left of the page, and new information is located at the right.
Factual or real information is found at the bottom of a page, while ideal information or information that is hoped for or strived for is found at the top.
The very center of multimodal text is reserved for the most important or dominant element of the page with other elements or information surrounding it in the margins.
The Power of the Image
Jones and Hafner (2012) further explained that in a multimodal text, images and text have a reciprocal, interactive relationship framing each other and providing contextualized information to help clarify the intended meaning of the message.
- Concurrence is where the message of the text and the image reinforce each other and share the same meaning.
- Complementary is where the message of the text or the image gives slightly different information. This new information helps to complete the details of the other mode clarifying the meaning. Images can complement a text by explaining the how and why of a text, by providing additional information, or by restating or specifying what is in the text.
- Divergence is where the mood and tone of an image differs from the mood and tone of the text. This technique is often meant to create a message of irony or humor.
The authors continued by sharing how images can be very powerful in eliciting strong emotions from the viewer and creating an interpersonal relationship greatly affecting the viewer’s attitude towards the subject matter. Photographers or creators of images employ certain techniques for their purpose.
- The first technique is involvement. They may engage or involve the viewer based on the subject’s gaze, the distance of the shot from the viewer, or the particular camera angle. Demand images have the subject looking directly at the viewer demanding their attention. An offer image has the subject looking away and makes a request more for reflection or thought.
- The second technique is power relations which specifically focuses on the angle of the camera. A shot taken from above makes the viewer feel more dominant, but a shot taken from below may make the subject of the image more powerful and dominant than the viewer.
- The third technique is modality which focuses on color saturation, contextualization, representation, depth, illumination, brightness, etc. which can all be used to create tone and mood.
Videos incorporate the same design principles as images listed above but with the addition of pacing, sound effects, and music.
Online Language and Social Interaction
A New Frontier
In the video “How Is the Internet Changing Language Today?” (2010), linguist David Crystal enumerated the various methods used to communicate electronically in the 21st century. Cell phones, email, instant messaging, social networking sites, webpages, and blogs are examples of the modern technology people use to keep in touch. He made the point that any time in history that there have been advances in technology, changes have been seen in the way that the English language is used. The same is true now as new forms of electronic communication become a part of daily routines and social culture. New styles of the English language are evolving for each new mode of communication. In the video Crystal stated, “Language has become expressively richer as a result of the internet.”
Keeping in Touch
Jones and Hafner (2012) shared the many affordances of synchronous and asynchronous interactions. These types of messages can be multilingual and multi-scriptural using a mix of languages and linguistic features that allow for creativity such as emoticons, shortened forms of words, acronyms, letter homophones, lexicalization of sounds, creative punctuation, and more. These digital communications allow for interaction between the person sending the message and the person receiving the message. These types of messages are extremely convenient and can be responded to in real time or at a later, more convenient time with less transaction cost of a face-to-face interaction. A person can keep it short and to the point, and there is no need for pleasantries or pretenses.
However, synchronous and asynchronous interactions have their constraints as well. The limited space or characters of many digital messages may be too restrictive to get one’s message across, or there may be a delay in the communication. Messages sent via digital communication can be misconstrued due to lack of media richness meaning facial expressions, gestures, and tone of voice all of which help convey emotion, purpose, and the overall context of the message.
On the other hand, a face-to-face interaction is quite different because it offers the full contextualization of the message with facial features, gestures, and tone of voice to help the receiver of the message interpret the sender’s purpose and intent. Although, many see the transaction cost of face-to-face interactions as being too high. One must make time for the interaction, exchange pleasantries, monitor facial expressions, gestures, emotions, and attentiveness.
For the Record
David Crystal - Linguist
“Some people dislike texting. Some are bemused by it. Some love it. I am fascinated by it, for it is the latest manifestation of the human ability to be linguistically creative and to adapt language to suit the demands of diverse settings. In texting, what we are seeing, in a small way, is language in evolution.” (Crystal, 2008, p. 11)
Andrea Lunsford - Stanford University
" 'I think we're in the midst of a literacy revolution the likes of which we haven't seen since Greek civilization,' she says. For Lunsford, technology isn't killing our ability to write. It's reviving it—and pushing our literacy in bold new directions." (Thompson, 2009, para. 3)
John Humphrys - BBC
"It is the relentless onward march of the texters, the SMS (Short Message Service) vandals who are doing to our language what Genghis Khan did to his neighbours eight hundred years ago. They are destroying it: pillaging our punctuation; savaging our sentences; raping our vocabulary. And they must be stopped. This, I grant you, is a tall order. The texters have many more arrows in their quiver than we who defend the old way." (Humphrys, 2007, para. 15-17)
Adolescents and Multimodal Texts
Digital Native or Digitally Deprived?
Just because a child has been born into the digital world does not automatically make him or her a digital native. In her article, “Shrek Meets Vygotsky: Rethinking Adolescents’ Multimodal Literacy Practices in Schools”, Mills (2010) explained the important role of socioeconomic class and ethnicity in a child’s digital literacy development. Mills shared research that supports the findings that children of middle class families have more exposure to the Internet and family digital tools; therefore, they have better online and digital skills than children of working class families or low-income families. Research supports the same is true of children of Caucasian families. Caucasian families ranked the highest in Internet usage surpassing Hispanics, African Americans, Native Americans, and Asians.
So, assuming a 21st century learner is inherently a digital native can be a gross misjudgment of a student’s digital skills. It is the same as assuming that a student from any English-speaking family will inherently have a vast, rich, formal vocabulary. Teachers know that the socioeconomic background of the family and often ethnicity play a huge role in the development of a student’s vocabulary and language skills. Mills’ article supports that the same is true with digital literacy skills.
Implications for Schools
Though students come with various backgrounds and skill levels, Mills (2010) explained that students are able to bring their own personal textual and multimodal experiences from their home and social worlds into the classroom and apply those experiences to new digital literacy activities and lessons. Schools must provide the expert resources that students need such as teachers, books, and technologies to help students become competent in the multimodal and digital literacy skills that they are not able to master on their own in their home and social worlds. Mills stated, “The goal of literacy education is to point youth in the right direction so that they can extend their current practices to a wider range of productive purposes.” (p.40)
Teaching in Action
Anstey, M., & Bull, G. (2010). Helping teachers to explore multimodal texts. Curriculum & Leadership Journal: An electronic journal for leaders in education 8, (16). Retrieved from http://www.curriculum.edu.au/leader/helping_teachers_to_explore_multimodal_texts,31522.html?issueID=12141
Churches, A. (2012). Bloom’s digital taxonomy. Retrieved from http://edorigami.wikispaces.com/Bloom%27s+Digital+Taxonomy
Crystal, D. (2010). How is the internet changing language today? [Video file]. Retrieved from http://www.youtube.com/watch?v=P2XVdDSJHqY
Crystal, D. (2008). Txting: frnd or foe? The Linguist, 47, (6), 8-11. Retrieved from http://w.davidcrystal.com/DC_articles/Internet16.pdf
Humphreys, J. (2007) I h8 txt msgs: How texting is wrecking our language. Mail Online. Retrieved from http://www.dailymail.co.uk/news/article-483511/I-h8-txt-msgs-How-texting-wrecking-language.html
Jones, R.H., & Hafner, C.A. (2012). Understanding digital literacies: A practical introduction. New York, NY: Routledge.
Mills, K. (2010). Shrek meets vygotsky: Rethinking adolescents’ multimodal literacy practices in schools. Journal of Adolescent & Adult Literacy, 54 (1), 35-45.
Scott DeWitt on Multimodality and Engagement [Video file]. Retrieved from http://www.youtube.com/watch?v=chLlxONi3ik
Smore. (n.d.). Retrieved from http//:www.smore.com
Thompson, C. (2009). Clive thompson on the new literacy. Wired Magazine. Retrieved from http://www.wired.com/techbiz/people/magazine/17-09/st_thompson