Picture or Text First? Explaining Sequence Effects when Learning with Pictures and Text

Educational Psychology Review

Volumn 27, Issue 1, 2015, Pages 153-180

  Eitel, Alexander ^a   Scheiter, Katharina ^a  

^a KNOWLEDGE MEDIA RESEARCH CENTER   (Germany)

Author keywords

Graphic organizers; Learning with text and pictures; Multimedia; Multiple external representations; Sequential presentation

Indexed keywords

EID: 84897358934     PISSN: 1040726X     EISSN: 1573336X     Source Type: Journal    
DOI: 10.1007/s10648-014-9264-4     Document Type: Review

References (109)

1. Ainsworth, S. (2006). DeFT: a conceptual framework for considering learning with multiple representations. Learning and Instruction, 16, 183–198. doi:10.1016/j.learninstruc.2006.03.001.
2. Ainsworth, S. E., & Loizou, A. T. (2003). The effects of self-explaining when learning with text or diagrams. Cognitive Science, 27, 669–681. doi:10.1016/S0364-0213(03)00033-8.
3. Ainsworth, S., Wood, D., & O’Malley, C. (1998). There is more than one way to solve a problem: evaluating a learning environment that supports the development of children’s multiplication skills. Learning and Instruction, 8, 141–157. doi:10.1016/S0959-4752(97)00013-3.
4. *Alvermann, D. E. (1981). The compensatory effect of graphic organizers on descriptive text. The Journal of Educational Research, 75, 44–48.
5. Anglin, G. J., Vaez, H., & Cunningham, K. L. (2004). Visual representations and learning: the role of static and animated graphics. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 865–916). Mahwah, NJ: Lawrence Erlbaum.
6. Antes, J. R., & Kristjanson, A. F. (1991). Discriminating artists from nonartists by their eye-fixation patterns. Perceptual and Motor Skills, 73, 893–894. doi:10.2466/pms.1991.73.3.893.
7. Baddeley, A. D., & Hitch, G. (1993). The recency effect: implicit learning with explicit retrieval? Memory & Cognition, 21, 146–155. doi:10.3758/BF03202726.
8. *Baggett, P. (1984). Role of temporal overlap of visual and auditory material in forming dual media associations. Journal of Educational Psychology, 76, 408. doi:10.1037/0022-0663.76.3.408
9. Bétrancourt, M. (2005). The animation and interactivity principles in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 287–296). Cambridge, MA: Cambridge University Press.
10. *Borges, M. A., & Robins, S. L. (1980). Contextual and motivational cue effects on the comprehension and recall of prose. Psychological Reports, 47, 263–268. doi:10.2466/pr0.1980.47.1.263
11. *Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11, 717–726. doi:10.1016/S0022-5371(72)80006-9
12. Butcher, K. R. (2006). Learning from text with diagrams: promoting mental model development and inference generation. Journal of Educational Psychology, 98, 182–197. doi:10.1037/0022-0663.98.1.182.
13. *Canham, M., & Hegarty, M. (2010). Effects of knowledge and display design on comprehension of complex graphics. Learning and Instruction, 20, 155–166. doi:10.1016/j.learninstruc.2009.02.014
14. Carney, R. N., & Levin, J. R. (2002). Pictorial illustrations still improve students’ learning from text. Educational Psychology Review, 14, 5–26. doi:10.1023/A:1013176309260.
15. Charness, N., Reingold, E. M., Pomplun, M., & Stampe, D. M. (2001). The perceptual aspect of skilled performance in chess: evidence from eye movements. Memory and Cognition, 29, 1146–1152. doi:10.3758/BF03206384.
16. Chi, M. T. H. (2000). Self-explaining expository texts: the dual process of generating inferences and repairing mental models. In R. Glaser (Ed.), Advances in instructional psychology (pp. 161–238). Mahwah, NJ: Lawrence Erlbaum.
17. Chin, J. M., & Schooler, J. W. (2008). Why do words hurt? Content, process, and criterion shift accounts of verbal overshadowing. European Journal of Cognitive Psychology, 20, 396–413. doi:10.1080/09541440701728623.
18. Craik, F. I. M., & Lockhart, S. E. (1972). Levels of processing: a framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671–684. doi:10.1016/S0022-5371(72)80001-X.
19. De Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2009). Towards a framework for attention cueing in instructional animations: guidelines for research and design. Educational Psychology Review, 21, 113–140.
20. *Dean, R. S., & Enemoh, P. A. C. (1983). Pictorial organization in prose learning. Contemporary Educational Psychology, 8, 20–27. doi:10.1016/0361-476X(83)90031-0
21. *Dean, R. S., & Kulhavy, R. W. (1981). Influence of spatial organization in prose learning. Journal of Educational Psychology, 73, 57–64. doi:10.1037/0022-0663.73.1.57
22. Dodson, C. S., Johnson, M. K., & Schooler, J. W. (1997). The verbal overshadowing effect: why descriptions impair face recognition. Memory and Cognition, 25, 129–139. doi:10.3758/BF03201107.
23. *Eitel, A., Scheiter, K., & Schüler, A. (2013a). How inspecting a picture affects processing of text in multimedia learning. Applied Cognitive Psychology, 27, 451–461. doi:10.1002/acp.2922
24. *Eitel, A., Scheiter, K., Schüler, A., Nyström, M., & Holmqvist, K. (2013b). How a picture facilitates the process of learning from text: Evidence for scaffolding. Learning and Instruction, 28, 48–63. doi:10.1016/j.learninstruc.2013.05.002
25. Fletcher, J. D., & Tobias, S. (2005). The multimedia principle. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 117–133). Cambridge, MA: Cambridge University Press.
26. Folker, S., Ritter, H., & Sichelschmidt, L. (2005). Processing and integrating multimodal material: the influence of color-coding. In B. G. Bara, L. Barsalou, & M. Bucciarelli (Eds.), Proceedings of the 27th annual conference of the cognitive science society (pp. 690–695). Mahwah, NJ: Erlbaum.
27. Gernsbacher, M. A., Varner, K. R., & Faust, M. (1990). Investigating differences in general comprehension skill. Journal of Experimental Psychology. Learning, Memory, and Cognition, 16, 430–445. doi:10.1037/0278-7393.16.3.430.
28. Ginns, P. (2006). Integrating information: a meta-analysis of the spatial contiguity and temporal contiguity effects. Learning and Instruction, 16, 511–525. doi:10.1016/j.learninstruc.2006.10.001.
29. Glenberg, A. M., & Langston, W. E. (1992). Comprehension of illustrated text: pictures help to build mental models. Journal of Memory and Language, 31, 129–151. doi:10.1016/0749-596X(92)90008-L.
30. Gyselinck, V., & Tardieu, H. (1999). The role of illustrations in text comprehension: what, when, for whom, and why? In H. van Oostendorp & S. Goldman (Eds.), The construction of mental representations during reading (pp. 195–218). Mahwah, NJ: Lawrence Erlbaum.
31. Gyselinck, V., Jamet, E., & Dubois, V. (2008). The role of working memory components in multimedia comprehension. Applied Cognitive Psychology, 22, 353–374. doi:10.1002/acp.1411.
32. Haider, H., & Frensch, P. A. (1996). The role of information reduction in skill acquisition. Cognitive Psychology, 30, 304–337. doi:10.1006/cogp.1996.0009.
33. Hegarty, M. (2011). The cognitive science of visual-spatial displays: implications for design. Topics in Cognitive Science, 3, 446–474. doi:10.1111/j.1756-8765.2011.01150.x.
34. Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742. doi:10.1006/jmla.1993.1036.
35. Hegarty, M., Kriz, S., & Cate, C. (2003). The roles of mental animations and external animations in understanding mechanical systems. Cognition and Instruction, 21, 325–360. doi:10.1207/s1532690xci2104_1.
36. Henderson, J. M. (2003). Human gaze control during real-world scene perception. Trends in Cognitive Sciences, 7, 498–504. doi:10.1016/j.tics.2003.09.006.
37. *Huff, M., & Schwan, S. (2008). Verbalizing events: overshadowing or facilitation?. Memory and Cognition, 36, 392–402. doi:10.3758/MC.36.2.392
38. *Huff, M., & Schwan, S. (2012). The verbal facilitation effect in learning to tie nautical knots. Learning and Instruction, 22, 376–385. doi:10.1016/j.learninstruc.2012.03.001
39. Hyönä, J. (2010). The use of eye movements in the study of multimedia learning. Learning and Instruction, 20, 172–176. doi:10.1016/j.learninstruc.2009.02.013.
40. Jarodzka, H., Scheiter, K., Gerjets, P., & Van Gog, T. (2010). In the eyes of the beholder: how experts and novices interpret dynamic stimuli. Learning and Instruction, 20, 146–154. doi:10.1016/j.learninstruc.2009.02.019.
41. Johnson-Laird, P. N. (1980). Mental models in cognitive science. Cognitive Science, 4, 71–115. doi:10.1016/S0364-0213(81)80005-5.
42. Just, M. A., & Carpenter, P. A. (1980). A theory of reading: from eye fixations to comprehension. Psychological Review, 87, 329–354. doi:10.1037//0033-295X.87.4.329.
43. *Kauffman, D. F., & Kiewra, K. A. (2010). What makes a matrix so effective? An empirical test of the relative benefits of signaling, extraction, and localization. Instructional Science, 38, 679–705. doi:10.1007/s11251-009-9095-8
44. *Kiewra, K. A., Kauffman, D. F., Robinson, D. H., Dubois, N. F., & Staley, R. K. (1999). Supplementing floundering text with adjunct displays. Instructional Science, 27, 373–401. doi:10.1023/A:1003270723360
45. Kulhavy, R. W., Stock, W. A., & Kealy, W. A. (1993). How geographic maps increase recall of instructional text. Educational Technology Research and Development, 41, 47–62. doi:10.1007/BF02297511.
46. Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–99. doi:10.1016/S0364-0213(87)80026-5.
47. Lesgold, A., Rubinson, H., Feltovich, P., Glaser, R., Klopfer, D., & Wang, Y. (1988). Expertise in a complex skill: diagnosing x-ray pictures. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 311–342). Hillsdale, NJ, England: Lawrence Erlbaum.
48. Levie, W. H., & Lentz, R. (1982). Effects of text illustrations: a review of research. Educational Communication and Technology, 30, 195–232.
49. Levin, J. R., Anglin, G. J., & Carney, R. N. (1987). On empirically validating functions of pictures in prose. In D. M. Willows & H. A. Houghton (Eds.), The psychology of illustration (Basic research, Vol. 1, pp. 51–85). New York: Springer.
50. Lowe, R. K. (1993). Constructing a mental representation from an abstract technical diagram. Learning and Instruction, 3, 157–179. doi:10.1016/0959-4752(93)90002-H.
51. Lowe, R. K. (1994). Selectivity in diagrams: reading beyond the lines. Educational Psychology, 14, 467–491. doi:10.1080/0144341940140408.
52. Lowe, R. K. (1996). Background knowledge and the construction of a situational representation from a diagram. European Journal of Psychology of Education, 11, 377–397. doi:10.1007/BF03173279.
53. Lowe, R. K. (2004). Interrogation of a dynamic visualization during learning. Learning and Instruction, 14, 257–274. doi:10.1016/j.learninstruc.2004.06.003.
54. Mautone, P. D., & Mayer, R. E. (2001). Signaling as a cognitive guide in multimedia learning. Journal of Educational Psychology, 93, 377–389. doi:10.1037/0022-0663.93.2.377.
55. Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge: Cambridge University Press.
56. Mayer, R. E., & Anderson, R. B. (1991). Animations need narrations: an experimental test of a dual-coding hypothesis. Journal of Educational Psychology, 83, 484–490. doi:10.1037/0022-0663.83.4.484.
57. Mayer, R. E., & Anderson, R. B. (1992). The instructive animation: helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84, 444–452. doi:10.1037/0022-0663.84.4.444.
58. Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86, 389–401. doi:10.1037/0022-0663.86.3.389.
59. Mayer, R. E., Moreno, R., Boire, M., & Vagge, S. (1999). Maximizing constructivist learning from multimedia communications by minimizing cognitive load. Journal of Educational Psychology, 91, 638–643. doi:10.1037/0022-0663.91.4.638.
60. *McCrudden, M. T., Schraw, G., & Lehman, S. (2009). The use of adjunct displays to facilitate comprehension of causal relationships in expository text. Instructional Science, 37, 65–86. doi:10.1007/s11251-007-9036-3
61. *McCrudden, M. T., Magliano, J. P., & Schraw, G. (2011). The effect of diagrams on online reading processes and memory. Discourse Processes, 48, 69–92. doi:10.1080/01638531003694561
62. McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1–43. doi:10.1207/s1532690xci1401_1.
63. Meissner, C. A., & Brigham, J. C. (2001). A meta‐analysis of the verbal overshadowing effect in face identification. Applied Cognitive Psychology, 15, 603–616. doi:10.1002/acp.728.
64. Meissner, C. A., Sporer, S. L., & Susa, K. J. (2008). A theoretical review and meta-analysis of the description-identification relationship in memory for faces. European Journal of Cognitive Psychology, 20, 414–455. doi:10.1080/09541440701728581.
65. Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81–97. doi:10.1037/h0043158.
66. Murdock, B. B., Jr. (1962). The serial position effect of free recall. Journal of Experimental Psychology, 64, 482–488. doi:10.1037/h0045106.
67. *O’Keefe, E. J., & Solman, R. T. (1987). The influence of illustrations on children's comprehension of written stories. Journal of Literacy Research, 19, 353–377. doi:10.1080/10862968709547611
68. Ozcelik, E., Arslan-Ari, I., & Cagiltay, K. (2010). Why does signaling enhance multimedia learning? Evidence from eye movements. Computers in Human Behavior, 26, 110–117. doi:10.1016/j.chb.2009.09.001.
69. Paivio, A. (1986). Mental representations: a dual coding approach. Oxford, UK: Oxford University Press.
70. *Peverly, S. (1981). The effects of diagrams-before text vs. diagrams-after text in the processing of novel text information. Retrieved from ERIC database. (ED212995)
71. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372–422. doi:10.1037/0033-2909.124.3.372.
72. Rayner, K. (2009). Eye movements and attention in reading, scene perception, and visual search. The Quarterly Journal of Experimental Psychology, 62, 1457–1506. doi:10.1080/17470210902816461.
73. Reigeluth, C. M., Merrill, M. D., Wilson, B. G., & Spiller, R. T. (1980). The elaboration theory of instruction: a model for sequencing and synthesizing instruction. Instructional Science, 9, 195–219. doi:10.1007/BF00177327.
74. Robinson, D. H. (1998). Graphic organizers as aids to text learning. Reading Research and Instruction, 37, 85–105. doi:10.1080/19388079809558257.
75. *Robinson, D. H., & Kiewra, K. A. (1995). Visual argument: Graphic organizers are superior to outlines in improving learning from text. Journal of Educational Psychology, 87, 455–467. doi:10.1037/0022-0663.87.3.455
76. *Robinson, D. H., & Schraw, G. (1994). Computational efficiency through visual argument: Do graphic organizers communicate relations in text too effectively? Contemporary Educational Psychology, 19, 399–415. doi:10.1006/ceps.1994.1029
77. *Robinson, D. H., Katayama, A. D., Dubois, N. F., & Devaney, T. (1998). Interactive effects of graphic organizers and delayed review on concept application. The Journal of Experimental Education, 67, 17–31. doi:10.1080/00220979809598342
78. *Robinson, D. H., Corliss, S. B., Bush, A. M., Bera, S. J., & Tomberlin, T. (2003). Optimal presentation of graphic organizers and text: a case for large bites?. Educational Technology Research and Development, 51, 25–41. doi:10.1007/BF02504542
79. Rummer, R., Schweppe, J., Fürstenberg, A., Scheiter, K., & Zindler, A. (2011). The perceptual basis of the modality effect in multimedia learning. Journal of Experimental Psychology: Applied, 17, 159–173. doi:10.1037/a0023588.
80. *Salmerón, L., Baccino, T., Cañas, J. J., Madrid, R. I., & Fajardo, I. (2009). Do graphical overviews facilitate or hinder comprehension in hypertext?. Computers and Education, 53, 1308–1319. doi:10.1016/j.compedu.2009.06.013
81. Salomon, G. (1984). Television is “easy” and print is “tough”: the differential investment of mental effort in learning as a function of perceptions and attributions. Journal of Educational Psychology, 76, 647–658. doi:10.1037/0022-0663.76.4.647.
82. Scaife, M., & Rogers, Y. (1996). External cognition: how do graphical representations work? International Journal of Human-Computer Studies, 45, 185–213. doi:10.1006/ijhc.1996.0048.
83. Scheiter, K., & Eitel, A. (2010). The effects of signals on learning from text and diagrams: how looking at diagrams earlier and more frequently improves understanding. In A. K. Goel, M. Jamnik, & N. H. Narayanan (Eds.), Diagrammatic representation and inference–6th International Conference, Diagrams 2010 (LNAI 6170 (pp. 264–270). Heidelberg: Springer.
84. Schmidt-Weigand, F., Kohnert, A., & Glowalla, U. (2010a). A closer look at split visual attention in system- and self-paced instruction in multimedia learning. Learning and Instruction, 20, 100–110. doi:10.1016/j.learninstruc.2009.02.011.
85. Schmidt-Weigand, F., Kohnert, A., & Glowalla, U. (2010b). Explaining the modality and contiguity effects: new insights from investigating students’ viewing behaviour. Applied Cognitive Psychology, 24, 226–237. doi:10.1002/acp.1554.
86. Schnotz, W. (2002). Towards an integrated view of learning from text and visual displays. Educational Psychology Review, 14, 101–120. doi:10.1023/A:1013136727916.
87. Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 49–69). Cambridge, MA: Cambridge University Press.
88. Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representations. Learning and Instruction, 13, 141–156. doi:10.1016/S0959-4752(02)00017-8.
89. Schnotz, W., & Kürschner, C. (2008). External and internal representations in the acquisition and use of knowledge: visualization effects on mental model construction. Instructional Science, 36, 175–190. doi:10.1007/s11251-007-9029-2.
90. Schooler, J. W. (2002). Verbalization produces a transfer inappropriate processing shift. Applied Cognitive Psychology, 16, 989–997. doi:10.1002/acp.930.
91. Schwonke, R., Berthold, K., & Renkl, A. (2009). How multiple external representations are used and how they can be made more useful. Applied Cognitive Psychology, 23, 1227–1243. doi:10.1037/a0013247.
92. Shah, P., Freedman, E. G., & Vekiri, I. (2005). The comprehension of quantitative information in graphical displays. In P. Shah & A. Miyake (Eds.), The Cambridge handbook of visuospatial thinking (pp. 426–476). New York, NY: Cambridge University Press.
93. *Shaw, S., Nihalani, P., Mayrath, M., & Robinson, D. H. (2012). Graphic organizers or graphic overviews? Presentation order effects with computer-based text. Educational Technology Research and Development, 60, 807–820. doi:10.1007/s11423-012-9257-2
94. *Simmons, D. C., Griffin, C. C., & Kameenui, E. J. (1988). Effects of teacher-constructed pre-and post-graphic organizer instruction on sixth-grade science students’ comprehension and recall. The Journal of Educational Research, 82, 15–21
95. *Snouffer, N. K., & Thistlethwaite, L. L. (1980). The effects of the structured overview and vocabulary pre-teaching upon comprehension levels of college freshmen reading physical science and history materials. Journal of the Association for the Study of Perception, 15, 11–16
96. *Stalbovs, K., Eitel, A., & Scheiter, K. (2013, August). Which cognitive processes predict successful learning with multimedia? Paper presented at the Fifteenth Biennial Conference of the European Association for Research on Learning and Instruction (EARLI), Munich, Germany
97. Stenning, K., & Oberlander, J. (1995). A cognitive theory of graphical and linguistic reasoning: logic and implementation. Cognitive Science, 19, 97–140. doi:10.1016/0364-0213(95)90005-5.
98. Sweller, J., van Merriënboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296. doi:10.1023/A:1022193728205.
99. *Ullrich, M. (2011). Einflüsse der Verarbeitungsreihenfolge auf den Wissenserwerb mit Texten und Bildern [Influences of processing order on knowledge acquisition from text and images] (Doctoral dissertation, University of Koblenz-Landau). Retrieved from http://kola.opus.hbz-nrw.de/volltexte/2011/631/
100. Van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic.
101. Van Gog, T., & Scheiter, K. (2010). Eye tracking as a tool to study and enhance multimedia learning. Learning and Instruction, 20, 95–99. doi:10.1016/j.learninstruc.2009.02.009.
102. Van Gog, T., Jarodzka, H., Scheiter, K., Gerjets, P., & Paas, F. (2009a). Attention guidance during example study via the model’s eye movements. Computers in Human Behavior, 25, 785–791. doi:10.1016/j.chb.2009.02.007.
103. Van Gog, T., Kester, L., Nievelstein, F., Giesbers, B., & Paas, F. (2009b). Uncovering cognitive processes: different techniques that can contribute to cognitive load research and instruction. Computers in Human Behavior, 25, 325–331. doi:10.1016/j.chb.2008.12.021.
104. Vekiri, I. (2002). What is the value of graphical displays in learning? Educational Psychology Review, 14, 261–312. doi:10.1023/A:1016064429161.
105. Verdi, M. P., & Kulhavy, R. W. (2002). Learning with maps and texts: an overview. Educational Psychology Review, 14, 27–46. doi:10.1023/A:1013128426099.
106. *Verdi, M. P., Kulhavy, R. W., Stock, W. A., Rittschof, K. A., & Johnson, J. T. (1996). Text learning using scientific diagrams: Implications for classroom use. Contemporary Educational Psychology, 21, 487–499. doi:10.1006/ceps.1996.0033
107. *Verdi, M. P., Johnson, J. T., Stock, W. A., Kulhavy, R. W., & Whitman-Ahern, P. (1997). Organized spatial displays and texts: Effects of presentation order and display type on learning outcomes. The Journal of Experimental Education, 65, 303–317
108. Weidenmann, B., Paechter, M., & Hartmannsgruber, K. (1999). Structuring and sequencing of complex text-picture combinations. European Journal of Psychology of Education, 14, 185–202. doi:10.1007/BF03172965.
109. Zwaan, R. A., & Radvansky, G. A. (1998). Situation models in language comprehension and memory. Psychological Bulletin, 123, 162–185. doi:10.1037/0033-2909.123.2.162.