Does the Modality Effect Exist? and if So, Which Modality Effect?

Journal of Psycholinguistic Research

Volumn 41, Issue 1, 2012, Pages 1-32

  Reinwein, Joachim ^a  

^a UNIVERSITÉ DU QUÉBEC À MONTRÉAL   (Canada)

Author keywords

Cognitive load theory; Illustrated text; Meta analysis; Modality effect; Working memory

Indexed keywords

ARTICLE; EPIDEMIOLOGY; HUMAN; LEARNING; META ANALYSIS; METHODOLOGY; MULTIMEDIA; PHYSIOLOGY; PUBLISHING; READING; SHORT TERM MEMORY; SPEECH PERCEPTION; VISION;

EFFECT MODIFIER, EPIDEMIOLOGIC; HUMANS; LEARNING; MEMORY, SHORT-TERM; MULTIMEDIA; PUBLICATION BIAS; READING; RESEARCH DESIGN; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 84855935332     PISSN: 00906905     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10936-011-9180-4     Document Type: Article

References (84)

1. Abel R. R., Kulhavy R. W. (1986) Maps, mode of text presentation, and children's prose learning. American Educational Research Journal 23(2): 263-274.
2. Altmann G. T. M., Kamide Y. (1999) Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition 73: 247-264.
3. Altmann G. T. M., Kamide Y. (2007) The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing. Journal of Memory and Language 57(4): 502-518.
4. Atkinson R. K. (2002) Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology 94: 416-427.
5. Baddeley A. D. (1986) Working memory. Oxford University Press, Oxford.
6. Baddeley A. D. (1992) Working memory. Science 255: 556-559.
7. Borenstein M., Hedges L. V., Higgins J. P. T., Rothstein H. R. (2009) Introduction to meta-analysis. Wiley, UK.
8. Brünken R., Leutner D. (2001) Aufmerksamkeitsverteilung oder Aufmerksamkeitsfokussierung? Empirische Ergebnisse zur "Split-Attention-Hypothese" beim Lernen mit Multimedia. Unterrichtswissenschaft 29: 357-366.
9. Brünken R., Plass J. L., Leutner D. (2003) Direct measurement of cognitive load in multimedia learning. Educational Psychologist 38: 53-61.
10. Brünken R., Plass J. L., Leutner D. (2004) Assessment of cognitive load in multimedia learning with dual-task methodology: Auditory load and modality effects. Instructional Science 32: 115-132.
11. Brünken R., Steinbacher S., Plass J. L., Leutner D. (2002) Assessment of cognitive load in multimedia learning using dual-task methodology. Experimental Psychology 49: 109-119.
12. Carney R. N., Levin J. R. (2002) Pictorial illustrations still improve students' learning from text. Educational Psychology Review 14(1): 5-26.
13. Chandler P., Sweller J. (1991) Cognitive load theory and the format of instruction. Cognition and Instruction 8: 293-332.
14. Chung, W. (2006). The effects of presentation pace and modality on learning a multimedia science lesson. Dissertation submitted to the Faculty of the Graduate School of the University of Texas at Austin (UMI Microform 3221810, ProQuest Information and Learning Company).
15. Craig S. D., Gholson B., Driscoll D. M. (2002) Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology 94(2): 428-434.
16. Douglas, E. S. (2007). Modality effects on adult learning. Dissertation submitted to the Department of Educational Psychology of the University of Nevada, Las Vegas (UMI Microform 3261073, ProQuest Information and Learning Company).
17. Duis S. S., Dean R. S., Derks P. (1994) The modality effect: A result of methodology?. International Journal of Neuroscience 78: 1-7.
18. Dunsworth Q., Atkinson R. K. (2007) Fostering multimedia learning of science: Exploring the role of an animated agent's image. Computers and Education 49: 677-690.
19. Ginns P. (2005) Meta-analysis of the modality effect. Learning and Instruction 4: 313-331.
20. Glass G. V., McGaw B., Smith M. L. (1981) Meta-analysis in social research. Sage, Beverly Hills.
21. Gyselinck V., Jamet E., Dubois V. (2008) The role of working memory components in multimedia comprehension. Applied Cognitive Psychology 22: 353-374.
22. Hannus M., Hyönä J. (1999) Utilization of illustrations during learning of science textbook passages among low- and high-ability children. Contemporary Educational Psychology 24: 95-123.
23. Hedges L. (1981) Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics 6: 107-128.
24. Higgins J., Thompson S. G., Deeks J. J., Altman D. G. (2003) Measuring inconsistency in meta-analyses. BMJ 327: 557-560.
25. Höffler T. N., Leutner D. (2007) Instructional animation versus static pictures: A meta-analysis. Learning and Instruction 17: 722-738.
26. Jeung H.-J., Chandler P., Sweller J. (1997) The role of visual indicators in dual sensory mode instruction. Educational Psychology 17(3): 329-343.
27. Jones D. M., Macken W. J., Nicholls A. P. (2004) The phonological store of working memory: Is it phonological and is it a store?. Journal of Experimental Psychology: Learning, Memory, and Cognition 30(3): 656-674.
28. Joseph, J. H. (1978). The instructional effectiveness of integrating abstract and realistic visualization. Thesis submitted to the Graduate School, Division of Curriculum and Instruction, of the Pennsylvania State University.
29. Kalyuga S. (2005) Prior knowledge principle in multimedia learning. In: Mayer R. E. (ed.) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 325-337.
30. Kalyuga S., Ayres P., Chandler P., Sweller J. (2003) The expertise reversal effect. Educational Psychologist 38: 23-31.
31. Kalyuga S., Chandler P., Sweller J. (1999) Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology 13: 351-371.
32. Kalyuga S., Chandler P., Sweller J. (2000) Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology 92(1): 126-136.
33. Kalyuga S., Chandler P., Tuovinen J., Sweller J. (2001) When problem solving is superior to studying worked examples. Journal of Educational Psychology 93: 579-588.
34. Kamide Y., Altmann G. T. M., Haywood S. L. (2003) The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language 49: 133-159.
35. Knoeferle P., Crocker M. W. (2006) The coordinated interplay of scene, utterance, and world knowledge: Evidence from eye tracking. Cognitive Science 30: 481-529.
36. Leahy W., Chandler P., Sweller J. (2003) When auditory presentations should and should not be a component of multimedia instruction. Applied Cognitive Psychology 17: 401-418.
37. Levin J. R., Divine-Hawkins P. (1974) Visual imagery as a prose-learning process. Journal of Reading Behavior 6(1): 23-30.
38. Lipsey M. W. (2003) Those confounded moderators in meta-analysis: Good, bad, and ugly. The Annals of the American Political and Social Science 587(1): 69-81.
39. Low R., Sweller J. (2005) The modality principle in multimedia learning. In: Mayer R. E. (ed.) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 147-158.
40. Main R. E., Griffiths B. (1977) Evaluation of audio and pictorial instructional supplements. AV Communication Review 25(2): 167-179.
41. Mann B. L. (1997) Evaluation of presentation modalities in a hypermedia system. Computers & Education 28(2): 133-143.
42. Mann B., Newhouse P., Pagram J., Campbell A., Schulz H. (2002) A comparison of temporal speech and text cueing in educational multimedia. Journal of Computer Assisted Learning 18: 296-308.
43. Mayer, R. E. (ed.) (2005a) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge.
44. Mayer R. E. (2005b) Principles for managing essential processing in multimedia learning: Segmenting, pretraining, and modality principles. In: Mayer R. E. (ed.) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 169-182.
45. Mayer R. E. (2005) Cognitive theory of multimedia learning. In: Mayer R. E. (ed.) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 31-48.
46. Mayer R. E., Bove W., Bryman A., Mars R., Tapangco L. (1996) When less is more: Meaningful learning from visual and verbal summaries of science textbook lessons. Journal of Educational Psychology 88(1): 64-73.
47. Mayer R. E., Dow G. T., Mayer S. (2003) Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based microworlds?. Journal of Educational Psychology 95(4): 806-813.
48. Mayer R. E., Moreno R. (1998) A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology 90(2): 312-320.
49. Mayer R. E., Moreno R. (2002) Animation as an aid to multimedia learning. Educational Psychology Review 14(1): 87-99.
50. Mayer R. E., Steinhoff K., Bower G., Mars R. (1995) A generative theory of textbook design: Using annotated illustrations to foster meaningful learning of science text. Educational Technology Research and Development 43: 31-43.
51. McNeill, A. L. (2004). The effects of training, modality, and redundancy on the development of a historical inquiry strategy in a multimedia learning environment. Dissertation submitted to the Faculty of Virginia Polytechnic Institute and State University (UMI Microform 3142239, ProQuest Information and Learning Company).
52. Mein, N. N. (2005). Impact of audio text, visual text and cueing on cognitive load and performance. Graduate School of Wayne State University, Detroit, Michigan (UMI Microform 3166768, ProQuest Information and Learning Company).
53. Moreno R. (2006) Does the modality principle hold for different media?: A test of the method-affects-learning hypothesis. Journal of Computer Assisted Learning 22(3): 149-158.
54. Moreno R., Mayer R. E. (1999) Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology 91(2): 358-368.
55. Moreno R., Mayer R. (2002) Learning science in virtual reality multimedia environments: Role of methods and media. Journal of Education Psychology 94: 598-610.
56. Moreno R., Mayer R., Spires H. A., Lester J. C. (2001) The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents?. Cognition and Instruction 19: 177-213.
57. Mousavi S. Y., Low R., Sweller J. (1995) Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology 87(2): 319-334.
58. Nugent G. C. (1982) Pictures, audio, and print: Symbolic representation and effect on learning. Educational Communication and Technology Journal 30(3): 163-174.
59. O'Neil H. F., Mayer R. E., Herl H., Niemi C., Olin K., Thurman R. A. (2000) Instructional strategies for virtual aviation training environments. In: O'Neil H. F., Andrews D. H. (eds) Aircrew training and assessment. Erlbaum, Mahwah, pp 105-130.
60. Paivio A. (1986) Mental representations: A dual-coding approach. Oxford University Press, New York.
61. Penney C. G. (1989) Modality effects and the structure of short-term memory. Memory and Cognition 17: 398-442.
62. Rayner K., Rotello C. M., Stewart A. J., Keir J., Duffy S. A. (2001) Integrating text and pictorial information: Eye movements when looking at print advertisements. Journal of Experimental Psychology: Applied 7: 219-226.
63. Rickheit G., Strohner H., Müsseler J., Nattkemper D. (1987) Recalling oral and written discourse. Journal of Educational Psychology 97: 438-444.
64. Sadoski M., Paivio A. (2001) Imagery and text-a dual coding theory of reading and writing. Erlbaum, Mahwah.
65. Schneider V. I., Healy A. F., Barshi I. (2004) Effects of instruction modality and readback on accuracy in following navigation commands. Journal of Experimental Psychology: Applied 10(4): 245-257.
66. Schnotz W. (2005) An integrated model of text and picture comprehension. In: Mayer R. E. (ed.) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 49-69.
67. Schnotz W., Kürschner C. (2007) A reconsideration of cognitive load theory. Educational Psychological Review 19: 469-508.
68. Segers E., Verhoeven L., Hulstijn-Hendrikse H. (2008) Cognitive processes in children's multimedia text learning. Applied Cognitive Psychology 22: 375-387.
69. Seufert T., Schütze M., Brünken R. (2009) Memory characteristics and modality in multimedia learning: An aptitude-treatment-interaction study. Learning and Instruction 19(1): 28-42.
70. Sewell E. H. Jr, Moore R. L. (1980) Cartoon embellishments in informative presentations. Educational Communication and Technology Journal 28(1): 39-46.
71. Stromfors, C. M. (2005). The impact of modality and working memory capacity on achievement in a multimedia environment. Dissertation submitted to the Arizona State University (UMI Microform 3178274, ProQuest Information and Learning Company).
72. Sweller J. (2005a) The redundancy principle in multimedia learning. In: Mayer R. E. (ed.) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 159-167.
73. Sweller, J. (2005b). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.), The Cambridge 1009 handbook of multimedia learning (pp. 19-30). Cambridge: Cambridge University Press.
74. Tabbers, H. K. (2002). The modality of text in multimedia instructions: refining the design guidelines. Unpublished doctoral dissertation, Educational Technology Expertise Centre, Open University of the Netherlands, Heerlen. http://www. ou. nl/Docs/Expertise/OTEC/Publicaties/huib%20tabbers/doctoral%20dissertation%20Huib%20Tabbers%20-%20web%20version. pdf. Accessed 03/25/2009.
75. Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2000). Multimedia instructions and cognitive load theory: split-attention and modality effects. Paper presented at the National Convention of the Association for Educational Communications and Technology, Long Beach, CA.
76. Tabbers H. K., Martens R. L., van Merriënboer J. J. G. (2001) The modality effect in multimedia instructions. In: Moore J. D., Stenning K. (eds) Proceedings of the twenty-third annual conference of the Cognitive Science Society. Lawrence Erlbaum Associates, Mahwah, pp 1024-1029.
77. Tabbers H. K., Martens R. L., van Merriënboer J. J. G. (2004) Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology 74: 71-81.
78. Tindall-Ford S., Chandler P., Sweller J. (1997) When two sensory modes are better than one. Journal of Experimental Psychology: Applied 3: 257-287.
79. Tindall-Ford S., Sweller J. (2006) Altering the modality of instructions to facilitate imagination: Interactions between the modality and imagination effects. Instructional Science 34: 343-365.
80. Underwood G., Jebbet L., Roberts K. T. (2004) Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search. Quarterly Journal of Experimental Psychology 57: 165-182.
81. van Merriënboer J. J. G. (1997) Training complex cognitive skills: A four-component instructional design model for technical training. Educational Technology Publications, Englewood Cliffs, NJ.
82. van Merriënboer J. J. G., Kester L. (2005) The four-component instructional design model: Multimedia principles in environments for complex learning. In: Mayer R. E. (ed.) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 71-93.
83. van Gerven, P. W. M. (2002). Efficient complex skills training into old age: Exploring the benefits of cognitive load theory. Dissertation submitted to the Open University of the Netherlands, Heerlen. http://www. personeel. unimaas. nl/p. vangerven/Index_files/Pascal%20van%20Gerven%20-%20Dissertation. pdf. Accessed 03/25/2009.
84. Wouters P., Paas F., van Merriënboer J. J. G. (2009) Observational learning from animated models: Effects of modality and reflection on transfer. Contemporary Educational Psychology 34: 1-8.