Classification of learning outcomes: Evidence from the computer games literature

Curriculum Journal

Volumn 16, Issue 4, 2005, Pages 455-474

  O'Neil, Harold F ^a   Wainess, Richard ^a   Baker, Eva L ^b  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Computer games; Evaluation; Learning outcomes; Models of learning; Motivation

Indexed keywords

EID: 85065380010     PISSN: 09585176     EISSN: 14693704     Source Type: Journal    
DOI: 10.1080/09585170500384529     Document Type: Article

References (127)

1. Adams, P. C., 1998. Teaching and learning with SimCity 2000 [electronic version]. Journal of Geography, 97 (2): 47–55.
2. Alessi, S. M., 2000. “ Simulation design for training and assessment ”. In Aircrew training and assessment, Edited by: O'Neil, H. F., Jr and Andrews, D. H., 197–222. Mahwah, NJ: Lawrence Erlbaum Associates.
3. Anderson, C. A., and Bushman, B. J., 2001. Effects of violent video games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: a meta-analytic review of the scientific literature. Psychological Science, 12: 353–359.
4. Artelt, C., 2005. Cross-cultural approaches to measuring motivation. Educational Assessment, 10: 231–255.
5. Arthur, W., Jr, Strong, M. H., Jordan, J. A., Williamson, J. E., Shebilske, W. L., and Regian, J. W., 1995. Visual attention: individual differences in training and predicting complex task performance. Acta Psychologica, 88: 3–23.
6. Baker, D., Prince, C., Shrestha, L., Oser, R., and Salas, E., 1993. Aviation computer games for crew resource management training. The International Journal of Aviation Psychology, 3: 143–156.
7. Baker, E. L., Aschbacher, P., and Bradley, C., 1985. Evaluation of WEST: ICAI system, Los Angeles, CA: University of California, Center for the Study of Evaluation. (report to Jet Propulsion Laboratory)
8. Baker, E. L., and Mayer, R. E., 1999. Computer-based assessment of problem solving. Computers in Human Behavior, 15: 269–282.
9. Baker, E. L., and O'Neil, H. F., Jr. 2002. Measuring problem solving in computer environments: current and future states. Computers in Human Behavior, 18: 609–622.
10. BBN/DARWARS. December 2004. Conference presentation folder provided at the 2004 Interservice/Industry Training, Simulation and Education Conference December, Orlando, FL
11. Benbasat, I., and Todd, P., 1993. An experimental investigation of interface design alternatives: icon vs. text and direct manipulation vs. menus. International Journal of Man-Machine Studies, 38: 369–402.
12. Betz, J. A., 1995–6. Computer games: increase learning in an interactive multidisciplinary environment. Journal of Educational Technology Systems, 24: 195–205.
13. Brougere, G., 1999. Some elements relating to children's play and adult simulation/gaming. Simulation & Gaming, 30: 134–146.
14. Brozik, D., and Zapalska, A., 2002. The PORTFOLIO GAME: decision making in a dynamic environment. Simulation & Gaming, 33: 242–255.
15. Brunken, R., Plass, J. L., and Leutner, D., 2003. Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38 (1): 53–61.
16. Carr, P. D., and Groves, G., 1998. “ The internet-based operations simulation game ”. In Selected papers for the 9th International Conference on College Teaching and Learning, Edited by: Chambers, J. A., 15–23. Jacksonville, Florida: Community College at Jacksonville.
17. Chalmers, P. A., 2003. The role of cognitive theory in human-computer interface. Computers in Human Behavior, 19: 593–607.
18. Chou, C., and Lin, H., 1998. The effect of navigation map types and cognitive styles on learners' performance in a computer-networked hypertext learning system [electronic version]. Journal of Educational Multimedia and Hypermedia, 7: 151–176.
19. Chou, C., Lin, H., and Sun, C. -t., 2000. Navigation maps in hierarchical-structured hypertext courseware [electronic version]. International Journal of Instructional Media, 27 (2): 165–182.
20. Chuang, S., 2004. The role of search strategies and feedback on a computer-based collaborative problem-solving task, Los Angeles: University of Southern California. Unpublished doctoral dissertation
21. Clark, R. E., 1999. “ The CANE model of motivation to learn and to work: a two-stage process of goal commitment and effort [electronic version] ”. In Trends in corporate training, Edited by: Lowyck, J., Leuven: University of Leuven Press.
22. Clark, R. E., ed. 2001. Learning from media: arguments, analysis, and evidence, Greenwich, CT: Information Age Publishing.
23. Clark, R. E., 2003a. Fostering the work motivation of teams and individuals. Performance Improvement, 42 (3): 21–29.
24. Clark, R. E., February 2003b. “ Strategies based on effective feedback during learning ”. In What works in distance education, Edited by: O'Neil, H. F., February, 18–19. Los Angeles: University of Southern California, Rossier School of Education. (report to the Office of Naval Research by the National Center for Research on Evaluation, Standards, and Student Testing)
25. Cobb, T., 1997. Cognitive efficiency: toward a revised theory of media. Educational Technology Research and Development, 45 (4): 21–35.
26. Cross, T. L., 1993. AgVenture: a farming strategy computer game. Journal of Natural Resources and Life Sciences Education, 22: 103–107.
27. Csikszentmihalyi, M., 1975. Beyond boredom and anxiety, San Francisco, CA: Jossey Bass.
28. Csikszentmihalyi, M., 1990. Flow: The psychology of optimal performance, New York: Cambridge University Press.
29. Davis, S., and Wiedenbeck, S., 2001. The mediating effects of intrinsic motivation, ease of use and usefulness perceptions on performance in first-time and subsequent computer users. Interacting with Computers, 13: 549–580.
30. Day, E. A., Arthur, W., Jr and Gettman, D., 2001. Knowledge structures and the acquisition of a complex skill. Journal of Applied Psychology, 86: 1022–1033.
31. de Jong, T., and van Joolingen, W. R., 1998. Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68: 179–202.
32. deCharms, R., 1986. Personal causation, New York: Academic Press.
33. Deci, E. L., 1975. Intrinsic motivation, New York: Plenum Press.
34. Dekkers, J., and Donatti, S., 1981. The interpretation of research studies on the use of simulation as an instructional strategy. Journal of Educational Research, 74 (6): 64–79.
35. Donchin, E., 1989. The learning strategies project. Acta Psychologica, 71: 1–15.
36. Druckman, D., 1995. “ The educational effectiveness of interactive games ”. In Simulation and gaming across disciplines and cultures: ISAGA at a watershed, Edited by: Crookall, D., and Aria, K., 178–187. Thousand Oaks, CA: Sage.
37. Dugdale, S., 1998. Mathematical problem-solving and computers: a study of learner-initiated application of technology in a general problem-solving context. Journal of Research on Computing in Education, 30: 239–253.
38. Farrell, I. H., and Moore, D. M., 2000. The effect of navigation tools on learners' achievement and attitude in a hypermedia environment. Journal of Educational Technology Systems, 29 (2): 169–181.
39. Galimberti, C., Ignazi, S., Vercesi, P., and Riva, G., 2001. Communication and cooperation in networked environment: an experimental analysis. CyberPsychology & Behavior, 4 (1): 131–146.
40. Garris, R., Ahlers, R., and Driskell, J. E., 2002. Games, motivation, and learning: a research and practice model. Simulation & Gaming, 33: 441–467.
41. Gee, J. P., 2003. What video games have to teach us about learning and literacy, New York: Palgrave Macmillan.
42. Gopher, D., Weil, M., and Bareket, T., 1994. Transfer of skill from a computer game trainer to flight. Human Factors, 36: 387–405.
43. Gredler, M. E., 1996. “ Educational games and simulations: a technology in search of a research paradigm ”. In Handbook of research for educational communications and technology, Edited by: Jonassen, D. H., 521–540. New York: Simon & Schuster Macmillan.
44. Green, C. S., and Bavelier, D., 2003. Action video game modifies visual selective attention. Nature, 423: 534–537.
45. Green, T. D., and Flowers, J. H., 2003. Comparison of implicit and explicit learning processes in a probabilistic task. Perceptual and Motor Skills, 97: 299–314.
46. Greenfield, P. M., deWinstanley, P., Kilpatrick, H., and Kaye, D., 1994. Action video games and informal education: effects on strategies for dividing visual attention. Journal of Applied Developmental Psychology, 15: 105–123.
47. Harter, S., 1978. Effectance motivation reconsidered: toward a developmental model. Human Development, 1: 34–64.
48. Hembree, R., 1988. Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58 (1): 47–77.
49. Herl, H. E., O'Neil, H. F., Jr, Chung, G., and Schacter, J., 1999. Reliability and validity of a computer-based knowledge mapping system to measure content understanding. Computers in Human Behavior, 15: 315–333.
50. Hidi, S., and Harackiewicz, J. M., 2000. Motivating the academically unmotivated: a critical issue for the 21st century. Review of Educational Research, 70: 151–179.
51. Hsieh, I., 2001. Types of feedback in a computer-based collaborative problem-solving Group Task, Los Angeles: University of Southern California. Unpublished doctoral dissertation
52. Hubbard, P., 1991. Evaluating computer games for language learning. Simulation & Gaming, 22: 22022–22023.
53. Jones, M. G., Farquhar, J. D., and Surry, D. W., 1995. Using metacognitive theories to design user interfaces for computer-based learning. Educational Technology, 35 (4): 12–22.
54. Kalyuga, S., Chandler, P., Touvinen, J., and Sweller, J., 2001. When problem-solving is superior to worked examples. Journal of Educational Psychology, 93: 579–588.
55. Katz, S., and Lesgold, A., 1991. “ Modeling the student in SHERLOCK II ”. In Proceedings of the IJCAI-91 Workshop W.4: Agent modelling for intelligent interaction Edited by: Kay, J., and Quilici, A., 93–127. Sydney, Australia
56. Khoo, G. -s., and Koh, T. -s., 1998. Using visualization and simulation tools in tertiary science education [electronic version]. The Journal of Computers in Mathematics and Science Teaching, 17 (1): 5–20.
57. King, K. W., and Morrison, M., 1998. A media buying simulation game using the internet. Journalism and Mass Communication Educator, 53 (3): 28–36.
58. Kirkpatrick, D. L., 1994. Evaluating training programs: the four levels, San Francisco, CA: Berrett-Koehler.
59. Kirschner, P. A., Sweller, J., and Clark, R. E., Why minimal guidance during instruction does not work: an analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, (in press)
60. Lee, J., 1999. Effectiveness of computer-based instructional simulation: a meta analysis. International Journal of Instructional Media, 26 (1): 71–85.
61. Leemkuil, H., de Jong, T., de Hoog, R., and Christoph, N., 2003. KM Quest: a collaborative internet-based simulation game. Simulation & Gaming, 34: 89–111.
62. Malone, T. W., 1981. Toward a theory of intrinsically motivating instruction. Cognitive Science, 4: 333–369.
63. Malone, T. W., and Lepper, M. R., 1987. Aptitude, learning and instruction III: cognitive and affective process analysis, Hillsdale, NJ: Lawrence Erlbaum Associates.
64. Marsh, H. W., Hau, K -T., Artelt, C., and Baumert, J., 2004. OECD's brief self-report measures of educational psychology's most useful constructs: cross-cultural, psychometric comparisons across 25 countries, Penrith South DC, New South Wales, Australia: University of West Sydney. Unpublished manuscript, SELF Research Centre
65. Mayer, R. E., 2004. Should there be a three-strikes rule against pure discovery learning?. American Psychologist, 59: 14–19.
66. Mayer, R. E., Dow, G. T., and Mayer, S., 2003. Multimedia learning in an interactive self-explaining environment: what works in the design of agent-based microworlds?. Journal of Educational Psychology, 95: 806–813.
67. Mayer, R. E., Mautone, P., and Prothero, W., 2002. Pictorial aids for learning by doing in a multimedia geology simulation game. Journal of Educational Psychology, 94: 171–185.
68. Mayer, R. E., and Moreno, R., 1998. A split-attention effect in multimedia learning: evidence for dual processing systems in working memory. Journal of Educational Psychology, 90: 312–320.
69. Mayer, R. E., Sobko, K., and Mautone, P. D., 2003. Social cues in multimedia learning: role of speaker's voice. Journal of Educational Psychology, 95: 419–425.
70. Mayer, R. E., and Wittrock, M. C., 1996. “ Problem-solving transfer ”. In Handbook of educational psychology, Edited by: Berliner, D. C., and Calfee, R. C., 47–62. New York: Simon & Schuster Macmillan.
71. Mislevy, R. J., Substance and structure in assessment arguments. Law, probability, and risk, (in press) Available online at: http://www.education.umd.edu/EDMS/mislevy/papers/Argument.pdf
72. Mislevy, R. J., Steinberg, L. S., and Almond, R. G., 2003. On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1: 3–67.
73. Mislevy, R. J., Wilson, M. R., Ercikan, K., and Chudowsky, N., 2003. “ Psychometric principles in student assessment ”. In International handbook of educational evaluation, Edited by: Kellaghan, T., and Stufflebeam, D., 489–531. Dordrecht, Netherlands: Kluwer Academic Press.
74. Moreno, R., and Mayer, R. E., 2000. Engaging students in active learning: the case for personalized multimedia messages. Journal of Educational Psychology, 92: 724–733.
75. Morris, C. S., Hancock, P. A., and Shirkey, E. C., 2004. Motivational effects of adding context relevant stress in PC-based game training. Military Psychology, 16: 135–147.
76. Oliver, M., and Pelletier, C., 2005. The things we learned on Liberty Island: designing games to help people become competent game players. proceedings of DIGRA 2005 Conference: Changing Views–Worlds in Play.
77. O'Neil, H. F., Jr. 1999. Perspectives on computer-based performance assessment of problem solving: editor's introduction. Computers in Human Behavior, 15: 255–268.
78. O'Neil, H. F., Jr. 2002. Perspective on computer-based assessment of problem solving [special issue]. Computers in Human Behavior, 18: 605–607.
79. O'Neil, H. F., Jr, Baker, E. L., and Fisher, J. Y.-C., 2002. A formative evaluation of ICT games, Los Angeles, CA: University of Southern California, Rossier School of Education. (report to the Institute for Creative Technologies, University of Southern California)
80. O'Neil, H. F., Jr and Fisher, Y. -C., 2004. “ A technology to support leader development: computer games ”. In Leader development for transforming organizations, Edited by: Day, D. V., Zaccaro, S. J., and Halpin, S. M., 99–121. Mahwah, NJ: Lawrence Erlbaum Associates.
81. O'Neil, H. F., Jr and Herl, H. E., April 1998. “ Reliability and validity of a trait measure of self-regulation ”. In paper presented at the annual meeting of the American Educational Research Association April, San Diego, CA
82. Parchman, S. W., Ellis, J. A., Christinaz, D., and Vogel, M., 2000. An evaluation of three computer-based instructional strategies in basic electricity and electronics training. Military Psychology, 12: 73–87.
83. Pekrun, R., Goetz, T., Titz, W., and Perry, R. P., 2002. Academic emotions in students' self-regulated learning and achievement: a program of qualitative and quantitative research. Educational Psychologist, 37: 91–105.
84. Pelletier, C., 2005. The uses of literacy in studying computer games: comparing students' oral and visual representations of games. English Teaching: Practice and Critique, 4 (1): 40–59.
85. Porter, D. B., Bird, M. E., and Wunder, A., 1990–1. Competition, cooperation, satisfaction, and the performance of complex tasks among Air Force cadets. Current Psychology: Research & Reviews, 9: 347–354.
86. Prislin, R., Jordan, J. A., Worchel, S., Semmer, F. T., and Shebilske, W. L., 1996. Effects of group discussion on acquisition of complex skills. Human Factors, 38: 404–416.
87. Quinn, C. N., 1991. Computers for cognitive research: a HyperCard adventure game. Behavior Research Methods, Instruments, & Computers, 23: 237–246.
88. Ramsberger, P. F., Hopwood, D., Hargan, C. S., and Underhill, W. G., 1983. Evaluation of a spatial data management system for basic skills education. Final Phase I Report for Period 7 October 1980–30 April 1983, Alexandria, VA: Human Resources Research Organization. (HumRRO FR-PRD-83-23)
89. Resnick, H., and Sherer, M., 1994. “ Computerized games in the human services–an introduction ”. In Electronic tools for social work practice and education, Edited by: Resnick, H., 5–16. Binghamton, NY: Haworth Press.
90. Rhodenizer, L., Bowers, C. A., and Bergondy, M., 1998. Team practice schedules: what do we know?. Perceptual and Motor Skills, 87: 31–34.
91. Ricci, K. E., Salas, E., and Cannon-Bowers, J. A., 1996. Do computer-based games facilitate knowledge acquisition and retention?. Military Psychology, 8: 295–307.
92. Rieber, L. P., 1996. Seriously considering play: designing interactive learning environments based on the blending of microworlds, simulations, and games. Educational Technology Research and Development, 44 (2): 43–58.
93. Rieber, L. P., and Parmley, M. W., 1995. To teach or not to teach? Comparing the use of computer-based simulations in deductive versus inductive approaches to learning with adults in science. Journal of Educational Computing Research, 13: 359–374.
94. Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., and Carlstrom. 2004. Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130: 261–288.
95. Rosenorn, T., and Kofoed, L. B., 1998. Reflection in learning processes through simulation/gaming. Simulation & Gaming, 29: 432–440.
96. Ruben, B. D., 1999. Simulations, games, and experience-based learning: the quest for a new paradigm for teaching and learning. Simulation & Gaming, 30: 498–505.
97. Ruddle, R. A., Payne, S. J., and Jones, D. M., 1999. The effects of maps on navigation and search strategies in very-large-scale virtual environments. Journal of Experimental Psychology: Applied, 5 (1): 54–75.
98. Salas, E., Bowers, C. A., and Rhodenizer, L., 1998. It is not how much you have but how you use it: toward a rational use of simulation to support aviation training. The International Journal of Aviation Psychology, 8: 197–208.
99. Salomon, G., 1983. The differential investment of mental effort in learning from different sources. Educational Psychology, 18: 42–50.
100. Santos, J., 2002. Developing and implementing an internet-based financial system simulation game. The Journal of Economic Education, 33 (1): 31–40.
101. Schacter, J., Herl, H. E., Chung, G., Dennis, R. A., and O'Neil, H. F., Jr. 1999. Computer-based performance assessments: a solution to the narrow measurement and reporting of problem-solving. Computers in Human Behavior, 13: 403–418.
102. Schank, R. C., 1997. Virtual learning: a revolutionary approach to build a highly skilled workforce, New York: McGraw-Hill.
103. Seifert, T. L., 2004. Understanding student motivation. Educational Research, 46: 137–149.
104. Shebilske, W. L., Regian, W., Arthur, W., Jr and Jordan, J. A., 1992. A dyadic protocol for training complex skills. Human Factors, 34: 369–374.
105. Shewokis, P. A., 2004. Memory consolidation and contextual interference effects with computer games. Perceptual and Motor Skills, 97: 381–389.
106. Spiker, V. A., and Nullmeyer, R. T., 2001. Benefits and limitations of simulation-based mission planning and rehearsal (n.d.), Unpublished manuscript.
107. Stolk, D., Alexandrian, D., Gros, B., and Paggio, R., 2001. Gaming and multimedia applications for environmental crisis management training. Computers in Human Behavior, 17: 627–642.
108. Sugrue, B., and Kim, K. -H., 2004. State of the industry. ASTD's annual review of trends in workplace learning and performance. Executive summary, Alexandria, VA: American Society for Training and Development.
109. Sweller, J., 1988. Cognitive load during problem solving: effects on learning. Cognitive Science, 12: 2572–2585.
110. Sweller, J., 1989. Cognitive technology: some procedures for facilitating learning and problem solving in mathematics and science. Journal of Educational Psychology, 81: 457–466.
111. Sweller, J., 1994. Cognitive load theory, learning difficulty and instructional design. Learning and Instruction, 4: 295–312.
112. Sweller, J., 2003. “ Evolution of human cognitive architecture ”. In The psychology of learning and motivation, Edited by: Ross, B., vol. 43, 215–266. San Diego, CA: Academic Press.
113. Sweller, J., van Merrienboer, J., and Paas, F., 1998. Cognitive architecture and instructional design. Educational Psychology Review, 10: 251–269.
114. Thiagarajan, S., 1998. The myths and realities of simulations in performance technology. Educational Technology, 38 (4): 35–41.
115. Thomas, P., and Macredie, R., 1994. Games and the design of human-computer interfaces. Educational Technology, 31 (2): 134–142.
116. Tkacz, S., 1998. Learning map interpretation: skill acquisition and underlying abilities. Journal of Environmental Psychology, 18: 237–249.
117. Toulmin, S. E., 1958. The uses of argument, Cambridge: Cambridge University Press.
118. van Merrienboer, J. J. G., Clark, R. E., and de Croock, M. B. M., 2002. Blueprints for complex learning: the 4C/ID-model. Educational Technology Research & Development, 50 (2): 39–64.
119. van Merrienboer, J. J. G., Kirschner, P. A., and Kester, L., 2003. Taking a load off a learner's mind: instructional design for complex learning. Educational Psychologist, 38 (1): 5–13.
120. Wainess, R., and O'Neil, H. F., Jr. 2003. Feasibility study, University of Southern California, Rossier School of Education. Unpublished manuscript
121. Washbush, J., and Gosen, J., 2001. An exploration of game-derived learning in total enterprise simulations. Simulation & Gaming, 32: 281–296.
122. Westbrook, J. I., and Braithwaite, J., 2001. The Health Care Game: an evaluation of a heuristic, web-based simulation. Journal of Interactive Learning Research, 12 (1): 89–104.
123. White, B. Y., and Frederiksen, J. R., 1998. Inquiry, modeling, and metacognition: making science accessible to all students. Cognition and Instruction, 16: 3–118.
124. Wolfe, J., 1997. The effectiveness of business games in strategic management course work [electronic version]. Simulation & Gaming Special Issue: Teaching Strategic Management, 28: 360–376.
125. Wolfe, J., and Roge, J. N., 1997. Computerized general management games as strategic management learning environments [electronic version]. Simulation & Gaming Special Issue: Teaching Strategic Management, 28: 423–441.
126. Yair, Y., Mintz, R., and Litvak, S., 2001. 3D-virtual reality in science education: an implication for astronomy teaching. Journal of Computers in Mathematics and Science Teaching, 20: 293–305.
127. Yu, F. -Y., 2001. Competition within computer-assisted cooperative learning environments: cognitive, affective, and social outcomes. Journal of Educational Computing Research, 24: 99–117.