Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments

International Journal of Human Computer Studies

Volumn 68, Issue 4, 2010, Pages 223-241

  Baker, Ryan S J d ^a   D'Mello, Sidney K ^b   Rodrigo, Ma Mercedes T ^c   Graesser, Arthur C ^b  

^a WORCESTER POLYTECHNIC INSTITUTE   (United States)

^b UNIVERSITY OF MEMPHIS   (United States)

^c ATENEO DE MANILA UNIVERSITY   (Philippines)

Author keywords

Affect; Affective computing; Affective persistence; Cognitive affective states; Educational games; Intelligent tutoring systems

Indexed keywords

AFFECTIVE COMPUTING; AFFECTIVE STATE; COMPUTER-BASED LEARNING ENVIRONMENTS; EDUCATIONAL GAME; FIELD OBSERVATIONS; INTELLIGENT TUTORING SYSTEM; LEARNING ENVIRONMENTS; MULTIPLE FACTORS; PHILIPPINES; SELF-REPORT;

COGNITIVE SYSTEMS; COMPUTER AIDED INSTRUCTION; EDUCATION COMPUTING; STUDENTS;

PROBLEM SOLVING;

EID: 76649103360     PISSN: 10715819     EISSN: 10959300     Source Type: Journal    
DOI: 10.1016/j.ijhcs.2009.12.003     Document Type: Article

References (106)

1. Anderson J.R., Corbett A.T., Koedinger K.R., and Pelletier R. Cognitive tutors: lessons learned. The Journal of the Learning Sciences 4 (1995) 167-207
2. Ang J., Dhillon R., Krupski A., Shriberg E., and Stolcke A. Prosody-based automatic detection of annoyance and frustration. Human-Computer Dialog. Proceedings of the International Conference on Spoken Language Processing, Denver, CO (2002) pp. 2037-2039
3. Arnold J. Affect in Language Learning (1999), Cambridge University Press, Cambridge, UK
4. Baker, R.S., 2005. Designing intelligent tutors that adapt to when students game the system. Doctoral Dissertation. CMU Technical Report CMU-HCII-05-104.
5. Baker R.S.J.d. Modeling and understanding students' off-task behavior in intelligent tutoring systems. Proceedings of ACM CHI 2007: Computer-Human Interaction (2007) pp. 1059-1068
6. Baker R.S.J.d., Corbett A.T., Aleven V., Koedinger K.R., de Carvalho A.M.J.A., and Raspat J. Educational software features that encourage and discourage "Gaming the System". Proceedings of the 14th International Conference on Artificial Intelligence in Education (2009) pp. 475-482
7. Baker R.S.J.d., Corbett A.T., Koedinger K.R., Evenson S.E., Roll I., Wagner A.Z., Naim M., Raspat J., Baker D.J., and Beck J. Adapting to when students game an intelligent tutoring system. Proceedings of the Eighth International Conference on Intelligent Tutoring Systems (2006) pp. 392-401
8. Baker R.S., Corbett A.T., Koedinger K.R., and Wagner A.Z. Off-task behavior in the cognitive tutor classroom: when students "Game the System". Proceedings of ACM CHI 2004: Computer-Human Interaction (2004) pp. 383-390
9. Baker R.S.J.d., Corbett A.T., Roll I., and Koedinger K.R. Developing a generalizable detector of when students game the system. User Modeling and User-Adapted Interaction 18 3 (2008) 287-314
10. Baker R.S.J.d., Corbett A.T., and Wagner A.Z. Human classification of low-fidelity replays of student actions. Proceedings of the Educational Data Mining Workshop at the Eighth International Conference on Intelligent Tutoring Systems (2006) pp. 29-36
11. Baker R., Walonoski J., Heffernan N., Roll I., Corbett A., and Koedinger K. Why students engage in "Gaming the System" behavior in interactive learning environments. Journal of Interactive Learning Research 19 2 (2008) 185-224
12. Barrett L.F. Are emotions natural kinds?. Perspectives on Psychological Science 1 (2006) 28-58
13. Bartel C.A., and Saavedra R. The collective construction of work group moods. Administrative Science Quarterly 45 2 (2000) 197-231
14. Bloom B.S. The 2 sigma problem: the search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher 13 (1984) 3-16
15. Bower G. How might emotions affect learning. In: Christianson S.A. (Ed). The Handbook of Emotion and Memory: Research and Theory (1992), Erlbaum, Hillsdale, NJ 3-31
16. Burleson, W., 2006. Affective learning companions: strategies for empathetic agents with real-time multimodal affective sensing to foster meta-cognitive and meta-affective approaches to learning, motivation, and perseverance. Doctoral Thesis, Massachusetts Institute of Technology.
17. Carroll J.M. Human-computer interaction: psychology as a science of design. Annual Review of Psychology 48 (1997) 61-83
18. Castellano G., Villalba S.D., and Camurri A. Recognizing human emotions from body movement and gesture dynamics. In: Paiva A., Prada R., and Picard R.W. (Eds). Affective Computing and Intelligent Interaction (2007), Springer-Verlag, Berlin, Heidelberg 71-82
19. Clifford M.M. Failure tolerance and academic risk-taking in ten- to twelve-year-old students. British Journal of Educational Psychology 58 (1988) 15-27
20. Clifford M.M. Risk taking: theoretical, empirical, and educational considerations. Educational Psychologist 26 (1991) 263-298
21. Cocea M., Hershkovitz A., and Baker R.S.J.d. The impact of off-task and gaming behaviors on learning: immediate or aggregate?. Proceedings of the 14th International Conference on Artificial Intelligence in Education (2009) pp. 507-514
22. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20 (1960) 37-46
23. Cohen P.A., Kulik J.A., and Kulik C.C. Educational outcomes of tutoring: a meta-analysis of findings. American Educational Research Journal 19 (1982) 237-248
24. Conati C. Probabilistic assessment of user's emotions in educational games. Journal of Applied Artificial Intelligence 16 (2002) 555-575
25. Conati C., and Zhao X. Building and evaluating an intelligent pedagogical agent to improve the effectiveness of an educational game. Proceedings of the Ninth International Conference on Intelligent User Interface (2004) pp. 6-13
26. Corbett A.T., and Anderson J.R. Knowledge tracing: modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction 4 4 (1995) 253-278
27. Craig S.D., Graesser A.C., Sullins J., and Gholson B. Affect and learning: an exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media 29 3 (2004) 241-250
28. Csikszentmihalyi M. Flow: The Psychology of Optimal Experience (1990), Harper-Row, New York
29. de Vicente A., and Pain H. Informing the detection of the students' motivational state: an empirical study. Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (2002) pp. 933-943
30. D'Mello S.K., Chipman P., and Graesser A.C. Posture as a predictor of learner's affective engagement. Proceedings of the 29th Annual Cognitive Science Society (2007) pp. 905-910
31. D'Mello S.K., Taylor R., and Graesser A.C. Monitoring affective trajectories during complex learning. Proceedings of the 29th Annual Cognitive Science Society (2007) pp. 203-208
32. Dodds P., and Fletcher J.D. Opportunities for new "smart" learning environments enabled by next-generation web capabilities. Journal of Educational Multimedia and Hypermedia 13 4 (2004) 391-404
33. Dweck C.S. Self-Theories: Their Role in Motivation, Personality, and Development (2000), Psychology Press, Philadelphia, PA
34. Dweck C.S. Messages that motivate: how praise molds students' beliefs, motivation, and performance (in surprising ways). In: Aronson J. (Ed). Improving Academic Achievement: Impact of Psychological Factors on Education (2002), Academic Press, Orlando, FL 61-87
35. Ekman P., and Friesen W.V. Nonverbal leakage and clues to deception. Psychiatry 32 (1969) 88-105
36. Ekman P., and Friesen W.V. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology 17 (1971) 124-129
37. Ekman P., and Friesen W.V. The Facial Action Coding System: A Technique for the Measurement of Facial Movement (1978), Consulting Psychologists Press, Palo Alto, CA
38. Elfenbein H.A., and Ambady N. On the universality and cultural specificity of emotion recognition: a meta-analysis. Psychological Bulletin 128 (2002) 203-235
39. Elfenbein H.A., and Ambady N. Is there an ingroup advantage in emotion recognition?. Psychological Bulletin 128 (2002) 243-249
40. Festinger L. A Theory of Cognitive Dissonance (1957), Row, Perterson & Company, Evanston, IL
41. Fredrickson B.L., and Branigan C. Positive emotions broaden the scope of attention and thought-action repertoires. Cognition and Emotion 19 (2005) 313-332
42. Gee J.P. Situated Language and Learning: A Critique of Traditional Schooling (2004), Routledge, Taylor & Francis, London, UK
43. Graesser A.C., D'Mello S.K., Chipman P., King B., and McDaniel B. Exploring relationships between affect and learning with AutoTutor. In: Luckin R., Koedinger K., and Greer J. (Eds). Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work (2007), IOS Press, Amsterdam 16-23
44. Graesser A.C., Rus V., D'Mello S., and Jackson G.T. AutoTutor: learning through natural language dialogue that adapts to the cognitive and affective states of the learner. In: Robinson D.H., and Schraw G. (Eds). Current Perspectives on Cognition, Learning and Instruction: Recent Innovations in Educational Technology that Facilitate Student Learning. Information Age Publishing (2008) 95-125
45. Graesser A.C., D'Mello S., and Person N.K. Metaknowledge in tutoring. In: Hacker D., Donlosky J., and Graesser A.C. (Eds). Handbook of Metacognition in Education (2009), Taylor & Francis, Mahwah, NJ
46. Graesser A.C., Lu S., Jackson G.T., Mitchell H., Ventura M., Olney A., and Louwerse M.M. AutoTutor:a tutor with dialogue in natural language. Behavior Research Methods, Instruments, and Computers 36 (2004) 180-193
47. Graesser A.C., Lu S., Olde B.A., Cooper-Pye E., and Whitten S. Question asking and eye tracking during cognitive disequilibrium: comprehending illustrated texts on devices when the devices break down. Memory and Cognition 33 (2005) 1235-1247
48. Graesser A.C., Person N., Harter D., and Tutoring Research Group. Teaching tactics and dialogue in AutoTutor. International Journal of Artificial Intelligence in Education 12 (2001) 257-279
49. Graesser A.C., Witherspoon A., McDaniel B., D'Mello S., Chipman P., and Gholson B. Detection of emotions during learning with AutoTutor. Proceedings of the 28th Annual Meeting of the Cognitive Science Society (2006), Erlbaum, Mahwah, NJ pp. 285-290
50. Grimm M., Kroschel K., Harris H., Nass C., Schuller B., Rigoll G., and Moosmayr T. On the necessity and feasibility of detecting driver's emotional state while driving. In: Paiva A., Prada R., and Picard R.W. (Eds). Affective Computing and Intelligent Interaction (2007), Springer-Verlag, Berlin, Heidelberg 126-138
51. Grimm M., Mower E., Kroschel K., and Narayan S. Combining categorical and primitives-based emotion recognition. 14th European Signal Processing Conference (EUSIPCO), Florence, Italy (2006)
52. Guhe, M., Gray, W.D., Schoelles, M.J., Ji, Q., 2004. Towards an affective cognitive architecture. Poster presented at the Cognitive Science Conference, Chicago, IL.
53. Hess U., Kappas A., and Scherer K.R. Multichannel communication of emotion: synthetic signal production. In: Scherer K.R. (Ed). Facets of Emotion: Recent Research (1988), Erlbaum, Hillsdale, NJ 161-182
54. Hone K. Empathic agents to reduce user frustration: the effects of varying agent characteristics. Interacting with Computers 18 (2006) 227-245
55. Hudlicka E., and McNeese D. Assessment of user affective and belief states for interface adaptation: application to an Air Force pilot task. User Modeling and User-Adapted Interaction 12 1 (2002) 1-47
56. Jones E., and Nisbett R. The Actor and the Observer: Divergent Perceptions of the Causes of Behavior (1971), General Learning Press, New York
57. Kapoor A., Burleson W., and Picard R.W. Automatic prediction of frustration. International Journal of Human-Computer Studies 65 8 (2007) 724-736
58. Klein J., Moon Y., and Picard R. This computer responds to user frustration-theory, design, and results. Interacting with Computers 14 2 (2002) 119-140
59. Koedinger K.R., and Corbett A.T. Cognitive tutors: technology bringing learning sciences to the classroom. In: Sawyer R.K. (Ed). The Cambridge Handbook of the Learning Sciences (2006), Cambridge University Press, New York, NY
60. Komatsu T., Ohtsuka S., and Ueda K. Comprehension of users' subjective interaction states during their interaction with an artificial agent by means of heard rate variability index. In: Paiva A., Prada R., and Picard R.W. (Eds). Affective Computing and Intelligent Interaction (2007), Springer-Verlag, Berlin, Heidelberg 266-277
61. Kort B., Reilly R., and Picard R. An affective model of interplay between emotions and learning: reengineering educational pedagogy-building a learning companion. Proceedings IEEE International Conference on Advanced Learning Technology: Issues, Achievements and Challenges (2001), IEEE Computer Society, Madison, Wisconsin pp. 43-48
62. Lehman B.A., Matthews M., D'Mello S.K., and Person N. Understanding students' affective states during learning. In: Woolf B., et al. (Ed). Ninth International Conference on Intelligent Tutoring Systems. ITS 2008, Lecture Notes in Computer Sciences vol. 5091 (2008), Springer-Verlag 50-59
63. Lehman B., D'Mello S.K., and Person N. All Alone with your emotions: an analysis of student emotions during effortful problem solving activities. Supplementary Proceedings of the Workshop on Emotional and Cognitive issues in ITS (WECITS) (2008)
64. Lepper M.R., and Cordova D.I. A desire to be taught: instructional consequences of intrinsic motivation. Motivation and Emotion 16 3 (1992) 187-208
65. Litman D.J., and Forbes-Riley K. Predicting student emotions in computer-human tutoring dialogues. Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (2004), Association for Computational Linguistics, East Stroudsburg, PA pp. 352-359
66. Litman D.J., and Silliman S. ITSPOKE: an intelligent tutoring spoken dialogue system. Proceedings of the human Language Technology Conference: Third Meeting of the North American Chapter of the Association of Computational Linguistics (2004) pp. 52-54
67. Mandler G. Mind and Body: Psychology of Emotion and Stress (1984), W.W. Norton & Company, New York
68. Mandler G. Emotion. In: Bly B.M., and Rumelhart D.E. (Eds). Cognitive science. Handbook of perception and cognition (2nd ed.) (1999), Academic Press, San Diego, CA
69. Mandryk R.L., and Atkins M.S. A fuzzy physiological approach for continuously modeling emotion during interaction with play environments. International Journal of Human-Computer Studies 6 4 (2007) 329-347
70. Marinier R.P., and Laird J.E. Computational modeling of mood and feeling from emotion. Proceedings of the Annual Meeting of the Cognitive Science Society (2007)
71. Marinier, R., Laird, J.A., 2006. Cognitive Architecture Theory of Comprehension and Appraisal. Agent Construction and Emotion.
72. McNeese M.D. New visions of human-computer interaction: making affect compute. International Journal of Human-Computer Studies 59 (2003) 33-53
73. McQuiggan S.W., Lee S., and Lester J.C. Early prediction of student frustration. Proceedings of the Second International Conference on Affective Computing and Intelligent Interaction (2007) pp. 698-709
74. Mentis H.M. Memory of frustrating experiences. In: Nahl D., and Bilal D. (Eds). Information and Emotion (2007), Information Today, Medford, NJ
75. Meyer D.K., and Turner J.C. Re-conceptualizing emotion and motivation to learn in classroom contexts. Educational Psychology Review 18 4 (2006) 377-390
76. Miserandino M. Children who do well in school: individual differences in perceived competence and autonomy in above-average children. Journal of Educational Psychology 88 (1996) 203-214
77. Nicaud J.-F., Bouhineau D., and Chaachoua H. Mixing microworld and CAS features in building computer systems that help students learn algebra. International Journal of Computers for Mathematical Learning 9 (2004) 169-211
78. Nicaud, J.F., Bouhineau, D., Mezerette, S., Andre, N., 2007. Aplusix II [Computer software].
79. Norman D.A. Emotional Design: Why We Love (or Hate) Everyday Things (2004), Basic Books, New York
80. In: Paiva A., Prada R., and Picard R.W. (Eds). Affective Computing and Intelligent Interaction (2007), Springer Verlag, Berlin, Heidelberg
81. Pantic M., and Rothkrantz L.J.M. Towards an affect-sensitive multimodal human-computer interaction. Proceedings of the IEEE, Special Issue on Multimodal Human-Computer Interaction (HCI) 91 9 (2003) 1370-1390
82. Patrick B., Skinner E., and Connell J. What motivates children's behavior and emotion? Joint effects of perceived control and autonomy in the academic domain. Journal of Personality and Social Psychology 65 (1993) 781-791
83. Perkins R.E., and Hill A.B. Cognitive and affective aspects of boredom. British Journal of Psychology 76 2 (1985) 221-234
84. Peterson P.L., and Fennema E. Effective teaching, student engagement in classroom activities, and sex-related differences in learning mathematics. American Educational Research Journal 22 3 (1985) 309-335
85. Picard R. Affective Computing (1997), MIT Press, Cambridge, MA
86. Picard R.W., Papert S., Bender W., Blumberg B., Berezeal C., Cavallo D., Machover T., Resnick M., Roy D., and Strohecker C. Affective learning-a manifesto. BT Technology Journal 22 4 (2004) 253-269
87. Planalp S., DeFrancisco V.L., and Rutherford D. Varieties of cues to emotion in naturally occurring settings. Cognition and Emotion 10 2 (1996) 137-153
88. Prendinger H., and Ishizuka M. The empathic companion: a character-based interface that addresses users' affective states. International Journal of Applied Artificial Intelligence 19 3-4 (2005) 267-285
89. Prensky M. Digital Game-Based Learning (2007), Paragon House, St. Paul, MN
90. Rodrigo M.M.T., Baker R.S.J.d., Lagud M.C.V., Lim S.A.L., Macapanpan A.F., Pascua S.A.M.S., Santillano J.Q., Sevilla L.R.S., Sugay J.O., Tep S., and Viehland N.J.B. Affect and Usage Choices in Simulation Problem Solving Environments. Proceedings of the 13th International Conference on Artificial Intelligence in Education (2007) pp. 145-152
91. Russell J. Core affect and the psychological construction of emotion. Psychological Review 110 (2003) 145-172
92. Russell J.A. Is there universal recognition of emotion from facial expression?: a review of the cross-cultural studies. Psychological Bulletin 115 (1994) 102-141
93. Schutzwohl A., and Borgstedt K. the processing of affectively valenced stimuli: the role of surprise. Cognition & Emotion 19 (2005) 583-600
94. Shafran I., Riley M., and Mohri M. Voice signatures. Proceedings IEEE Automatic Speech Recognition and Understanding Workshop (2003), IEEE, Piscataway, NJ pp. 31-36
95. Sierra Online., 2001. The Incredible Machine: Even More Contraptions [Computer Software].
96. Silvia P., and Abele A. Can positive affect induce self-focused attention? Methodological and measurement issues. Cognition and Emotion 16 (2002) 845-853
97. Sylwester R. How emotions affect learning. Educational Leadership 52 2 (1994) 60-65
98. Stein N.L., and Levine L.J. Making sense out of emotion. In: Kessen W., Ortony A., and Kraik F. (Eds). Memories, Thoughts, and Emotions: ESSAYS in Honor of George Mandler (1991), Erlbaum, Hillsdale, NJ 295-322
99. Stein N.L., Hernandez M.W., and Trabasso T. Advances in modeling emotions and thought: the importance of developmental, online, and multilevel analysis. In: Lewis M., Haviland-Jones J.M., and Barrett L.F. (Eds). Handbook of Emotions. 3rd ed (2008), Guilford Press, New York 574-586
100. VanLehn K. Mind Bugs: The Origins of Procedural Misconceptions (1990), MIT Press, Cambridge, MA
101. VanLehn K., Lynch C., Schulze K., Shapiro J.A., Shelby R., Taylor L., Treacy D., Weinstein A., and Wintersgill M. The Andes physics tutoring system: five years of evaluations. International Journal of Artificial Intelligence and Education 15 3 (2005) 1-47
102. VanLehn K., Graesser A.C., Jackson G.T., Jordan P., Olney A., and Rose C.P. When are tutorial dialogues more effective than reading?. Cognitive Science (2007)
103. VanLehn K., Siler S., Murray C., Yamauchi T., and Baggett W. Why do only some events cause learning during human tutoring?. Cognition and Instruction 21 3 (2003) 209-249
104. Walonoski J.A., and Heffernan N.T. Detection and analysis of off-task gaming behavior in intelligent tutoring systems. Proceedings of the Eighth International Conference on Intelligent Tutoring Systems (2006) pp. 382-391
105. Wagner J., Vogt T., and Andre E. A systematic comparison of different HMM designs for emotion recognition from acted and spontaneous speech. In: Paiva A., Prada R., and Picard R.W. (Eds). Affective Computing and Intelligent Interaction (2007), Springer-Verlag, Berlin, Heidelberg 114-125
106. Whang M.C., Lim J.S., and Boucsein W. Preparing computers for affective communication: a psychophysiological concept and preliminary results. Human Factors 45 4 (2003) 623-634