Emotions in serious games: From experience to assessment

International Journal of Emerging Technologies in Learning

Volumn 5, Issue SPECIAL ISSUE 2, 2010, Pages 7-16

  Anolli, L ^a   Mantovani, F ^a   Confalonieri, L ^a   Ascolese, A ^a   Peveri, L ^a  

^a UNIVERSITY OF MILANO BICOCCA   (Italy)

Author keywords

Affective gaming; Emotions; Multimodality; Serious games

Indexed keywords

COMPUTER AIDED INSTRUCTION; HUMAN COMPUTER INTERACTION; USER EXPERIENCE;

AFFECTIVE GAMING; EMOTIONAL EXPERIENCES; EMOTIONS; LEARNING ENVIRONMENTS; LEARNING OPPORTUNITY; MULTI-MODALITY; PEDAGOGICAL ELEMENTS; TRAINING APPLICATIONS;

SERIOUS GAMES;

EID: 84866843532     PISSN: 18688799     EISSN: 18630383     Source Type: Journal    
DOI: 10.3991/ijet.v5s3.1496     Document Type: Article

References (111)

1. D.R. Michael and S.L. Chen, Serious Games: Games That Educate, Train, and Inform. Muska & Lipman/Premier-Trade, 2006.
2. C. Aldrich, The Complete Guide to Simulations and Serious Games. Pfeiffer, 2009.
3. N. Frijda. The emotions. Cambridge: Cambridge University Press, 1986.
4. N. Frijda. The laws of emotion. Mahwah: Erlbaum, 2007.
5. B. Rogoff "Cognition as a collaborative process," in Cognition, perception and language, D. Kuhn and R.S. Siegler, Eds. New York: Wiley, 1990, pp. 679-744.
6. B. Rogoff, The cultural nature of human development. New York: Oxford University Press, 2003.
7. J.P. Gee "What video games have to teach us about learning and literacy," Comp. Entert. (CIE), vol.1, pp. 1-4, 2003.
8. R. Van Eck "Building Artificially Intelligent Learning Games," in Intelligent Information Technologies: Concepts, Methodologies, Tools, and Applications, V. Sugumaran, Ed. 2007.
9. R. Zemsky and W.F. Massy, "Thwarted innovation what happened to e-learning and why," Lear. Allian. Univer. Pennsyl. Tech. Rep. Dec. 2004.
10. T. Mayes and S. De Freitas, "Review of e-learning theories, frameworks and models," JISC- E-pedag. Desk Stud. E-learn. Mod. Tech. Rep. 2004.
11. J. Dalziel, "Implementing learning design: the learning activity management system," in Proc. 20th Ann. Conf. Australasian Society Computers Learning Tertiary Education, Adelaide, Australia, 2003.
12. J. LeDoux, The Emotional Brain: The Mysterious Underpinnings of Emotional Life. London, England: Weidenfeld & Nicholson, 1998.
13. C. Sansone and D.B. Thoman "Does what we feel affect what we learn? Some answers and new questions," Learn. Instruc., vol.15(5), pp. 507-515, 2005. doi:10.1016/j.learninstruc.2005. 07.015
14. R. Pekrun, T. Goetz, W. Titz and R.P Perry "Academic emotions in students' self-regulated learning and achievement: a program of qualitative and quantitative research," Educat. Psychol., vol. 37(2), pp. 91-105, 2002. doi:10.1207/S15326985EP3702_4
15. K. O'Regan "Emotion and e-learning," J. Asynchr. Learn. Netw., vol. 7(3), pp. 78-92, 2003.
16. M. Wosnitza and S. Volet "Origin, direction and impact of emotions in social online learning," Learn. Instruc., vol. 15(5), pp. 449-464, 2005. doi:10.1016/j.learninstruc.2005.07.009
17. L. Anolli and F. Mantovani, La simulazione e i Serious Games. Teoria e applicazioni. Bologna: Il Mulino, 2011.
18. P. Rozin "Preadaptation and the puzzles and properties of pleasure," in Well being: The foundations of hedonic psychology, D. Kahneman, E. Diener and N. Schwarz, Eds. New York: Russell Sage, 1999, pp. 109-133.
19. K. Oatley and M.J. Jenkins, Understanding emotions. Malden, MA, and Oxford, UK: Blackwell, 1996.
20. C.H. Bühler, Kindheit und Jugend. Leipzig: Hirzel, 1931.
21. A.W. Kruglanski and D.M. Webster "Motivated closing of the mind: Seizing and Freezing," Psychol. Rev., vol. 103(2), pp. 263-283, 1996. doi:10.1037/0033-295X.103.2.263
22. C.R. Nordstom, D. Wendland and K.B. Williams "To err is human: An examination of the effectiveness of error management training," J. Busin. Psychol.,vol.12, pp. 269-282, 1998. doi:10.1023/A:1025019212263
23. K. Lewin (1951), Teoria e sperimentazione in psicologia sociale. Bologna: Il Mulino, 1972.
24. J. Bruner, Child's talk: Learning to use language. New York: Norton,1983.
25. B. Sutton-Smith, The Ambiguity of Play. Harvard University Press, 1997.
26. C. Klimmt "Serious Games and Social Change," in Serious Games. Mechanisms and Effects, U. Ritterfeld, M. Cody and P. Vorderer, Eds. New York: Routledge, 2009.
27. N. Keith and M. Frese "Effectiveness of error management training: A meta-analysis," J. App. Psychol., vol.93, pp. 59-69, 2008. doi:10.1037/0021-9010.93.1.59
28. E.A. Locke, K.N. Shaw, L.M. Saari and G.P. Latham, G.P. "Goal setting and task performance: 1969-1980," Psychol. Bull.,vol. 90, pp.125-152, 1981. doi:10.1037/0033-2909.90.1.125
29. A. Bandura and D. Cervone "Self-evaluative and self-efficacy mechanisms governing the motivational effects of goal systems," J. Person. Soc. Psychol., vol. 45, pp. 1017-1028, 1983. doi:10.1037/0022-3514.45.5.1017
30. R. Ilies and T.A. Judge "Goal regulation across time: The effects of feedback and affect," J. App. Psychol., vol. 90, pp. 453-467, 2005. doi:10.1037/0021-9010.90.3.453
31. G.P. Latham, Work motivation: History, theories, research, and practice. Thousand Oaks, CA: Sage, 2007.
32. A. Bandura, Self-efficacy: The exercise of control. New York, NY: Freeman, 1997.
33. E.A. Locke and G.P. Latham, A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall, 1990.
34. A.N Kluger and A. DeNisi "The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory," Psychol. Bull., vol.119, pp. 254-284, 1996. doi:10.1037/0033-2909.119.2.254
35. K.G. Brown "Using computers to deliver training: Which employees learn and why?," Person. Psychol., vol.54, pp. 271-296, 2001. doi:10.1111/j.1744-6570.2001.tb00093.x
36. W.L. Cron, J.W. Slocum, D. VandeWalle and Q. Fu "The role of goal orientation on negative emotions and goal setting when initial performance falls short of one's performance goal," Hum. Perfor., vol. 18, pp. 55-80, 2005. doi:10.1207/s15327043hup1801_3
37. E. Fox, R. Russo, R. Bowles and K. Dutton "Do threatening stimuli draw or hold attention in subclinical anxiety?," J. Exp. Psychol., vol. 130, pp. 681-700, 2001.
38. E Eich and J.W. Schooler "Cognition/emotion interactions," in Cognition and emotion, E. Eich, J.F. Kihlstrom, G.H. Bower, J.P. Forgas and P.M. Niedenthal, Eds. Oxford: Oxford University Press, 2000, pp. 3-29.
39. K.R. Scherer "Appraisal considered as a process of multi-level sequential checking," in Appraisal processes in emotion: Theory, Methods, Research, K.R. Scherer, A. Schorr and T. Johnstone, Eds. New York, NY: Oxford University Press, 2001, pp. 92-120.
40. N.H. Frijda, The Emotions. Cambridge: Cambridge University Press, 1986.
41. J.A. Coan and J.J.B. Allen, The handbook of emotion elicitation and assessment. New York: Oxford University Press, 2007.
42. M. Johnston and S. Bangalore "Finite-state Multimodal Integration and Understanding," J. Natur. Lang. Engin., vol. 11, pp. 159-187, 2005. doi:10.1017/S1351324904003572
43. J.A. Russell and A. Mehrabian "Environmental effects on drug use," J. Nonverb. Behav., vol 2, pp.109-123, 1977. doi:10.1007/ BF01145827
44. K.R. Scherer "Criteria for emotion-antecedent appraisal: A review," in Cognitive perspectives on emotion and motivation, V. Hamilton, G.H. Bower and N.H. Frijda, Eds. Dordrecht: Kluwer, 1988, pp. 89-126.
45. M.M Bradley and P.J. Lang "Measuring emotion: The Self-Assessment Manikin and the semantic differential," J. Behav. Ther. & Experim. Psychia., vol. 25(1), pp. 49-59, 1994.
46. R. Gary and D. Watson "Emotion, mood and temperament: similarities, differences and a synthesis," in Emotion at work: Theory, research and application in management, R.L Payne and C.L. Cooper, Eds. West Sussex, Uk: Wiley, 2001, pp. 21-43.
47. V. Nowlis, "Research with the Mood Adjective Check List," in Affect, cognition, and personality: Empirical studies, S.S. Tompkins and C.E. Izard, Eds. New York: Springer, 1965, pp. 352-389.
48. D.M. McNair, M. Lorr and L.F. Droppleman, Manual: profile of mood states. San Diego, CA: Educational and Industrial Testing Service, 1971.
49. M. Zuckerman and B. Lubin "Construction of new scales for the Multiple Affect Adjective Check List," J. Psychopath. Behav. Asses., vol.5, pp. 119-129, 1985.
50. D. Watson and L.A. Clark "The Panas-X: Manual for the Positive and Negative Affect Schedule- Expanded Form" unpublished
51. C.E. Izaard, D.Z. Libero, P. Putnam and O.M. Haynes "Stability of emotion experiences and their relations to traits of personality," J Soc Person Psych, vol. 64, pp. 847-860, 1993. doi:10.1037/ 0022-3514.64.5.847
52. A.Öhman, A. Hamm and K. Hugdahl "Cognition and the autonomic nervous system: Orienting, anticipation, and conditioning," in Handbook of psychophysiology, J.T. Cacioppo, L.G. Tassinary and G.G. Berntson, Eds. New York: Cambridge University Press, 2000, pp. 533-575.
53. G.G. Berntson and J.T. Cacioppo "From homeostasis to allodynamic regulation," in Handbook of psychophysiology, J.T. Cacioppo, L.G. Tassinari and G.G. Berntson, Eds. New York: Cambridge University Press, 2000, pp. 459-481.
54. A.Ohman, J.J. Soares "Unconscious anxiety: phobic responses to masked stimuli,". J. Abnorm. Psychol., vol. 103(2), 231-240, 1994. doi:10.1037/0021-843X.103.2.231
55. J.T. Cacioppo, G.G. Berntson, J.T. Larsen, K.M. Poehlmann and T.A. Ito "The psychophysiology of emotion," in The handbook of emotion, M. Lewis and J.M. Haviland-Jones, Eds. New York: Guildford Press, 2000.
56. P. Ekman, R.W. Levenson and W.V. Friesen "Autonomic nervous system activity distinguishes among emotions," Science, vol. 221(4616), pp. 1208-1210, 1983. doi:10.1126/science.6612338
57. I.C. Christie and B.H. Friedman "Autonomic specificity of discrete emotion and dimensions of affective space: A multivariate approach," Inter. J. Psychophys., vol. 51(2), pp. 143-153, 2004. doi:10.1016/j.ijpsycho.2003.08.002
58. G. Stemmler, M. Heldmann, C.A. Pauls and T. Scherer "Constraints for emotion specificity in fear and anger: The context counts," Psychophys., vol. 38(2), pp. 275-291, 2001. doi:10.1017/S0048577201991668
59. E. Duffy, Activation and behavior. New York: Wiley, 1962.
60. R.B. Malmo "Activation: A neuropsychological dimension," Psychol. Rev., vol. 66(6), pp. 367-386, 1959. doi:10.1037/h0047858
61. M.M. Bradley and P.J. Lang "Measuring emotion: Behavior, feeling, and physiology," in Cognitive neuroscience of emotion, R.D. Lane and L. Nadel, Eds. New York: Oxford University Press, 2000. pp. 242-276.
62. P.J. Lang, M.K. Greenwald, M.M. Bradley and A.O. Hamm "Looking at pictures: Affective, facial, visceral, and behavioral reactions," Psychophys.,vol. 30(3), pp. 261-273, 1993. doi:10.1111/j.1469-8986.1993.tb03352.x
63. M.B Arnold, Emotion and personality. New York: Columbia University Press, 1960.
64. W.B. Cannon "Again the James-Lange and the thalamic theories of emotion," Psychol. Rev., vol. 38(4), pp. 281-295, 1931. doi:10.1037/h0072957
65. C. Darwin, The expression of emotions in Man and Animals. London: Murray, 1872.
66. P. Ekman "Facial expressions of emotion: New findings, new questions," Psychol. Scien., vol. 3, pp. 34-38, 1992. doi:10.1111/j.1467-9280.1992.tb00253.x
67. C.E Izard, Human Emotions. New York: Plenum Press, 1977.
68. S.S Tomkins, Affect, Imagery and Consciousness: the Negative Affect. New York: Springer, 1962.
69. P. Ekman "Expression And The Nature Of Emotion," in Approaches to Emotion, K. Scherer and P. Ekman, Eds. Hillsdale New Jersey: Lawrence Erlbaum, 1984, pp. 319-344.
70. P. Ekman "Moods Emotions And Traits," in The Nature of Emotion: Fundamental Questions, P. Ekman and R. Davidson, Eds. New York: Oxford University Press, 1994, pp. 15-19.
71. P. Ekman and W.V. Friesen "Felt- False- And Miserable Smiles," J. Nonver. Behav., vol. 6(4), pp. 238-252, 1982. doi:10.1007/ BF00987191
72. N. Ravaja, K. Kallinen, T. Saari and L. Keltikangas-Järvinen "Suboptimal exposure to facial expressions when viewing video messages from a small screen: Effects on emotion, attention, and memory," .J Experim. Psychol. App., vol. 10, pp. 120-131, 2004.
73. C.V.O Witvliet and S.R. Vrana "Psychophysiological responses as indices of affective dimensions," Psychophys., vol. 32, pp. 436-443, 1995. doi:10.1111/j.1469-8986.1995.tb02094.x
74. Y. Dunham and M.R. Banaji "Platonic blindness and the challenge of understanding context," in The Mind in Context, B. Mesquita, L. Feldman-Barrett and E.R. Smith, Eds. New York: The Guilford Press, 2010, pp. 201-213.
75. L.F. Barrett and E.A. Kensinger "Context is routinely encoded during emotion perception," Psychol. Science, vol.21, pp. 595-599, 2010. doi:10.1177/0956797610363547
76. B. de Gelder "Towards the neurobiology of emotional body language,". Nat Rev., Neurosci., vol. 7 (3), pp. 242-249, 2006. doi:10.1038/nrn1872
77. T. Masuda, P.C. Ellsworth, B. Mesquita, J. Leu, S. Tanida and E. Van de Veerdonk. "Placing the Face in Context: Cultural Differences in the Perception of Facial Emotion," J. Personal. Social Psychol.,vol. 94, pp. 365-38, 2008. doi:10.1037/0022-3514.94.3.365
78. J.A. Russell and J.M. Fernández-Dols, The Psychology of Facial Expression. Cambridge: Cambridge University Press, 1997.
79. P. Ekman and W.V. Friesen, Manual for Facial Action Coding System. Palo Alto, CA: Consulting Psychology Press, 1978.
80. P. Ekman, W.V. Friesen and S.S Tomkins "Facial Affect Scoring Technique - A First Validity Study," Semio., vol. 3, pp. 37-58, 1971. doi:10.1515/semi.1971.3.1.37
81. C.E Izard, 1984 "Emotion Cognition Relationship and Human development," in Emotions in personality and psychopathology C.E. Izard, J. Kagan, R.B. and Zajonc, Eds. Cambridge: Cambridge University Press, 1984.
82. H. Als, E. Troninck and T.B. Brazelton "Affective reciprocity and the development of autonomy," J. Americ. Acad.Chil. Psychia., vol. 19, 1980.
83. P. Ekman, W.V. Friesen and J.C. Hager, Facial Action Coding System: the Manual. 2nd edition. Salt Lake City: Research Nexus eBook, 2002.
84. J.F. Cohn and P. Ekman "Measuring facial action by manual coding, facial EMG, and automatic facial image analysis," in Handbook of nonverbal behavior research methods in the affective sciences, J.A. Harrigan, R. Rosenthal and K. Scherer, Eds. New York: Oxford, 2005, pp. 9-64.
85. P. Ekman and E.L. Rosenberg, What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (2nd ed.). Oxford: Oxford University Press, 2005.
86. P.N. Juslin and P. Laukka "Communication of emotions in vocal expression and music performance: Different channels, same code?," Psychol. Bull., vol. 129, pp. 770-814, 2003. doi:10.1037/0033-2909.129.5.770
87. K.R. Scherer "Vocal affect expression: A review and a model for future research," Psychol. Bull., vol. 99, pp. 143-165, 1986. doi:10.1037/0033-2909.99.2.143
88. K.R. Scherer "Vocal communication of emotion: A review of research paradigms," Speech Communn., vol. 40, pp. 227-256, 2003. doi:10.1016/S0167-6393(02)00084-5
89. A.R. Damasio, Descarte's Error: Emotion, Reason and Human Brain. New York: Grosset Putnam,1994.
90. M. Kipp and J.C. Martin "Gesture and Emotion: Can basic gestural form features discriminate emotions?," in Proc. Int. Conf. Affect. Compu. Intellig. Interact., 2009.
91. H. Walbott "Bodily expression of emotion. European," J. Soc. Psychol., vol. 28, pp. 879-896, 1998.
92. D. McNeill, Hand and Mind: What Gesture Reveals About Thought. Chicago: University of Chicago Press, 1992.
93. A. Kendon, Nonverbal Communication, Interaction and Gesture: Selections from Semiotica. The Hague: Mouton and Co, 1981 doi:10.1515/9783110880021
94. F.D. Fincham "Marital conflict: Correlates, structure and context," Cur. Direct. Psychol. Scien., vol.12, pp.23-27, 2003. doi:10.1111/1467-8721.01215
95. J.P. Vincent, L.C. Friedman, J. Nugent and L. Messerly "Demand characteristics in observations of marital interaction," J. Consul. Clinic., vol. 47, pp. 557-566, 1979. doi:10.1037/0022-006X.47.3.557
96. L. Anolli "The detection of the hidden design of meaning," in The hidden structure of interaction. From neurons to culture patterns, L. Anolli, S. Duncan Jr., M.S. Magnusson and G. Riva, Eds. Amsterdam: IOS, 2005, pp. 23-50.
97. B. Woolf, W. Burleson, I. Arroyo, T. Dragon, D. Cooper and R. Picard "Affect-aware tutors: recognising and responding to student affect," J. Learn. Technol. Inder. Enterpr., vol 4, pp. 129-163, 2009. doi:10.1504/IJLT.2009.028804
98. S. D'Mello, T. Jackson, S. Craig, B. Morgan, P. Chipman, H. White, et. al. "Auto Tutor Detects and Responds to Learners Affective and Cognitive States," Work. Emotion. Cognitiv. Issu. Int. Conf. Intell. Tutor. Syst., Montreal, Canada, Jun 2008.
99. A. Drachen, L.E. Nacke, G.N. Yannakakis and A.L. Pedersen "Correlation between Heart Rate, Electrodermal Activity and Player Experience in First-Person Shooter Games,"Proc. ACM SIGGRAPH, 2010.
100. R.W. Picard, Affective Computin. Cambridge: The Mit Press, 1997.
101. E. Hudlicka "Affective game engines: motivation and requirements," Proc. 4th Int.Conf. Found. of Digit. Game. Orlando, Florida, 2009.
102. K. Gileade and A. Dix "Using frustration in the design of adaptive videogames," Proc. of ACE 2004, pp. 228-232, 2004. doi:10.1145/1067343.1067372
103. R. Nakatsu, M. Rauterberg and P. Vorderer "A new framework for entertainment computing: from passive to active experience," in Entertainment Computing, F. Kishino et al. Eds. Lecture Notes in Computer Science, vol. 3711, 2005, pp. 1-12. doi:10.1007/ 11558651_1
104. J. Sykes and S. Brown "Affective gaming: Measuring emotion through the Gamepad," New Hor., pp. 732-733, 2003.
105. K. Gilleade and J. Allanson "A Toolkit for exploring affective interface adaptation in videogames," Proc. Hum.-Comput. Interact. Int., Crete, Greece, 2003.
106. W.M. van den Hoogen, W.A. IJsselsteijn, Y.A.W de Kort and K. Poels. "Toward real-time behavioral indicators of player experiences: Pressure patterns and postural responses," Proc. Meas. Behav., Maastricht, The Netherlands, pp. 100-101, 2008.
107. R.J. Pagulayan, K. Keeker, D. Wixon, R. Romero and T. Fuller "User-centered design in games," in Handbook for HumaniJET Computer Interaction in Interactive Systems, J. Jacko and A. Sears, Eds. Erlbaum, 2003, p. 883-906.
108. L. Nacke, A. Drachen and S. Goebel "Methods for Evaluating Gameplay Experience in a Serious Gaming Context," Int. J. Comput. Scien. Sport, vol. 9, 2010.
109. C. Becker, A. Nakasone, H. Prendinger, M. Ishizuka and I. Wachsmuth "Physiologically interactive gaming with the 3D agent Max," Int. Work. Conver.Inform, Kitakyushu, Japan, pp. 37-42, 2005.
110. M. Ben Moussa and N. Magnenat-Thalmann "Applying Affect Recognition in Serious Games: The PlayMancer Project," 2nd. Int. Work. Motion Game., pp. 53-62, 2009.
111. O.Kocsis, T. Ganchev, I. Mporas, G. Papadopoulos and N. Fakotakis "Multi-modal System Architecture for Serious Gaming," in Advan. Inform. Commun. Techn., vol. 296, pp. 441-447, 2009.