Emotion assessment from physiological signals for adaptation of game difficulty

IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

Volumn 41, Issue 6, 2011, Pages 1052-1063

  Chanel, Guillaume ^a   Rebetez, Cyril ^a   Bétrancourt, Mireille ^a   Pun, Thierry ^a  

^a UNIVERSITY OF GENEVA   (Switzerland)

Author keywords

Electroencephalography; emotion assessment; games; pattern classification; signal analysis

Indexed keywords

ELECTROENCEPHALOGRAM SIGNALS; EMOTION ASSESSMENT; GAMES; PHYSIOLOGICAL SIGNALS; SHORT PERIODS; TETRIS GAME;

ELECTROENCEPHALOGRAPHY; ELECTROPHYSIOLOGY;

RATING;

EID: 80054805114     PISSN: 10834427     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCA.2011.2116000     Document Type: Article

References (44)

1. M. Prensky, "Computer games and learning: Digital game-based learning," in Handbook of Computer Games Studies, J. Raessens and J. Goldstein, Eds. Cambridge, MA: MIT Press, 2005.
2. R. W. Picard, Affective Computing. Cambridge, MA: MIT Press, 1997.
3. C. Liu, P. Agrawal, N. Sarkar, and S. Chen, "Dynamic difficulty adjustment in computer games through real-time anxiety-based affective feedback," Int. J. Human-Comput. Interact., vol. 25, no. 6, pp. 506-529, Aug. 2009.
4. K. R. Scherer, Appraisal Considered as a Process of Multi-Level Sequential Checking. Oxford, U.K.: Oxford Univ. Press, 2001.
5. R. W. Picard, E. Vyzas, and J. Healey, "Toward machine emotional intelligence: Analysis of affective physiological state," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 10, pp. 1175-1191, Oct. 2001.
6. C. L. Lisetti and F. Nasoz, "Using noninvasive wearable computers to recognize human emotions from physiological signals," J. Appl. Signal Process., no. 11, pp. 1672-1687, 2004.
7. P. Rani, C. Liu, N. Sarkar, and E. Vanman, "An empirical study of machine learning techniques for affect recognition in human-robot interaction," Pattern Anal. Appl., vol. 9, no. 1, pp. 58-69, May 2006.
8. R. L. Mandryk and M. S. Atkins, "A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies," Int. J. Human-Comput. Stud., vol. 65, no. 4, pp. 329-347, Apr. 2007.
9. G. Chanel, C. Rebetez, M. Bétrancourt, and T. Pun, "Boredom, engagement and anxiety as indicators for adaptation to difficulty in games," in Proc. 12th Int. MindTrek Conf.: Entertainment Media Ubiquitous Era, 2008, pp. 13-17.
10. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor, "Emotion recognition in human-computer interaction," IEEE Signal Process. Mag., vol. 18, no. 1, pp. 32-80, Jan. 2001.
11. D. Sander, D. Grandjean, and K. R. Scherer, "A systems approach to appraisal mechanisms in emotion," Neural Netw., vol. 18, no. 4, pp. 317-a352, May 2005.
12. R. J. Davidson, "Affective neuroscience and psychophysiology: Toward a synthesis," Psychophysiology, vol. 40, no. 5, pp. 655-665, Sep. 2003.
13. L. I. Aftanas, N. V. Reva, A. A. Varlamov, S. V. Pavlov, and V. P.Makhnev, "Analysis of evoked EEG synchronization and desynchronization in conditions of emotional activation in humans: Temporal and topographic characteristics," Neurosci. Behav. Physiol., vol. 34, no. 8, pp. 859-867, Oct. 2004.
14. A. P. R. Smith, K. E. Stephan, M. D. Rugg, and R. J. Dolan, "Task and content modulate amygdala-hippocampal connectivity in emotional retrieval," Neuron, vol. 49, no. 4, pp. 631-638, Feb. 2006.
15. K. Takahashi, "Remarks on emotion recognition from bio-potential signals," in Proc. 2nd Int. Conf. Auton. Robots Agents, Palmerston North, New Zealand, 2004.
16. G. Chanel, J. J. M. Kierkels, M. Soleymani, and T. Pun, "Short-term emotion assessment in a recall paradigm," Int. J. Human-Comput. Stud., vol. 67, no. 8, pp. 607-627, Aug. 2009.
17. F. G. Freeman, P. J. Mikulka, M. W. Scerbo, and L. Scott, "An evaluation of an adaptive automation system using a cognitive vigilance task," Biol. Psychol., vol. 67, no. 3, pp. 283-297, Nov. 2004.
18. A. T. Pope, E. H. Bogart, and D. S. Bartolome, "Biocybernetic system evaluates indexes of operator engagement in automated task," Biol. Psychol., vol. 40, no. 1/2, pp. 187-195, May 1995.
19. M. Besserve, M. Philippe, G. Florence, F. Laurent, L. Garnero, and J. Martinerie, "Prediction of performance level during a cognitive task from ongoing EEG oscillatory activities," Clin. Neurophysiol., vol. 119, no. 4, pp. 897-908, Apr. 2008.
20. C. Berka, D. J. Levendowski, M. M. Cvetinovic, M. M. Petrovic, G. Davis, M. N. Lumicao, V. T. Zivkovic, M. V. Popovic, and R. Olmstead, "Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset," Int. J. Human-Comput. Interact., vol. 17, no. 2, pp. 151-170, Jun. 2004.
21. G. F.Wilson and C. A. Russell, "Real-time assessment ofmental workload using psychophysiological measures and artificial neural networks," Hum. Factors, vol. 45, no. 4, pp. 635-643, Winter 2003.
22. S. H. Fairclough, "Psychophysiological inference and physiological computer games," in Proc. Brainplay: Brain-Comput. Interfaces Games, Workshop Int. Conf. Adv. Comput. Entertainment, 2007.
23. P. Rani, N. Sarkar, and C. Liu, "Maintaining optimal challenge in computer games through real-time physiological feedback," in Proc. 11th HCI Int., Las Vegas, NV, 2005, pp. 184-192.
24. M. Csikszentmihalyi, Flow: The Psychology of Optimal Experience. New York: Harper Collins, 1991.
25. K. Salen and E. Zimmerman, Rules of Play: Game Design Fundamentals. Cambridge, MA: MIT Press, 2004.
26. R. Oostenveld and P. Praamstra, "The five percent electrode system for high-resolution EEG and ERP measurements," Clin. Neurophysiol., vol. 112, no. 4, pp. 713-719, Apr. 2001.
27. J. D. Morris, "SAM: The self-assessment manikin, an efficient crosscultural measurement of emotional response," J. Advertising Res., vol. 35, no. 6, pp. 63-68, Nov. 1995.
28. G. Stemmler, M. Heldmann, C. A. Pauls, and T. Scherer, "Constraints for emotion specificity in fear and anger: The context counts," Psychophysiology, vol. 38, no. 2, pp. 275-291, Mar. 2001.
29. D. J. McFarland, L. M. McCane, S. V. David, and J. R. Wolpaw, "Spatial filter selection for EEG-based communication," Electroencephalogr. Clin. Neurophysiol., vol. 103, no. 3, pp. 386-394, Sep. 1997.
30. P. J. Lang, M. K. Greenwald, M. M. Bradley, and A. O. Hamm, "Looking at pictures: Affective, facial, visceral, and behavioral reactions," Psychophysiology, vol. 30, no. 3, pp. 261-273, May 1993.
31. J. A. Healey, "Wearable and automotive systems for affect recognition from physiology," Ph.D. dissertation, Cambridge, MA, 2000.
32. P. Rainville, A. Bechara, N. Naqvi, and A. R. Damasio, "Basic emotions are associated with distinct patterns of cardiorespiratory activity," Int. J. Psychophysiol., vol. 61, no. 1, pp. 5-18, Jul. 2006.
33. G. G. Berntson, J. T. Bigger, Jr., D. L. Eckberg, P. Grossman, P. G. Kaufmann, M. Malik, H. N. Nagaraja, S. W. Porges, J. P. Saul, P. H. Stone, and M. W. van der Molen, "Heart rate variability: Origins, methods, and interpretive caveats," Psychophysiology, vol. 34, no. 6, pp. 623-648, Nov. 1997.
34. J. Kim, "Emotion recognition from physiological measurement," in Proc. Humaine Eur. Netw. Excellence Workshop, 2004.
35. P. Ekman, R. W. Levenson, and W. V. Friesen, "Autonomic nervoussystem activity distinguishes among emotions," Science, vol. 221, no. 4616, pp. 1208-1210, Sep. 1983.
36. R. A. McFarland, "Relationship of skin temperature changes to the emotions accompanying music," Appl. Psychophysiol. Biofeedback, vol. 10, pp. 255-267, Sep. 1985.
37. C. M. Bishop, Pattern Recognition and Machine Learning. NewYork: Springer-Verlag, 2006.
38. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York: Wiley-Interscience, 2001.
39. L. Yu and H. Liu, "Efficient feature selection via analysis of relevance and redundancy," J. Mach. Learn. Res., vol. 5, pp. 1205-1224, 2004.
40. G. Chanel, K. Ansari-Asl, and T. Pun, "Valence-arousal evaluation using physiological signals in an emotion recall paradigm," in Proc. IEEE SMC Int. Conf. Syst., Man, Cybern., Smart Cooperative Syst. Cybern.: Adv. Knowl. Security Humanity, 2007, pp. 2662-2667.
41. P. Pudil, F. J. Ferri, J. Novovicova, and J. Kittler, "Floating search methods for feature selection with nonmonotonic criterion functions," in Proc. IEEE Int. Conf. Pattern Recog., 1994, vol. 2, pp. 279-283.
42. C. Sanderson and K. K. Paliwal, "Identity verification using speech and face information," Dig. Signal Process., vol. 14, no. 5, pp. 449-480, Sep. 2004.
43. D. Ruta and B. Gabrys, "An overview of classifier fusion methods," Comput. Inf. Syst., vol. 7, no. 1, pp. 1-10, Feb. 2000.
44. C. D. Katsis, N. Katertsidis, G. Ganiatsas, and D. I. Fotiadis, "Toward emotion recognition in car racing drivers: A biosignal processing approach," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no. 3, pp. 502-512, May 2008.