Classification of affects using head movement, skin color features and physiological signals

Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

Volumn , Issue , 2012, Pages 2664-2669

  Monkaresi, Hamed ^a   Hussain, M Sazzad ^a   Calvo, Rafael A ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Affective computing; machine learning; multichannel physiology; multimodal fusion; video analysis

Indexed keywords

AFFECTIVE COMPUTING; AUTOMATED DETECTION; AUTOMATED TECHNIQUES; DETECTION TECHNIQUE; FACIAL COLOR; FUSION MODEL; HEAD MOVEMENTS; MENTAL HEALTH; MULTI-CHANNEL; MULTI-MODAL FUSION; MULTIPLE MODALITIES; NEW APPLICATIONS; PHYSIOLOGICAL MEASURES; PHYSIOLOGICAL RESPONSE; PHYSIOLOGICAL SIGNALS; PICTURE SYSTEM; SKIN COLOR; USER-DEPENDENT; USERS' AFFECTIVE STATE; VIDEO ANALYSIS;

CYBERNETICS; HUMAN COMPUTER INTERACTION; LEARNING SYSTEMS; PHYSIOLOGICAL MODELS;

PHYSIOLOGY;

EID: 84872371733     PISSN: 1062922X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICSMC.2012.6378149     Document Type: Conference Paper

References (24)

1. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, "A survey of affect recognition methods: audio, visual, and spontaneous expressions.," IEEE transactions on pattern analysis and machine intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
2. R. A. Calvo and S. D'Mello, "Affect detection: an interdisciplinary review of models, methods, and their applications," Affective Computing, IEEE Transaction on, vol. 1, no. 1, pp. 18-37, 2010.
3. S. D. Craig, S. D'Mello, A. Witherspoon, and A. Graesser, "Emote aloud during learning with AutoTutor: Applying the Facial Action Coding System to cognitive-affective states during learning," Cognition & Emotion, vol. 22, no. 5, pp. 777-788, Aug. 2008. (Pubitemid 351983084)
4. B. McDaniel, S. D'Mello, B. King, P. Chipman, K. Tapp, and A. Graesser, "Facial features for affective state detection in learningenvironments, " in 29th Annual meeting of the cognitive science society, 2007, pp. 467-472.
5. R. Sharma, V. I. Pavlovic, and T. S. Huang, "Toward multimodal human-computer interface," Proceedings of the IEEE, vol. 86, no. 5, pp. 853-869, 1998.
6. A. Jaimes and N. Sebe, "Multimodal human-computer interaction: A survey," Computer Vision and Image Understanding, vol. 108, no. 1-2, pp. 116-134, Oct. 2007.
7. S. D'Mello and A. Graesser, "Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features," User Modeling and User-Adapted Interaction, vol. 20, no. 2, pp. 147-187, May 2010.
8. S. Hussain, R. A. Calvo, and P. A. Pour, "Hybrid fusion approach for detecting affects from multichannel physiology," in Affective Computing and Intelligent Interaction, S. D'Mello, A. Graesser, B. Schuller, and J.-C. Martin, Eds. Springer, 2011, pp. 568-577.
9. P. Lang and M. Bradley, "International affective picture system (IAPS): Technical manual and affective ratings," Psychology, 1997.
10. P. Ekman, R. J. Davidson, and W. V. Friesen, "The Duchenne smile: emotional expression and brain physiology II.," Journal of personality and social psychology, vol. 58, no. 2, pp. 342-53, Feb. 1990.
11. P. Ekman, Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage. New York: W. W. Norton & Company, 2001.
12. J. A. Russell, "A Circumplex model of affect," Journal of Personality and Social Psychology, vol. 39, no. 6. pp. 1161-1178, 1980.
13. J. Wagner, J. Kim, and E. Andre, "From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification," in 2005 IEEE International Conference on Multimedia and Expo, 2005, pp. 940-943.
14. M. Pantic and L. J. M. Rothkrantz, "Toward an affect-sensitive multimodal human-computer interaction," Proceedings of the IEEE, vol. 91, no. 9, pp. 1370-1390, Sep. 2003. (Pubitemid 40890819)
15. C. Busso et al., "Analysis of emotion recognition using facial expressions, speech and multimodal information," in Proceedings of the 6th International Conference on Multimodal Interfaces, 2004, pp. 205-211.
16. J. Kim, "Bimodal emotion recognition using speech and physiological changes," Robust Speech Recognition and Understanding, pp. 265-280, 2007.
17. A. Kapoor and R. W. Picard, "Multimodal affect recognition in learning environments," in Proceedings of the 13th annual ACM international conference on Multimedia, 2005, pp. 677-682.
18. R. Picard, "Affective computing: challenges," International Journal of Human-Computer Studies, vol. 59, no. 1-2, pp. 55-64, Jul. 2003.
19. R. Lienhart and J. Maydt, "An extended set of Haar-like features for rapid object detection," in IEEE ICIP 2002, 2002, vol. 1, pp. 900-903.
20. M. A. Hall, "Correlation-based feature selection for discrete and numeric class machine learning," in ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning, 2000, pp. 359-366.
21. I. H. Witten and E. Frank, Data mining: practical machine learning tools and techniques. Morgan Kaufmann, 2005, p. 525.
22. L. I. Kuncheva, Combining pattern classifiers: methods and algorithms. John Wiley & Sons, 2004, p. 350.
23. J. Cohen, "A coefficient of agreement for nominal scales," Educational and Psychological Measurement, vol. 20, no. 1, pp. 37-46, Apr. 1960.
24. M. A. Nicolaou, H. Gunes, and M. Pantic, "Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space," Affective Computing, IEEE Transactions on, vol. 2, no. 2, pp. 92-105, 2011.