A two-dimensional facial-affect estimation system for human-robot interaction using facial expression parameters

Advanced Robotics

Volumn 27, Issue 4, 2013, Pages 259-273

  Schacter, David ^a   Wang, Christopher ^a   Nejat, Goldie ^a   Benhabib, Beno ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

automated affect estimation; facial expressions; human robot interaction; support vector regression

Indexed keywords

FACIAL EXPRESSION PARAMETERS; FACIAL EXPRESSIONS; PHYSIOLOGICAL SIGNALS; REAL-TIME ESTIMATION; REAL-WORLD SCENARIO; SOCIAL HUMAN-ROBOT INTERACTIONS; SUPPORT VECTOR REGRESSION (SVR); TWO DIMENSIONAL MODEL;

HUMAN COMPUTER INTERACTION; HUMAN ROBOT INTERACTION; MAN MACHINE SYSTEMS; TWO DIMENSIONAL;

ESTIMATION;

EID: 84876165788     PISSN: 01691864     EISSN: 15685535     Source Type: Journal    
DOI: 10.1080/01691864.2013.755278     Document Type: Article

References (45)

1. Papousek I, Freudenthaler HH, Schulter G. The interplay of perceiving and regulating emotions in becoming infected with positive and negative moods. Pers. Indiv. Diff. 2008;45:463-467.
2. Ekman P. Emotions revealed: recognizing faces and feeling to improve communication and emotional life. New York (NY): Times Books; 2003.
3. Ekman P. Emotion in the human face. Cambridge: Cambridge University Press; 1982.
4. Darwin C. The expressions of emotions in man & animals. 1st ed. New York (NY): Philosophical Library; 1872.
5. Posner J, Russell JA, Peterson BS. The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development and psychopathology. Dev. Psychopathol. 2005;17:715-734.
6. Plutchik R. A general psychoevolutionary theory of emotion, Emotion: theory, research, and experience, Vol. 1. In: Plutchik R, Kellerman H, editors. Theories of emotion. New York (NY): Academic; 1980. 3-33.
7. Russell JA. A circumplex model of affect. J. Personality Social Psychol. 1980;39:1161-1178.
8. Pantic M, Rothkrantz LJM. Toward an affect-sensitive multimodal human-computer interaction. Proc. IEEE. 2003;91:370-1390.
9. Al-Hamadi A, Niese R, Pathan SS, Michaelis B. Geometric and optical flow based method for facial expression recognition in color image sequences. In: Bolc L, Kulikowski JL, Wojciechowski K, editors. Lecture notes in computer science. Proceedings of the ICCVG. Berlin: Springer-Verlag; 2009. 228-38.
10. Anderson K, McOwan PW. A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybernetics Part B. 2006;36:96-105. (Pubitemid 43174144)
11. Cohen I, Ashutosh G, Huang TS. Emotion recognition from facial expressions using multilevel HMM. Neural Inf. Proc. Syst. 2000;2.
12. Bartlett MS, Littlewort G, Fasel I, Movellan Jr. Real time face detection and facial expression recognition: development and applications to human computer interaction. Conf. Comput. Vision Pattern Recognit. 2003;5:53.
13. Ioannou SV, Raouzaiou AT, Tzouvaras VA, Mailis TP, Karpouzis KC, Kollias SD. Emotion recognition through facial expression analysis based on a neurofuzzy network. Neural Networks 2005;18:423-435. (Pubitemid 40922649)
14. Shin Y-S. Facial expression recognition of various internal states via manifold learning. J. Comput. Sci. Technol. 2009;24:745-752.
15. Shin Y-S. Facial expression recognition based on the two dimensional structure of affect. In: Fred A, Caelli TM, Duin RPW, Campilho AC, de Ridder D, editors. Lecture notes in computer science SSPR&SPR. Berlin: Springer- Verlag; 2004. 912-20. (Pubitemid 39743078)
16. Shin Y-S. Facial expression recognition in various internal states using independent component analysis. In: Perales FJ, Fisher RB, editors. Lecture notes in computer science. International Conference on AMDO, Vol. 4069. Berlin: Springer-Verlag; 2006. 291-9.
17. Cao J, Wang H, Hu P, Miao J. PAD model based facial expression analysis. In: Bebis G et al, editors. International symposium on advances in visual computing. Berlin: Springer-Verlag; 2008. 450-9.
18. Kanluan I, Grimm M, Kroschel K. Audio-visual emotion recognition using an emotion recognition space concept. Sixteenth European Signal Processing Conference; 2008; Lausanne, Switzerland.
19. Nicolaou MA, Gunes H, Pantic M. Continuous prediction of spontaneous affect from multiple cues and modalities in valence arousal space. IEEE Trans. Affective Comput. 2011;2:1-15.
20. Konishi Y, Kinoshita K, Lao S, Kawade M. Real-time estimation of smile intensity. Asian Conference on Computer Vision; 2007; Tokyo, Japan. p. 1.
21. Jang ES, Capin T, Osternamm J. Visual SNHC tools. In: Pereira F, Ebrahimi T, editors. The M-PEG4 book (Prentice Hall IMSC Multimedia Series); Upper Saddle River, NJ: Prentice-Hall; 2002.
22. Chouvatut V, Madarasmi S, Tuceryan M. Face reconstruction and camera pose using multi-dimensional descent. World Acad. Sci. Eng. Technol. 2009;60:730-735.
23. Lei L, Miao Z. Individual facial expression transferring using active appearance model and FAP in MPEG-4. In: International Conference on Image Analysis and Signal Processing; 2010; Zhejiang, China. p. 150-154.
24. Lucey S, Wang Y, Cox M, Sridharan S, Cohn J. Efficient constrained local model fitting for non rigid face alignment. Image Vision Comput. 2009;27: 1804-1813.
25. Zhu Z, Ji Q. Robust pose invariant facial feature detection and tracking in real-time. In: International Conference on Pattern Recognition; 2006; Hong Kong. p. 1092-1095. (Pubitemid 46532089)
26. Wang Y, Ai H, Wu B, Huang C. Real time facial expression recognition with adaboost. International Conference on Pattern Recognition; 2004; Cambridge, UK. p. 926- 929.
27. Valstar MF, Martinez B, Binefa X, Pantic M. Facial point detection using boosted regression and graph models. In: IEEE International Conference on Computer Vision and Pattern Recognition; 2010; San Francisco, USA. p. 2729- 2736.
28. Rudovic O, Patras I, Pantic M. Coupled Gaussian process regression for pose-invariant facial expression recognition. In: European Conference on Computer Vision; 2010; Crete, Greece. p. 350-363.
29. Pizarro D, Peyras J, Bartoli A. Light-invariant fitting of active appearance models. In: IEEE Conference on Computer Vision and Pattern Recognition; 2008; Anchorage, AK. p. 1-6.
30. Lyons MJ, Akamatsu S, Kamachi M, Gyoba J. Coding facial expressions with gabor wavelets. In: IEEE International Conference on Automatic Face and Gesture Recognition; 1998; Nara, Japan. p. 200-205.
31. Cristinacce D., Cootes T.. Automatic feature localization with constrained local models. Pattern Recognit. 2008;41:3054-3067.
32. Gower JC. Generalized procrustes analysis. Psychometrika. 1975;40:33-51.
33. Pantic M, Rothkrantz LJM. An expert system for multiple emotional classification of facial expressions. In: IEEE International Conference on Tools with Artificial Intelligence; 1999; Chicago, IL. p. 113-120. (Pubitemid 30526062)
34. Kobayashi H, Hara F. The recognition of basic facial expressions by neural network. In: IEEE International Joint Conference on Neural Networks; 1991; Singapore. p. 460-466.
35. Huang C-L, Huang Y-M. Facial expression recognition using model-based feature extraction and action parameters classification. J. Visual Commun. Image Represent. 1997;8:278-290. (Pubitemid 127435634)
36. Vapnik VN. Statistical learning theory. New York (NY): Wiley-Interscience; 1998.
37. Smola AJ, Scholkopf B. A tutorial on support vector regression. Stat. Comput. 2004;14:199-222. (Pubitemid 39063488)
38. Clarke SM, Griebsch JH, Simpson TW. Analysis of support vector regression for approximation of complex engineering analyses. J. Mech. Des. 2005;127:1077-1087. (Pubitemid 41709565)
39. Kim K-J. Financial time series forecasting using support vector machines. Neurocomputing. 2003;55:307-319. (Pubitemid 37336681)
40. Fan C, Sarrafzadeh H, Dadgostar F, Gholamhosseini H. Facial expression analysis by support vector regression. In: Image and Vision Computing New Zealand Conference; 2005; Dunedin, New Zealand. p. 311-316.
41. Fletcher R. Practical methods of optimization. New York (NY): John Wiley and Sons; 1989.
42. Smets K, Verdonk B, Jordaan EM. Evaluation of performance measures for SVR hyperparameter selection. In: International Joint Conference on Neural Networks; 2007; Orlando, FL. p. 637-642.
43. Rottenberg J, Ray R, Gross J. Emotion elicitation using film clips. In: Coan JA, Allen JJB, editors. The handbook of emotion elicitation and assessment. New York (NY): Oxford University Press; 2007. 9-28. Chapter 2, Section 1.
44. Chuang C-C, Su S-F, Jeng J-T, Hsiao C-C. Robust support vector regression networks for function approximation with outliers. IEEE Trans. Neural Networks. 2002;13:1322-1330. (Pubitemid 35428522)
45. Gunes H, Pantic M. Automatic, dimensional and continuous emotion recognition. Int. J. Synth. Emotions. 2010;1:68-99.