3D shape estimation in video sequences provides high precision evaluation of facial expressions

Image and Vision Computing

Volumn 30, Issue 10, 2012, Pages 785-795

  Jeni, László A ^a   Lorincz, András ^b   Nagy, Tamás ^b   Palotai, Zsolt ^c   Sebok, Judit ^b   Szabó, Zoltán ^b   Takács, Dániel ^b,d  

^a UNIVERSITY OF TOKYO   (Japan)

^b EÖTVÖS LORÁND UNIVERSITY   (Hungary)

^c Sparsense Inc ^*  (United States)

^d Realeyes Data Services Ltd   (United Kingdom)

Author keywords

Action unit recognition; BU 4DFE; Constrained Local Model; Emotion classification; Shape information

Indexed keywords

CLASSIFICATION (OF INFORMATION); DATABASE SYSTEMS; ESTIMATION; FACE RECOGNITION; THREE DIMENSIONAL COMPUTER GRAPHICS; VIDEO RECORDING;

ACTION UNIT; BU-4DFE; CK; EMOTION CLASSIFICATION; LOCAL MODEL; SHAPE INFORMATION;

THREE DIMENSIONAL;

EID: 84866737321     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2012.02.003     Document Type: Article

References (35)

1. P. Lucey, J. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews The Extended Cohn-Kanade Dataset (CK+): a complete dataset for action unit and emotion-specified expression 3rd IEEE Workshop on CVPR for Human Communicative Behavior Analysis (CVPR4HB10) 2010 94 101
2. Y. Tong, W. Liao, and Q. Ji Facial action unit recognition by exploiting their dynamic and semantic relationships IEEE Trans. Pattern Anal. Mach. Intell. 29 10 2007 1683 1699
3. L. Yin, X. Chen, Y. Sun, T. Worm, and M. Reale A high-resolution 3D dynamic facial expression database 8th IEEE International Conference on Automatic Face Gesture Recognition, IEEE 2008 1 6 tracking Number: 66
4. D. Cristinacce, and T. Cootes Automatic feature localisation with constrained local models Pattern Recognit. 41 10 2008 3054 3067 10.1016/j.patcog.2008.01.024
5. J.M. Saragih, S. Lucey, and J.F. Cohn Deformable model fitting by regularized landmark mean-shift Int. J. Comput. Vis. 91 2 2011 200 215
6. P. Lucey, J.F. Cohn, K.M. Prkachin, P. Solomon, and I. Matthews Painful data: the UNBC-McMaster shoulder pain expression archive database 9th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2011) 2011 57 64
7. S.W. Chew, P.J. Lucey, S. Lucey, J. Saragih, J.F. Cohn, and S. Sridharan Person-independent facial expression detection using constrained local models IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), FERA 2011 Workshop on Facial Expression Recognition and Analysis Challenge 2011 915 920
8. I. Matthews, and S. Baker Active appearance models revisited Int. J. Comput. Vis. 60 2 2004 135 164
9. S. Taheri, P. Turaga, and R. Chellappa Towards view-invariant expression analysis using analytic shape manifolds IEEE International Conference on Automatic Face and Gesture Recognition (FG'11) 2011 306 313
10. L.A. Jeni, H. Hashimoto, and A. Lorincz Efficient, pose invariant facial emotion classification using constrained local model and 2D shape information Extended Abstracts, Workshop on Gesture Recognition at CVPR 2011 2011 extended abstract available at http://clopinet.com/isabelle/Projects/CVPR2011
11. L.A. Jeni, D. Takacs, and A. Lorincz High quality facial expression recognition in video streams using shape related information only IEEE International Workshop on Benchmarking Facial Image Analysis Technologies at ICCV 2011
12. C.-C. Chang, and C.-J. Lin LIBSVM: a library for support vector machines ACM Trans. Intell. Syst. Technol. 2 2011 27:1 27:27 software available at http://www.csie.ntu.edu.tw/cjlin/libsvm
13. J. Shawe-Taylor, and N. Cristianini Kernel Methods for Pattern Analysis 2004 Cambridge University Press
14. T. Kanade, J. Cohn, and Y. Tian Comprehensive database for facial expression analysis IEEE International Conference on Automatic Face and Gesture Recognition (FG'00) 2000 46 53
15. J.M. Susskind, G. Littlewort, M.S. Bartlett, J. Movellan, and A.K. Anderson Human and computer recognition of facial expressions of emotion Neuropsychologia 45 1 2007 152 162
16. P. Ekman What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS) 2005 Oxford University Press USA
17. Y. Sun, M. Reale, and L. Yin Recognizing partial facial action units based on 3d dynamic range data for facial expression recognition IEEE International Conference on Automatic Face and Gesture Recognition (FG'08) 2011 1 8
18. Y. Sun, and L. Yin Facial expression recognition based on 3d dynamic range model sequences European Conference on Computer Vision 2008 58 71
19. Y. Sun, X. Chen, M. Rosato, and L. Yin Tracking vertex flow and model adaptation for three-dimensional spatiotemporal face analysis IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 40 3 2010 461 474
20. V. Le, H. Tang, and T. Huang Expression recognition from 3d dynamic faces using robust spatio-temporal shape features IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), FERA 2011 Workshop on Facial Expression Recognition and Analysis Challenge 2011 414 421
21. G. Sandbach, S. Zafeiriou, M. Pantic, and D. Rueckert A dynamic approach to the recognition of 3d facial expressions and their temporal models IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), FERA 2011 Workshop on Facial Expression Recognition and Analysis Challenge 2011 406 413
22. A. Lorincz, G. Szirtes, B. Takács, I. Biederman, and R. Vogels Relating priming and repetition suppression Int. J. Neural Syst. 12 2002 187 202
23. W. Zheng, H. Tang, Z. Lin, and T.S. Huang Emotion recognition from arbitrary view facial images European Conference on Computer Vision 2010 490 503
24. O. Rudovic, I. Patras, and M. Pantic Coupled Gaussian process regression for pose-invariant facial expression recognition European Conference on Computer Vision 2010 350 363
25. M. Bartlett, G. Littlewort-Ford, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan Recognizing facial expression: machine learning and application to spontaneous behavior IEEE Conference on Computer Vision and Pattern Recognition 2005 568 573
26. M. Bartlett, G. Littlewort-Ford, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan Fully automatic facial action recognition in spontaneous behavior IEEE Conference on Face and Gesture Recognition (FG'06) 2006 223 230
27. Y. Chang, C. Hu, R. Feris, and M. Turk Manifold-based analysis of facial expression Image Vision Comput. 24 6 2006 605 614
28. J. Whitehill, and C. Omlin Haar features for FACS AU recognition IEEE Conference on Face and Gesture Recognition (FG'06) 2006 97 101
29. G. Littlewort, M. Bartlett, I. Fasel, J. Susskind, and J. Movellan Dynamics of facial expression extracted automatically from video Image Vision Comput. 24 6 2006 615 625
30. M. Valstar, and M. Pantic Fully automatic facial action unit detection and temporal analysis IEEE Conference on Computer Vision and Pattern Recognition Workshop 2006 149 156
31. S. Koelstra, M. Pantic, and I. Patras A dynamic texture-based approach to recognition of facial actions and their temporal models IEEE Trans. Pattern Anal. Mach. Intell. 32 11 2010 1940 1954
32. Y. Tong, W. Liao, and Q. Ji Facial action unit recognition by exploiting their dynamic and semantic relationships IEEE Trans. Pattern Anal. Mach. Intell. 29 10 2007 1683 1699
33. T. McLellan, L. Johnston, J. Dalrymple-Alford, and R. Porter Sensitivity to genuine versus posed emotion specified in facial displays Cogn. Emotion 24 8 2010 1277 1292
34. P. Ekman Darwin, deception, and facial expression Ann. N. Y. Acad. Sci. 1000 10 2006 205 221
35. G.E. Hinton Training products of experts by minimizing contrastive divergence Neural Comput. 14 8 2002 1771 1800