Coupled Gaussian processes for pose-invariant facial expression recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 35, Issue 6, 2013, Pages 1357-1369

  Rudovic, Ognjen ^a   Pantic, Maja ^a   Patras, Ioannis ^b  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Gaussian process regression; Head pose estimation; Multiview pose invariant facial expression emotion recognition

Indexed keywords

ACTIVE APPEARANCE MODELS; FACIAL EXPRESSION RECOGNITION; FACIAL EXPRESSIONS; GATING FUNCTIONS; GAUSSIAN PROCESS REGRESSION; GAUSSIAN PROCESSES; POINT DISTRIBUTION MODELS; TRAINING DATA;

GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); GESTURE RECOGNITION; REGRESSION ANALYSIS;

FACE RECOGNITION;

ALGORITHM; ARTICLE; AUTOMATED PATTERN RECOGNITION; BIOMETRY; COMPUTER ASSISTED DIAGNOSIS; FACIES; HEAD; HISTOLOGY; HUMAN; IMAGE ENHANCEMENT; METHODOLOGY; NORMAL DISTRIBUTION; REGRESSION ANALYSIS;

ALGORITHMS; BIOMETRY; FACIES; HEAD; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; NORMAL DISTRIBUTION; PATTERN RECOGNITION, AUTOMATED; REGRESSION ANALYSIS;

EID: 84879129471     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2012.233     Document Type: Article

References (49)

1. M. Pantic, A. Nijholt, A. Pentland, and T. Huang, "Human-Centred Intelligent Human-Computer Interaction (HCI2): How Far Are We from Attaining It?" Int'l J. Autonomous and Adaptive Comm. Systems, vol. 1, no. 2, pp. 168-187, 2008.
2. A. Vinciarelli, M. Pantic, and H. Bourlard, "Social Signal Processing: Survey of an Emerging Domain," Image and Vision Computing J., vol. 27, no. 12, pp. 1743-1759, 2009.
3. Z. Zeng, M. Pantic, G.I. Roisman, and T.S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
4. M.S. Bartlett, G. Littlewort, M.G. Frank, C. Lainscsek, I.R. Fasel, and J.R. Movellan, "Automatic Recognition of Facial Actions in Spontaneous Expressions," J. Multimedia, vol. 1, no. 6, pp. 22-35, 2006.
5. Y. Zhang and Q. Ji, "Active and Dynamic Information Fusion for Facial Expression Understanding from Image Sequences," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 699-714, May 2005. (Pubitemid 40608403)
6. Y. Tong, W. Liao, and Q. Ji, "Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1683-1699, Oct. 2007.
7. Z. Zhu and Q. Ji, "Robust Real-Time Face Pose and Facial Expression Recovery," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 681-688, 2006,
8. Y. Tong, J. Chen, and Q. Ji, "A Unified Probabilistic Framework for Spontaneous Facial Action Modeling and Understanding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 2, pp. 258-273, Feb. 2010.
9. S. Moore and R. Bowden, "Local Binary Patterns for Multi-View Facial Expression Recognition," Computer Vision and Image Understanding, vol. 115, no. 4, pp. 541-558, 2011.
10. W. Zheng, H. Tang, Z. Lin, and T.S. Huang, "Emotion Recognition from Arbitrary View Facial Images," Proc. European Conf. Computer Vision, pp. 490-503, 2010,
11. H. Tang, M. Hasegawa-Johnson, and T.S. Huang, "Non-Frontal View Facial Expression Recognition Based on Ergodic Hidden Markov Model Supervectors," Proc. Int'l Conf. Multimedia and Expo, pp. 1202-1207, 2010,
12. M.J. Black and Y. Yacoob, "Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion," Int'l J. Computer Vision, vol. 25, pp. 23-48, 1997. (Pubitemid 127506559)
13. S. Kumano, K. Otsuka, J. Yamato, E. Maeda, and Y. Sato, "Pose-Invariant Facial Expression Recognition Using Variable-Intensity Templates," Int'l J. Computer Vision, vol. 83, no. 2, pp. 178-194, 2009.
14. P. Ekman and W.V. Friesen, Unmasking the Face: A Guideline to Recognising Emotions from Facial Clues, vol. 3. Prentice Hall, 1978.
15. C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2007.
16. Y. Sun and L. Yin, "Facial Expression Recognition Based on 3D Dynamic Range Model Sequences," Proc. European Conf. Computer Vision, pp. 58-71, 2008.
17. L.A. Jeni, A. Lrincz, T. Nagy, Z. Palotai, J. Sebk, Z. Szab, and D. Takcs, "3D Shape Estimation in Video Sequences Provides High Precision Evaluation of Facial Expressions," Image and Vision Computing, vol. 30, pp. 785-795, 2012.
18. J. Wang, L. Yin, X. Wei, and Y. Sun, "3D Facial Expression Recognition Based on Primitive Surface Feature Distribution," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1399-1406, 2006,
19. R.E. Kaliouby and P. Robinson, "Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, p. 154. 2004,
20. W.-K. Liao and I. Cohen, "Belief Propagation Driven Method for Facial Gestures Recognition in Presence of Occlusions," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, p. 158. 2006,
21. S. Lucey, A.B. Ashraf, and J. Cohn, Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face. I-Tech Education & Publishing, 2007.
22. J. Sung and D. Kim, "Real-Time Facial Expression Recognition Using STAAM and Layered GDA Classifier," Image and Vision Computing J., vol. 27, no. 9, pp. 1313-1325, 2009.
23. F. Dornaika and J. Orozco, "Real Time 3D Face and Facial Feature Tracking," J. Real-Time Image Processing, vol. 2, no. 1, pp. 35-44, 2007. (Pubitemid 47551870)
24. R. Gross, I. Matthews, and S. Baker, "Generic vs. Person Specific Active Appearance Models," Image and Vision Computing J., vol. 23, pp. 1080-1093, 2005. (Pubitemid 41329840)
25. X. Chai, S. Shan, X. Chen, and W. Gao, "Locally Linear Regression for Pose-Invariant Face Recognition," IEEE Trans. Image Processing, vol. 16, no. 7, pp. 1716-1725, July 2007. (Pubitemid 47018870)
26. M.A.O. Vasilescu and D. Terzopoulos, "Multilinear Analysis of Image Ensembles: Tensorfaces," Proc. European Conf. Computer Vision, pp. 447-460, 2002.
27. Y. Hu, Z. Zeng, L. Yin, X. Wei, X. Zhou, and T.S. Huang, "Multi- View Facial Expression Recognition," Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 1-6, 2008,
28. W. Zheng, H. Tang, Z. Lin, and T. Huang, "A Novel Approach to Expression Recognition from Non-Frontal Face Images," Proc. IEEE Int'l Conf. Computer Vision, pp. 1901-1908, 2009.
29. O. Rudovic, I. Patras, and M. Pantic, "Coupled Gaussian Process Regression for Pose-Invariant Facial Expression Recognition," Proc. European Conf. Computer Vision, pp. 350-363, 2010.
30. M. Valstar, B. Martinez, X. Binefa, and M. Pantic, "Facial Point Detection Using Boosted Regression and Graph Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2729-2736, 2010.
31. E. Murphy-Chutorian and M.M. Trivedi, "Head Pose Estimation in Computer Vision: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 4, pp. 607-626, Apr. 2009.
32. M.F. Valstar and M. Pantic, "Fully Automatic Recognition of the Temporal Phases of Facial Actions," IEEE Trans. Systems, Man, and Cybernetics, vol. 42, no. 1, pp. 28-43, Feb. 2012.
33. B. Schölkopf, A.J. Smola, R.C. Williamson, and P.L. Bartlett, "New Support Vector Algorithms," Neural Computation, vol. 12, pp. 1207-1245, 2000.
34. K. Grochow, S.L. Martin, A. Hertzmann, and Z. Popović, "Style-Based Inverse Kinematics," Proc. ACM Int'l Conf. Computer Graphics and Interactive Techniques, pp. 522-531, 2004.
35. C.E. Rasmussen and C.K.I. Williams, Gaussian Processes for Machine Learning. The MIT Press, 2005.
36. S.J. Julier and J.K. Uhlmann, "A Non-Divergent Estimation Algorithm in the Presence of Unknown Correlations," Proc. Am. Control Conf., pp. 2369-2373, 1997.
37. V. Tresp and M. Taniguchi, "Combining Estimators Using Non-Constant Weighting Functions," Proc. Neural Information Processing Systems Conf., pp. 419-426, 1995.
38. V. Tresp, "A Bayesian Committee Machine," Neural Computing, vol. 12, no. 11, pp. 2719-2741, 2000.
39. P. Boyle and M. Frean, "Dependent Gaussian Processes," Neural Information Processing Systems, pp. 217-224, 2005.
40. K. Yu, V. Tresp, and A. Schwaighofer, "Learning Gaussian Processes from Multiple Tasks," Proc. 22nd Int'l Conf. Machine Learning, pp. 1012-1019, 2005.
41. E.V. Bonilla, K.M.A. Chai, and C.K.I. Williams, "Multi-Task Gaussian Process Prediction," Neural Information Processing Systems, 2008.
42. M. Alvarez and N. Lawrence, "Sparse Convolved Multiple Output Gaussian Processes," Proc. Neural Information Processing Systems Conf., pp. 57-64, 2008.
43. L. Bo and C. Sminchisescu, "Twin Gaussian Processes for Structured Prediction," In'l J. Computer Vision, vol. 87, nos. 1/2, pp. 28-52, 2010.
44. S. Yu, B. Krishnapuram, R. Rosales, and R.B. Rao, "Bayesian Co-Training," J. Machine Learning Research, vol. 12, pp. 2649-2680, 2011.
45. R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, "Multi-Pie," Image and Vision Computing J., vol. 28, no. 5, pp. 807-813, 2010.
46. G. McKeown, M.F. Valstar, R. Cowie, and M. Pantic, "The Semaine Corpus of Emotionally Coloured Character Interactions," Proc. Int'l Conf. Multimedia and Expo, pp. 1079-1084, 2010.
47. C.-C. Chang and C.-J. Lin, "LIBSVM: A Library for Support Vector Machines," ACM Trans. Intelligent Systems and Technology, vol. 2, pp. 1-27, 2011.
48. T. Cootes and C. Taylor, "Active Shape Models - Smart Snakes," Proc. British Machine Vision Conf., pp. 266-275, 1992.
49. Y. Hu, Z. Zeng, L. Yin, X. Wei, J. Tu, and T. Huang, "A Study of Non-Frontal-View Facial Expressions Recognition," Proc. Int'l Conf. Pattern Recognition, pp. 1-4, 2008.