Facial expression recognition system based on rigid and non-rigid motion separation and 3D pose estimation

Pattern Recognition

Volumn 42, Issue 5, 2009, Pages 962-977

  Wang, Te Hsun ^a   James Lien, Jenn Jier ^a  

^a NATIONAL CHENG KUNG UNIVERSITY   (Taiwan)

Author keywords

3D pose estimation; Facial expression recognition; FACS action units; Perspective projection; Rigid and non rigid motion separation

Indexed keywords

COMPUTER KEYBOARDS; HIDDEN MARKOV MODELS; ROTATION; SEPARATION;

3D POSE ESTIMATION; ACTION UNIT; FACIAL EXPRESSION RECOGNITION; NON-RIGID MOTION; PERSPECTIVE PROJECTIONS;

FACE RECOGNITION;

EID: 58249094344     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2008.09.035     Document Type: Article

References (54)

1. Lien J.J., Kanade T., Li C.C., and Cohn J.F. Detection, tracking, and classification of action units in facial expression. Robotics and Autonomous Systems, Special Issue: Face Expression in Human-Robot Interaction 31 (2000) 131-146
2. Abboud B., and Davoine F. Bilinear factorization for facial expression analysis and synthesis. IEE Proc. Vision Image Signal Process. 152 3 (2005) 327-333
3. G. Antonini, M. Sorci, M. Bierlaire, J.P. Thiran, Discrete choice models for static facial expression recognition, Advanced Concepts for Intelligent Vision Systems, 2006, pp. 710-721.
4. M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, J. Movellan, Fully automatic facial action recognition in spontaneous behavior. In: International Conference on Automatic Face and Gesture Recognition, 2006, pp. 223-230.
5. J.J. Bazzo, M.V. Lamar, Recognizing facial actions using Gabor wavelets with neutral face average difference. In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 505-510.
6. Black M., and Yacoob Y. Recognizing facial expressions in image sequences using local parameterized models of image motion. Int. J. Comput. Vision (1997) 23-48
7. Chuang C.F., and Shih F.Y. Recognizing facial action units using independent component analysis and support vector machine. Pattern Recognition 39 9 (2006) 1795-1798
8. I. Cohen, N. Sebe, F.G. Gozman, M.C. Cirelo, T.S. Huang, Learning Bayesian network classifiers for facial expression recognition using both labeled and unlabeled data. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, 2003, pp. 595-601.
9. D. Datcu, L.J.M. Rothkrantz, Facial expression recognition with relevance vector machines. In: IEEE International Conference on Multimedia and Expo, 2005, pp. 193-196.
10. I. Essa, T. Darrell, A. Pentland, Tracking facial motion. In: IEEE Workshop on Non-Rigid and Articulate Motion, 1994, pp. 36-42.
11. I. Essa, Analysis, Interpretation and synthesis of facial expressions, MIT Media Lab., Technical Report #303, 1995.
12. Gizatdinova Y., and Surakka V. Feature-based detection of facial landmarks from neutral and expressive facial images. IEEE Trans. Pattern Anal. Mach. Intell. 28 1 (2006) 135-139
13. Gao Y., Leung M.K.H., Hui S.C., and Tananda M.W. Facial expression recognition from line-based caricatures. IEEE Trans. Syst. Man Cybern. Part A 33 3 (2003) 407-412
14. Guo G., and Dyer C.R. Learning from examples in the small sample case: face expression recognition. IEEE Trans. Syst. Man Cybern. Part B 35 3 (2005) 477-488
15. S. Lucey, I. Matthews, C. Hu, Z. Ambadar, F. de la Torre, J. Cohn, AAM derived face representations for robust facial action recognition. In: International Conference on Automatic Face and Gesture Recognition, 2006, pp. 155-160.
16. Ma L., and Khorasani K. Facial expression recognition using constructive feedforward neural networks. IEEE Trans. Syst. Man Cybern. Part B 34 3 (2004) 1588-1595
17. A.M. Martinez, Recognizing expression variant faces from a single sample image per class. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, 2003, pp. 353-358.
18. Pantic M., and Patras I. Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans. Syst. Man Cybern. Part B 36 2 (2006) 433-449
19. N. Rose, Facial expression classification using Gabor and log-Gabor filters. In: International Conference on Automatic Face and Gesture Recognition, 2006, pp. 346-350.
20. M. Rosenblum, Y. Yacoob, L.S. Davis, Human emotion recognition from motion using a radial basis function network architecture, University of Maryland, CS-TR-3304, 1994.
21. Y. Saatci, C. Town, Cascaded classification of gender and facial expression using active appearance models. In: International Conference on Automatic Face and Gesture Recognition, 2006, pp. 393-398.
22. N. Sebe, M.S. Lew, I. Cohen, Y. Sun, T. Gevers, T.S. Huang, Authentic facial expression analysis. In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 517-522.
23. Y. Shinohara, N. Otsu, Facial expression recognition using fisher weight maps In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 499-504.
24. Tian Y., Kanade T., and Cohn J.F. Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23 2 (2002) 97-115
25. Y. Tian, T. Kanade, J.F. Cohn, Evaluation of Gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity. In: International Conference on Automatic Face and Gesture Recognition, 2002, pp. 218-223.
26. F. De la Torre, Y. Yacoob, L.S. Davis, A probabilistic framework for rigid and non-rigid appearance based tracking and recognition. In: International Conference on Automatic Face and Gesture Recognition, 2000, pp. 491-498.
27. J. Whitehill, C.W. Omlin, Local versus global segmentation for facial expression recognition. In: International Conference on Automatic Face and Gesture Recognition, 2006, pp. 357-362.
28. Yacoob Y., and Davis L.S. Recognizing human facial expressions from long image sequence using optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 18 6 (1996) 636-642
29. L. Yin, X. Wei, Multi-scale primal feature based facial expression modeling and identification. In: International Conference on Automatic Face and Gesture Recognition, 2006, pp. 603-608.
30. L. Zalewski, S. Gong, Synthesis and recognition of facial expressions in virtual 3D views. In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 493-498.
31. Chen H.Y., Huang C.L., and Fu C.M. Hybrid-boost learning for multi-pose face detection and facial expression recognition. Pattern Recognition 41 3 (2008) 1173-1185
32. Zhang Y., and Ji Q. Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27 5 (2005) 699-714
33. Anderson K., and McOwan P.W. A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybern. Part B 36 1 (2006) 96-105
34. S.B. Gokturk, J.Y. Bouguet, C. Tomasi, B. Girod, Model-based face tracking for view-independent facial expression recognition. In: International Conference on Automatic Face and Gesture Recognition, 2002, pp. 272-278.
35. B. Braathen, M.S. Bartlett, G. Littlewort, E. Smith, J.R. Movellan, An approach to automatic recognition of spontaneous facial actions. In: International Conference on Automatic Face and Gesture Recognition, 2002, pp. 345-350.
36. Z. Zhu, Q. Ji, Robust real-time pose and facial expression recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2006, pp. 681-688.
37. T.K. Marks, J. Hershey, J.C. Roddey, J.R. Movellan, Joint tracking of pose, expression, and texture using conditionally Gaussian filters. In: Advances in Neural Information Processing Systems, vol. 17, 2005, pp. 889-896.
38. Ekman P., and Friesen W.V. Facial Action Coding System (1978), Consulting Psychologist Press Inc., San Francisco, CA
39. T. Horprasert, Y. Yacoob, L.S. Davis, Computing 3-D head orientation from a monocular image sequence. In: IEEE Conference on Automatic Face and Gesture Recognition, 1996, pp. 242-247.
40. S. Goldenstein, C. Vogler, D. Metaxas, 3D facial tracking from corrupted movie sequences. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, 2004, pp. 880-885.
41. Malassiotos S., and Strintzis M.G. Robust real-time 3D head pose estimation from range data. Pattern Recognition 38 8 (2005) 1153-1165
42. L. Vacchetti, V. Lepetit, P. Fua, Fusing online and offline information for stable 3D tracking in real-time. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2003, pp. 241-248.
43. Wiles C.S., Maki A., and Matsuda N. Hyperpatches for 3D model acquisition and tracking. IEEE Trans. Pattern Anal. Mach. Intell. 23 12 (2001) 1391-1403
44. J. Yao, W.K. Cham, Efficient model-based linear head motion recovery from movies. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2004, pp. 414-421.
45. C.T. Tu, J.J. Lien, Automatic location of facial feature points and synthesis of facial sketches using direct combined model. In: IEEE Trans. Syst. Man Cybern. Part B. (submitted for publication in 2009).
46. Klette R., Schluns K., and Koschan A. Computer Vision: Three-Dimensional Data from Images (1998), Springer, Berlin
47. C.S. Chen, C.K. Yu, Y.P. Hung, New calibration-free approach for augmented reality based on parameterized cuboid structure. In: IEEE International Conference on Computer Vision, 1999, pp. 30-37.
48. B. Lucas, T. Kanade, An iterative image registration technique with an application in stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674-679, 1981.
49. Aloimonos Y. Perspective approximations. Image and Vision Computings 8 3 (1990) 179-192
50. Rabiner L.R. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77 2 (1989) 257-285
51. B. Guenter, C. Grimm, D. Wood, H. Malvar, F. Pighin, Making faces, ACM SIGGRAPH, 1998, pp. 55-66.
52. Z. Liu, Z. Zhang, C. Jacobs, M. Cohen, Rapid modeling of animated faces from video, Microsoft Research, Technical Report, MSR-TR-2000-11, 2000.
53. Terzopoulos D., and Waters K. Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. Pattern Anal. Mach. Intell. 15 6 (1993) 569-579
54. J.Y. Wu, J.J. Lien, 3D face reconstruction under missing correspondence: a factorization approach. In: IPPR Conference on Computer Vision, Graphics and Image Processing, 2006.