Dynamic cascades with bidirectional bootstrapping for action unit detection in spontaneous facial behavior

IEEE Transactions on Affective Computing

Volumn 2, Issue 2, 2011, Pages 79-91

  Zhu, Yunfeng ^a   De La Torre, Fernando ^b   Cohn, Jeffrey F ^c   Zhang, Yu Jin ^a  

^a TSINGHUA UNIVERSITY   (China)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

Author keywords

action unit detection; bidirectional bootstrapping.; dynamic cascade boosting; Facial expression analysis; FACS

Indexed keywords

ACTION UNIT; ALTERNATIVE APPROACH; BIDIRECTIONAL BOOTSTRAPPING.; FACIAL ACTION; FACIAL EXPRESSION ANALYSIS; FACS; FALSE POSITIVE; ITERATIVE APPROACH; REAL-WORLD TESTS; SPONTANEOUS FACIAL BEHAVIOR; TRAINING DATA; TRAINING IMAGE; TRAINING SAMPLE;

ADAPTIVE BOOSTING; COMPUTER KEYBOARDS;

FACE RECOGNITION;

EID: 84857912820     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/T-AFFC.2011.10     Document Type: Article

References (50)

1. P. Ekman and W. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, 1978.
2. J.F. Cohn, Z. Ambadar, and P. Ekman, "Observer-Based Measurement of Facial Expression with the Facial Action Coding System," The Handbook of Emotion Elicitation and Assessment, Oxford Univ. Press, 2007.
3. J.F. Cohn and P. Ekman, "Measuring Facial Action by Manual Coding, Facial EMG, and Automatic Facial Image Analysis," Handbook of Nonverbal Behavior Research Methods in the Affective Sciences, Carnegie Mellon Univ., 2005.
4. P. Ekman and E.L. Rosenberg, What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford Univ. Press, 2005.
5. M. Pantic, N. Sebe, J.F. Cohn, and T. Huang, "Affective Multimodal Human-Computer Interaction," Proc. 13th Ann. ACM Int'l Conf. Multimedia, pp. 669-676, 2005.
6. Face Processing: Advanced Modeling and Methods, W. Zhao and R. Chellappa, eds. Academic Press, 2006.
7. S. Li and A. Jain, Handbook of Face Recognition. Springer, 2005.
8. Y. Tian, J.F. Cohn, and T. Kanade, "Facial Expression Analysis," Handbook of Face Recognition, S.Z. Li and A.K. Jain, eds., Springer, 2005.
9. Y. Tian, T. Kanade, and J.F. Cohn, "Recognizing Action Units for Facial Expression Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 97-115, Feb. 2001. (Pubitemid 32254704)
10. S. Lucey, A.B. Ashraf, and J.F. Cohn, "Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face," Face Recognition, K. Delac and M. Grgic, eds., pp. 275-286, InTECH Education and Publishing, 2007.
11. M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I.R. Fasel, and J.R. Movellan, "Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 568-573, 2005.
12. M.F. Valstar and M. Pantic, "Combined Support Vector Machines and Hidden Markov Models for Modeling Facial Action Temporal Dynamics," Proc. IEEE Int'l Conf. Computer Vision, pp. 118-127, 2007.
13. Y. Tong, W.H. Liao, and Q. Ji, "Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1683-1699, Oct. 2007.
14. P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition), pp. 511-518, 2001.
15. R. Xiao, H. Zhu, H. Sun, and X. Tang, "Dynamic Cascades for Face Detection," Proc. IEEE 11th Int'l Conf. Computer Vision, pp. 1-8, 2007.
16. S.Y. Yan, S.G. Shan, X.L. Chen, W. Gao, and J. Chen, "Matrix- Structural Learning (MSL) of Cascaded Classifier from Enormous Training Set," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-7, 2007.
17. Y.F. Zhu, F. De la Torre, and J.F. Cohn, "Dynamic Cascades with Bidirectional Bootstrapping for Spontaneous Facial Action Unit Detection," Proc. Workshops Affective Computing and Intelligent Interaction, Sept. 2009.
18. M.G. Frank, M.S. Bartlett, and J.R. Movellan, "The M3 Database of Spontaneous Emotion Expression," 2011.
19. M. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Automatic Recognition of Facial Actions in Spontaneous Expressions," J. Multimedia, vol. 1, no. 6, pp. 22-35, 2006.
20. M.J. Black and Y. Yacoob, "Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion," Int'l J. Computer Vision , vol. 25, no. 1, pp. 23-48, 1997. (Pubitemid 127506559)
21. F. De la Torre, Y. Yacoob, and L. Davis, "A Probabilisitc Framework for Rigid and Non-Rigid Appearance Based Tracking and Recognition," Proc. Seventh IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 491-498, 2000.
22. Y. Chang, C. Hu, R. Feris, and M. Turk, "Manifold Based Analysis of Facial Expression," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop, pp. 81-89, 2004.
23. C. Lee and A. Elgammal, "Facial Expression Analysis Using Nonlinear Decomposable Generative Models," Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, pp. 17-31, 2005.
24. M. Bartlett, G. Littlewort, I. Fasel, J. Chenu, and J. Movellan, "Fully Automatic Facial Action Recognition in Spontaneous Behavior," Proc. Seventh IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 223-228, 2006.
25. M. Pantic and L. Rothkrantz, "Facial Action Recognition for Facial Expression Analysis from Static Face Images," IEEE Trans. Systems, Man, and Cybernetics, vol. 34, no. 3, pp. 1449-1461, June 2004.
26. M. Pantic and I. Patras, "Dynamics of Facial Expression: Recognition of Facial Actions and Their Temporal Segments from Face Profile Image Sequences," IEEE Trans. Systems, Man, and Cybernetics-Part B: Cybernetics, vol. 36, no. 2, pp. 433-449, Apr. 2006.
27. P. Yang, Q. Liu, X. Cui, and D. Metaxas, "Facial Expression Recognition Using Encoded Dynamic Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
28. Y. Sun and L.J. Yin, "Facial Expression Recognition Based on 3D Dynamic Range Model Sequences," Proc. 10th European Conf. Computer Vision: Part II, pp. 58-71, 2008.
29. B. Braathen, M. Bartlett, G. Littlewort, and J. Movellan, "First Steps towards Automatic Recognition of Spontaneous Facial Action Units," Proc. ACM Workshop Perceptive User Interfaces, 2001.
30. P. Lucey, J.F. Cohn, S. Lucey, S. Sridharan, and K.M. Prkachin, "Automatically Detecting Action Units from Faces of Pain: Comparing Shape and Appearance Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, pp. 12-18, 2009.
31. J.F. Cohn and T. Kanade, "Automated Facial Image Analysis for Measurement of Emotion Expression," The Handbook of Emotion Elicitation and Assessment, J.A. Coan and J.B. Allen, eds., pp. 222-238, Oxford Univ. Press Series in Affective Science, 2006.
32. J.F. Cohn and M.A. Sayette, "Spontaneous Facial Expression in a Small Group Can Be Automatically Measured: An Initial Demonstration," Behavior Research Methods, vol. 42, no. 4, pp. 1079-1086, 2010.
33. M.F. Valstar, I. Patras, and M. Pantic, "Facial Action Unit Detection Using Probabilistic Actively Learned Support Vector Machines on Tracked Facial Point Data," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, pp. 76-83, 2005.
34. M. Valstar and M. Pantic, "Fully Automatic Facial Action Unit Detection and Temporal Analysis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 149-157, 2006.
35. S. Lucey, I. Matthews, C. Hu, Z. Ambadar, F. De la Torre, and J.F. Cohn, "Aam Derived Face Representations for Robust Facial Action Recognition," Proc. Seventh IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 155-160, 2006.
36. Z. Zeng, M. Pantic, G.I. Roisman, and T.S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
37. T.F. Cootes, G. Edwards, and C. Taylor, "Active Appearance Models," Proc. European Conf. Computer Vision, pp. 484-498, 1998. (Pubitemid 128093046)
38. F. De la Torre and M. Nguyen, "Parameterized Kernel Principal Component Analysis: Theory and Applications to Supervised and Unsupervised Image Alignment," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
39. I. Matthews and S. Baker, "Active Appearance Models Revisited," Int'l J. Computer Vision, vol. 60, pp. 135-164, 2004.
40. V. Blanz and T. Vetter, "A Morphable Model for the Synthesis of 3D Faces," Proc. ACM SIGGRAPH, 1999.
41. Y.L. Tian, T. Kanade, and J.F. Cohn, "Evaluation of Gabor-Wavelet-Based Facial Action Unit Recognition in Image Sequences of Increasing Complexity," Proc. Seventh IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 229-234, 2002.
42. A.B. Ashraf, S. Lucey, T. Chen, K. Prkachin, P. Solomon, Z. Ambadar, and J.F. Cohn, "The Painful Face: Pain Expression Recognition Using Active Appearance Models," Proc. ACM Int'l Conf. Multimodal Interfaces, pp. 9-14, 2007.
43. P. Lucey, J.F. Cohn, S. Lucey, S. Sridharan, and K.M. Prkachin, "Automatically Detecting Pain Using Facial Actions," Proc. Int'l Conf. Affective Computing and Intelligent Interaction, 2009.
44. D. Lowe, "Object Recognition from Local Scale-Invariant Features," Proc. IEEE Int'l Conf. Computer Vision, pp. 1150-1157, 1999. (Pubitemid 32079997)
45. K. Sung and T. Poggio, "Example-Based Learning for View-Based Human Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 39-51, Jan. 1998. (Pubitemid 128741306)
46. L. Breiman, Classification and Regression Trees. Chapman and Hall, 1998.
47. R. Schapire and Y. Freund, "Experiments with a New Boosting Algorithm," Proc. 13th Int'l Conf. Machine Learning, pp. 148-156, 1996.
48. E. Tola, V. Lepetit, and P. Fua, "A Fast Local Descriptor for Dense Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
49. K. Mikolajczyk and C. Schmid, "A Performance Evaluation of Local Descriptors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005. (Pubitemid 41555406)
50. P.. Fua, V. Lepetit, and E. Tola, "Daisy: An Efficient Dense Descriptor Applied to Wide Baseline Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 815-830, May 2010.