A dynamic texture-based approach to recognition of facial actions and their temporal models

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 32, Issue 11, 2010, Pages 1940-1954

  Koelstra, Sander ^a   Pantic, Maja ^b   Patras, Ioannis ^a  

^a QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

dynamic texture; facial expression; Facial image analysis; motion

Indexed keywords

COHN-KANADE DATABASE; DATA SETS; DESCRIPTORS; DYNAMIC TEXTURES; EVENT RECOGNITION; EXTENDED VERSIONS; FACE REGIONS; FACIAL ACTION; FACIAL EXPRESSIONS; FACIAL GESTURES; FACIAL IMAGES; FREE-FORM DEFORMATION; GENERALIZATION PERFORMANCE; INPUT IMAGE; INPUT VIDEOS; MOTION; MOTION HISTORY IMAGES; MOTION ORIENTATION HISTOGRAM; MOTION REPRESENTATION; NONRIGID REGISTRATION; NOVEL METHODS; TEMPORAL DOMAIN; TEMPORAL MODELS; TEMPORAL SEGMENTS;

HIDDEN MARKOV MODELS; IMAGE ANALYSIS; IMAGE SEGMENTATION; TEXTURES;

FACE RECOGNITION;

ALGORITHM; ARTICLE; AUTOMATED PATTERN RECOGNITION; BIOLOGICAL MODEL; COMPUTER ANALYSIS; FACE; FACIAL EXPRESSION; GESTURE; HUMAN; INFORMATION RETRIEVAL; METHODOLOGY; RECOGNITION; TIME; VIDEORECORDING;

ALGORITHMS; COMPUTING METHODOLOGIES; FACE; FACIAL EXPRESSION; GESTURES; HUMANS; INFORMATION STORAGE AND RETRIEVAL; MODELS, BIOLOGICAL; PATTERN RECOGNITION, AUTOMATED; RECOGNITION (PSYCHOLOGY); TIME FACTORS; VIDEO RECORDING;

EID: 78149285836     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2010.50     Document Type: Article

References (44)

1. E. Aarts, "Ambient Intelligence Drives Open Innovation," ACM Interactions, vol. 12, no. 4, pp. 66-68, 2005.
2. K. Anderson and P. McOwan, "A Real-Time Automated System for Recognition of Human Facial Expressions," IEEE Trans. Systems, Man, and Cybernetics, vol. 36, no. 1, pp. 96-105, Feb. 2006.
3. M. Bartlett, G. Littlewort-Ford, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 568-573, 2005.
4. M. Bartlett, G. Littlewort-Ford, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Fully Automatic Facial Action Recognition in Spontaneous Behavior," Proc. IEEE Conf. Face and Gesture Recognition, pp. 223-230, 2006.
5. Y. Chang, C. Hu, R. Feris, and M. Turk, "Manifold-Based Analysis of Facial Expression," J. Image and Vision Computing, vol. 24, no. 6, pp. 605-614, 2006.
6. D. Chetverikov and R. Péteri, "A Brief Survey of Dynamic Texture Description and Recognition," Proc. Conf. Computer Recognition Systems, vol. 5, pp. 17-26, 2005.
7. I. Cohen, N. Sebe, F. Cozman, M. Cirelo, and T. Huang, "Learning Bayesian Network Classifiers for Facial Expression Recognition Both Labeled and Unlabeled Data," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 595-601, 2003.
8. J. Davis and A. Bobick, "The Representation and Recognition of Human Movement Using Temporal Templates," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 928-934, 1997.
9. E. Douglas-Cowie, R. Cowie, I. Sneddon, C. Cox, O. Lowry, M. McRorie, J. Martin, L. Devillers, S. Abrilian, A. Batliner, N. Amir, and K. Karpouzis, "The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data," Lecture Notes in Computer Science, vol. 4738, pp. 488-500, Springer, 2007.
10. P. Ekman, W. Friesen, and J. Hager, The Facial Action Coding System: A Technique for the Measurement of Facial Movement. A Human Face, 2002.
11. P. Ekman and E. Rosenberg, What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford Univ. Press, 2005.
12. J. Friedman, T. Hastie, and R. Tibshirani, "Additive Logistic Regression: A Statistical View of Boosting," The Annals of Statistics, vol. 28, no. 2, pp. 337-407, 2000.
13. S. Gokturk, J. Bouguet, C. Tomasi, and B. Girod, "Model-Based Face Tracking for Viewindependent Facial Expression Recognition," Proc. IEEE Conf. Face and Gesture Recognition, pp. 272-278, 2002.
14. G. Guo and C. Dyer, "Learning from Examples in the Small Sample Case Face Expression Recognition," IEEE Trans. Systems, Man, and Cybernetics, vol. 35, no. 3, pp. 477-488, June 2005.
15. T. Kanade, J. Cohn, and Y. Tian, "Comprehensive Database for Facial Expression Analysis," Proc. IEEE Conf. Face and Gesture Recognition, pp. 46-53, 2000.
16. S. Koelstra and M. Pantic, "Non-Rigid Registration Using Free-Form Deformations for Recognition of Facial Actions and Their Temporal Dynamics," Proc. IEEE Conf. Face and Gesture Recognition, pp. 1-8, 2008.
17. I. Kotsia and I. Pitas, "Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines," IEEE Trans. Image Processing, vol. 16, no. 1, pp. 172-187, Jan. 2007.
18. G. Littlewort, M. Bartlett, I. Fasel, J. Susskind, and J. Movellan, "Dynamics of Facial Expression Extracted Automatically from Video," Image and Vision Computing, vol. 24, no. 6, pp. 615-625, 2006.
19. Z. Lu, W. Xie, J. Pei, and J. Huang, "Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms," Proc. IEEE Workshop Motion and Video Computing, vol. 2, pp. 241-246, 2005.
20. S. Lucey, A. Ashraf, and J. Cohn, "Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face," Face Recognition, K. Delac and M. Grgic, eds., pp. 275-286, I-Tech Education and Publishing, 2007.
21. M. Pantic and M. Bartlett, "Machine Analysis of Facial Expressions," Face Recognition, K. Delac and M. Grgic, eds., pp. 377-416, I-Tech Education and Publishing, 2007.
22. M. Pantic and I. Patras, "Detecting Facial Actions and Their Temporal Segments in Nearly Frontal-View Face Image Sequences," Proc. IEEE Conf. Systems, Man, and Cybernetics, vol. 4, pp. 3358-3363, 2005.
23. M. Pantic and I. Patras, "Dynamics of Facial Expressions Recognition of Facial Actions and Their Temporal Segments from Face Profile Image Sequences," IEEE Trans. Systems, Man, and Cybernetics, vol. 36, no. 2, pp. 433-449, Apr. 2006.
24. M. Pantic, A. Pentland, A. Nijholt, and T. Huang, "Human Computing and Machine Understanding of Human Behavior: A Survey," Lecture Notes on Artificial Intelligence, vol. 4451, pp. 47-71, Springer, 2007.
25. M. Pantic and L. Rothkrantz, "Automatic Analysis of Facial Expressions The State of the Art," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, Dec. 2000.
26. M. Pantic and L. Rothkrantz, "Facial Action Recognition for Facial Expression Analysis from Static Face Images," IEEE Trans. Systems, Man, and Cybernetics, vol. 34, no. 3, pp. 1449-1461, June 2004.
27. M. Pantic, M. Valstar, R. Rademaker, and L. Maat, "Web-Based Database for Facial Expression Analysis," Proc. IEEE Conf. Multimedia and Expo, pp. 317-321, 2005.
28. R. Polana and R. Nelson, "Temporal Texture and Activity Recognition," Motion-Based Recognition, pp. 87-115, 1997.
29. D. Rueckert, L. Sonoda, C. Hayes, D. Hill, M. Leach, and D. Hawkes, "Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images," IEEE Trans. Medical Imaging, vol. 18, no. 8, pp. 712-721, Aug. 1999.
30. P. Saisan, G. Doretto, Y. Wu, and S. Soatto, "Dynamic Texture Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 58-63, 2001.
31. Y. Tian, T. Kanade, and J. Cohn, "Recognizing Action Units for Facial Expression Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 97-115, Feb. 2001.
32. Y. Tian, T. Kanade, and J. Cohn, "Evaluation of Gabor-Wavelet-Based Facial Action Unit Recognition in Image Sequences of Increasing Complexity," Proc. IEEE Conf. Face and Gesture Recognition, pp. 218-223, 2002.
33. Y. Tong, W. Liao, and Q. Ji, "Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1683-1699, Oct. 2007.
34. M. Valstar and M. Pantic, "Fully Automatic Facial Action Unit Detection and Temporal Analysis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 3, no. 149, 2006.
35. M. Valstar and M. Pantic, "Combined Support Vector Machines and Hidden Markov Models for Modeling Facial Action Temporal Dynamics," Lecture Notes on Computer Science, vol. 4796, pp. 118-127, Springer, 2007.
36. M. Valstar, M. Pantic, and I. Patras, "Motion History for Facial Action Detection from Face Video," Proc. IEEE Conf. Systems, Man, and Cybernetics, pp. 635-640, 2004.
37. D. Vukandinovic and M. Pantic, "Fully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers," Proc. IEEE Conf. Systems, Man, and Cybernetics, vol. 2, pp. 1692-1698, 2005.
38. Z. Wen and T. Huang, "Capturing Subtle Facial Motions in 3D Face Tracking," Proc. Int'l Conf. Computer Vision, vol. 2, pp. 1343-1350, 2003.
39. J. Whitehill and C. Omlin, "Haar Features for FACS AU Recognition," Proc. IEEE Int'l Conf. Face and Gesture Recognition, pp. 97-101, 2006.
40. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A Survey of Affect Recognition Methods: Audio, Visual and Spontaneous Expressions," Proc. ACM Conf. Multimodal Interfaces, pp. 126-133, 2007.
41. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
42. Y. Zhang and Q. Ji, "Active and Dynamic Information Fusion for Facial Expression Understanding from Image Sequence," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 699-714, May 2005.
43. G. Zhao and M. Pietikäinen, "Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 915-928, June 2007.
44. G. Zhao and M. Pietikäinen, "Boosted Multi-Resolution Spatio-temporal Descriptors for Facial Expression Recognition," Pattern Recognition Letters, vol. 30, no. 12, pp. 1117-1127, Sept. 2009.