Facial expression recognition in dynamic sequences: An integrated approach

Pattern Recognition

Volumn 47, Issue 3, 2014, Pages 1271-1281

  Fang, Hui ^a,d   Mac Parthaláin, Neil ^b   Aubrey, Andrew J ^c   Tam, Gary K L ^a,c   Borgo, Rita ^a   Rosin, Paul L ^c   Grant, Philip W ^a   Marshall, David ^c   Chen, Min ^d  

^a SWANSEA UNIVERSITY   (United Kingdom)

^b ABERYSTWYTH UNIVERSITY   (United Kingdom)

^c CARDIFF UNIVERSITY   (United Kingdom)

^d UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Dynamic feature extraction and visualisation; Facial expression analysis

Indexed keywords

AUTOMATIC FACIAL EXPRESSION ANALYSIS; EXTRACTING INFORMATION; FACIAL EXPRESSION ANALYSIS; FACIAL EXPRESSION RECOGNITION; HUMAN COGNITIVE SYSTEMS; HUMAN FACIAL EXPRESSIONS; MACHINE LEARNING TECHNIQUES; SUBJECTIVE INFORMATION; EXPRESSION RECOGNITION;

COGNITIVE SYSTEMS; FEATURE EXTRACTION; GESTURE RECOGNITION; LEARNING SYSTEMS; VIDEO RECORDING; ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION);

FACE RECOGNITION;

EID: 84888348216     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2013.09.023     Document Type: Conference Paper

References (50)

1. C. Darwin The Expression of the Emotions in Man and Animals 1872 J. Murray London
2. P. Ekman, and W. Friesen Constants across cultures in the face and emotion J. Pers. Soc. Psychol. 17 1971 124 129
3. N. Suwa, M. Sugie, K. Fujimora, A preliminary note on pattern recognition of human emotional expression, in: Proceedings of the International Joint Conference on Pattern Recognition, 1978, pp. 408-410.
4. P. Viola, and M. Jones Robust real-time face detection Int. J. Comp. Vis. 57 2004 137 154
5. H. Fang, N.P. Costen, From rank-n to rank-1 face recognition based on motion similarity, in: Proceedings of the British Conference on Machine Vision, 2009, pp. 1-11.
6. C. Cornelis, M.D. Cock, A.M. Radzikowska, Vaguely quantified rough sets, in: Proceedings of the International Conference on Rough Sets, Fuzzy Sets, Data Mining and Granular Computing, vol. 4482, Springer, 2007, pp. 87-94.
7. L. Breiman Random forests Mach. Learn. 45 2001 5 32
8. M. Pantic, I. Patras, Detecting facial actions and their temporal segments in nearly frontal-view face image sequences, in: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, vol. 4, 2005, pp. 3358-3363.
9. M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, J. Movellan, Recognizing facial expression: machine learning and application to spontaneous behavior, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, Washington, DC, USA, 2005, pp. 568-573.
10. Y.-I. Tian, T. Kanade, and J. Cohn Recognizing action units for facial expression analysis IEEE Trans. Pattern Anal. Mach. Intell. 23 2001 97 115
11. M. Pantic, and L.J.M. Rothkrantz Automatic analysis of facial expressions the state of the art IEEE Trans. Pattern Anal. Mach. Intell. 22 2000 1424 1445
12. B. Fasel, and J. Luettin Automatic facial expression analysis a survey Pattern Recognition 36 2003 259 275
13. Q. Zhang, Z. Liu, B. Guo, D. Terzopoulos, and H.-Y. Shum Geometry-driven photorealistic facial expression synthesis IEEE Trans. Vis. Comp. Graphics 12 2006 48 60
14. Z. Zeng, M. Pantic, G. Roisman, and T. Huang A survey of affect recognition methods audio, visual and spontaneous expressions IEEE Trans. Pattern Anal. Mach. Intell. 31 2009 39 58
15. Y. Zhang, and Q. Ji Active and dynamic information fusion for facial expression understanding from image sequences IEEE Trans. Pattern Anal. Mach. Intell. 27 2005 699 714
16. M. Pantic, and I. Patras Dynamics of facial expression recognition of facial actions and their temporal segments from face profile image sequences IEEE Trans. Syst. Man Cybern., Part B 36 2006 433 449
17. Y. Tong, W. Liao, and Q. Ji Facial action unit recognition by exploiting their dynamic and semantic relationships IEEE Trans. Pattern Anal. Mach. Intell. 29 2007 1683 1699
18. Y. Zhang, Q. Ji, Z. Zhu, and B. Yi Dynamic facial expression analysis and synthesis with mpeg-4 facial animation parameters IEEE Trans. Circuits Syst. Video Technol. 18 2008 1 15
19. G. Donato, M. Bartlett, J. Hager, P. Ekman, and T. Sejnowski Classifying facial actions IEEE Trans. Pattern Anal. Mach. Intell. 21 1999 974 989
20. M. Valstar, and M. Pantic Fully automatic recognition of the temporal phases of facial actions IEEE Trans. Syst. Man Cybern., Part B 42 2012 28 43
21. T. Kanade, J. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in: Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition, 2000, pp. 46-53.
22. C. Shan, S. Gong, and P. McOwan Facial expression recognition based on local binary patterns a comprehensive study Image Vis. Comput. 27 2009 803 816
23. P. Ekman, and E.L. Rosenberg What the Face Reveals Basic and Applied Studies of Spontaneous Expression using the Facial Action Coding System (FACS) 2005 Oxford University Press Oxford, New York
24. I. Kotsia, and I. Pitas Facial expression recognition in image sequences using geometric deformation features and support vector machines IEEE Trans. Image Process. 16 2007 172 187
25. G. Zhao, and M. Pietikäinen Dynamic texture recognition using local binary patterns with an application to facial expressions IEEE Trans. Pattern Anal. Mach. Intell. 29 2007 915 928
26. I. Cohen, N. Sebe, A. Garg, L.S. Chen, and T.S. Huang Facial expression recognition from video sequences temporal and static modeling Comput. Vis. Image Underst. 91 2003 160 187
27. M.F. Valstar, H. Gunes, M. Pantic, How to distinguish posed from spontaneous smiles using geometric features, in: Proceedings of the International Conference on Multimodal Interfaces, 2007, pp. 38-45.
28. T. Cootes, G. Edward, and C. Taylor Active appearance models IEEE Trans. Pattern Anal. Mach. Intell. 23 2001 681 685
29. D. Cristinacce, and T. Cootes Automatic feature localisation with constrained local models Pattern Recognition 41 2008 3054 3067
30. P. Tresaderrn, H. Bhaskar, S. Adeshina, C. Taylor, T. Cootes, Combining local and global shape models for deformable object matching, in: Proceedings of the British Conference on Machine Vision, 2009, pp. 1-12.
31. T. Cootes, C. Twining, V. Petrovic, K. Babalola, and C. Taylor Computing accurate correspondences across groups of images IEEE Trans. Pattern Anal. Mach. Intell. 32 2010 1994 2005
32. A. Aubrey, V. Kajicc, I. Cingovska, P. Rosin, D. Marshall, Mapping and manipulating facial dynamics, in: Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition, 2011, pp. 639-645.
33. M. Bartlett, P. Viola, T. Sejnowski, J. Larsen, J. Hager, P. Ekman, Classifying facial actions, in: Proceedings of the Neural Information Processing Systems, vol. 8, pp. 823-829.
34. M. Pantic, and L.J.M. Rothkrantz Facial action recognition for facial expression analysis from static face images IEEE Trans. Syst. Man Cybern., Part B 34 2004 1449 1461
35. J.R. Quinlan C4.5 Programs for Machine Learning 1993 Morgan Kaufmann
36. R. Jensen, C. Cornelis, A new approach to fuzzy-rough nearest neighbour classification, in: Proceedings of the International Conference on Rough Sets and Current Trends in Computing, vol. 5306, Springer, 2008, pp. 310-319.
37. J.C. Platt, Fast training of support vector machines using sequential minimal optimization, in: Advances in Kernel Methods, MIT Press, Cambridge, MA, USA, 1999, pp. 185-208.
38. S. Le Cessie, and J.C. Van Houwelingen Ridge estimators in logistic regression Appl. Stat. 41 1992 191 201
39. C. Juanjuan, Z. Zheng, S. Han, Z. Gang, Facial expression recognition based on PCA reconstruction, in: Proceedings of the International Conference on Computer Science and Education, 2010, pp. 195-198.
40. M. Pantic, M.F. Valstar, R. Rademaker, L. Maat, Web-based database for facial expression analysis, in: Proceedings of the IEEE International Conference on Multimedia and Expo, Amsterdam, The Netherlands, 2005, pp. 317-321.
41. M.F. Valstar, M. Pantic, Induced disgust, happiness and surprise: an addition to the MMI facial expression database in: Proceedings of the International Conference on Language Resources and Evaluation, Workshop on EMOTION, 2010, pp. 65-70.
42. R. Contreras, O. Starostenko, V. Alarcon-Aquino, L. Flores-Pulido, Facial feature model for emotion recognition using fuzzy reasoning, in: Proceedings of the Conference on Pattern Recognition: Advances in Pattern Recognition, Springer-Verlag, Berlin, Heidelberg, 2010, pp. 11-21.
43. S. ur Rehman, L. Liu, H. Li, Lipless tracking and emotion estimation, in: Proceedings of the International Conference on Signal-Image Technologies and Internet-Based System, 2007, pp. 768-774.
44. S. Koelstra, M. Pantic, and I. Patras A dynamic texture-based approach to recognition of facial actions and their temporal models IEEE Trans. Pattern Anal. Mach. Intell. 32 2010 1940 1954
45. T. Cootes, C. Taylor, D. Cooper, and J. Graham Active shape models - their training and application Comput. Vis. Image Underst. 61 1995 38 59
46. B. Abboud, F. Davoine, and M. Dang Facial expression recognition and synthesis based on an appearance model Signal Process.: Image Commun. 19 2004 723 740
47. V. Consonni, D. Ballabio, and R. Todeschini Evaluation of model predictive ability by external validation techniques J. Chemometrics 24 2010 194 201
48. R. Kohavi A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection 1995 Morgan Kaufmann pp. 1137-1143
49. G. Tam, H. Fang, A. Aubrey, P. Grant, P. Rosin, D. Marshall, and M. Chen Visualization of time-series data in parameter space for understanding facial dynamics Comput. Graphics Forum 30 2011 901 910
50. D. Holten, and J.J. Van Wijk Evaluation of cluster identification performance for different PCP variants Comput. Graphics Forum 29 2010 793 802