Facial action recognition combining heterogeneous features via multikernel learning

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 42, Issue 4, 2012, Pages 993-1005

  Senechal, Thibaud ^a   Rapp, Vincent ^a   Salam, Hanan ^b   Seguier, Renaud ^b   Bailly, Kevin ^a   Prevost, Lionel ^c  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^b CNRS   (France)

^c UNIVERSITÉ DES ANTILLES   (France)

Author keywords

multikernel learning; Active appearance model (AAM); facial action unit (AU); facial expression recognition and analysis (FERA) challenge; local Gabor binary pattern (LGBP)

Indexed keywords

ACTIVE APPEARANCE MODELS; FACIAL ACTION; FACIAL EXPRESSION RECOGNITION; LOCAL GABOR BINARY PATTERNS; MULTI-KERNEL;

GESTURE RECOGNITION; GRAPHIC METHODS; RADIAL BASIS FUNCTION NETWORKS; STATISTICAL METHODS; SUPPORT VECTOR MACHINES;

FACE RECOGNITION;

EID: 84864127007     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2012.2193567     Document Type: Article

References (54)

1. A. Mehrabian and M. Wiener, "Decoding of inconsistent communications," J. Pers. Soc. Psychol., vol. 6, no. 1, pp. 109-114, May 1967.
2. N. Ambady and R. Rosenthal, "Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis," Psychol. Bull., vol. 111, no. 2, pp. 256-274, 1992.
3. J. Russell and J. Dols, The Psychology of Facial Expression. Cambridge, U.K.: Cambridge Univ. Press, 1997, ser. Studies in Emotion and Social Interaction. [Online]. Available: http://books.google.fr/books?id= rr8e5oMlj44C
4. M. Knapp and J. Hall, Nonverbal Communication in Human Interaction, 4th ed. San Diego, CA: Harcourt College Pub., 1996.
5. M. Pantic, Face for Ambient Interface. New York: Springer-Verlag, 2006, pp. 35-66.
6. D. Keltner and P. Ekman, Facial Expression of Emotion. New York: Guilford Pub., 2000.
7. K. Scherer, "What are emotions? And how can they be measured?" Soc. Sci. Inf., vol. 44, no. 4, pp. 695-729, 2005. (Pubitemid 41666684)
8. P. Ekman and W. Friesen, Facial Action Coding System (facs): A Technique for the Measurement of Facial Actions. Menlo Park, CA: Consulting Psychologists Press, 1978.
9. M. Pantic and L. Rothkrantz, "Expert system for automatic analysis of facial expressions," Image Vis. Comput., vol. 18, no. 11, pp. 881-905, Aug. 2000.
10. P. Phillips, P. Flynn, T. Scruggs, K. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, "Overview of the face recognition grand challenge," in Proc. IEEE CVPR, Jun. 2005, pp. 947-954. (Pubitemid 43897294)
11. P. Phillips, H. Moon, P. Rauss, and S. Rizvi, "The feret evaluation methodology for face-recognition algorithms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 10, pp. 1090-1104, Oct. 2000.
12. T. Banziger and K. R. Scherer, "Using actor portrayals to systematically study multimodal emotion expression: The Gemep corpus," in Proc. Affective Comput. Intell. Interact., 2007, pp. 476-487.
13. W. Zhang, S. Shan, W. Gao, X. Chen, and H. Zhang, "LGBPHS: A novel non-statistical model for face representation and recognition," in Proc. IEEE ICCV, 2005, pp. 786-791.
14. T. Cootes, G. Edwards, and C. Taylor, "Active appearance models," in Proc. IEEE ECCV, 1998, pp. 484-498. (Pubitemid 128093046)
15. A. Rakotomamonjy, F. Bach, S. Canu, and Y. Grandvalet, "SimpleMKL," J. Mach. Learn. Res., vol. 9, pp. 2491-2521, 2008.
16. Y. Zhu, F. De la Torre, J. Cohn, and Y.-J. Zhang, "Dynamic cascades with bidirectional bootstrapping for spontaneous facial action unit detection," in Proc. ACII, 2009, pp. 1-8.
17. P. Viola and M. Jones, "Robust real-time face detection," Int. J. Comput. Vis., vol. 57, no. 2, pp. 137-154, May 2004.
18. M. Eckhardt, I. Fasel, and J. Movellan, "Towards practical facial feature detection," Int. J. Pattern Recognit. Artif. Intell., vol. 23, no. 3, pp. 379- 400, May 2009.
19. T. Wu, N. Butko, P. Ruvolo, J. Whitehill, M. Bartlett, and J. Movellan, "Action unit recognition transfer across datasets," in Proc. IEEE FG/FERA, 2011, pp. 889-896.
20. V. Rapp, T. Senechal, K. Bailly, and L. Prevost, "Multiple kernel learning svm and statistical validation for facial landmark detection," in Proc. IEEE Int. Conf. FG, 2011, pp. 265-271.
21. I. Kotsia and I. Pitas, "Facial expression recognition in image sequences using geometric deformation features and support vectormachines," IEEE Trans. Image Process., vol. 16, no. 1, pp. 172-187, Jan. 2007. (Pubitemid 46437480)
22. A. Asthana, J. Saragih, M. Wagner, and R. Goecke, "Evaluating aam fitting methods for facial expression recognition," in Proc. 3rd Int. Conf. ACII, 2009, pp. 1-8.
23. J. Saragih, S. Lucey, and J. Cohn, "Face alignment through subspace constrained mean-shifts," in Proc. IEEE ICCV, 2009, pp. 1034-1041.
24. Z. Wen and T. Huang, "Capturing subtle facial motions in 3D face tracking," in Proc. IEEE ICCV, 2003, pp. 1343-1350.
25. J. Cohn, L. Reed, Z. Ambadar, J. Xiao, and T. Moriyama, "Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior," in Proc. IEEE Int. Conf. SMC, 2004, vol. 1, pp. 610-616. (Pubitemid 40415926)
26. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 1, pp. 39-58, Jan. 2009.
27. M. Valstar andM. Pantic, "Fully automatic facial action unit detection and temporal analysis," in Proc. IEEE Int. Conf. CVPR, 2006, p. 149.
28. I. Cohen, N. Sebe, A. Garg, L. Chen, and T. Huang, "Facial expression recognition from video sequences: Temporal and static modeling," Comput. Vis. Image Understand., vol. 91, no. 1/2, pp. 160-187, Jul. 2003.
29. N. Sebe, M. Lew, Y. Sun, I. Cohen, T. Gevers, and T. Huang, "Authentic facial expression analysis," Image Vis. Comput., vol. 25, no. 12, pp. 1856- 1863, Dec. 2007. (Pubitemid 47333322)
30. M. Bartlett, G. Littlewort,M. Frank, I. Fasel, and J.Movellan, "Automatic recognition of facial actions in spontaneous expressions," J. Multimedia, vol. 1, no. 6, pp. 22-35, Sep. 2006.
31. J. Bazzo andM. Lamar, "Recognizing facial actions using Gabor wavelets with neutral face average difference," in Proc. IEEE Int. Conf. AFGR, 2004, pp. 505-510.
32. Y. Tong, W. Liao, and Q. Ji, "Facial action unit recognition by exploiting their dynamic and semantic relationships," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1683-1699, Oct. 2007.
33. J. Whitehill and C. Omlin, "Haar features for facs au recognition," in Proc. IEEE Int. Conf. AFGR, 2006, pp. 97-101. (Pubitemid 44711334)
34. C. Chuang and F. Shih, "Recognizing facial action units using independent component analysis and support vector machine," Pattern Recognit., vol. 39, no. 9, pp. 1795-1798, Sep. 2006. (Pubitemid 43867576)
35. S. Moore, R. Bowden, and U. Guildford, "The effects of pose on facial expression recognition," in Proc. BMVC, 2009, pp. 1-11.
36. B. Fasel, F. Monay, and D. Gatica-Perez, "Latent semantic analysis of facial action codes for automatic facial expression recognition," in Proc. 6th ACM Int. Workshop MIR, 2004, pp. 181-188. (Pubitemid 40390955)
37. M. Valstar, B. Jiang, M. Méhu, M. Pantic, and K. Scherer, "The first facial expression recognition and analysis challenge," in Proc. IEEE Int. Conf. FG, 2011, pp. 921-926.
38. I. Essa and A. Pentland, "Coding, analysis, interpretation, and recognition of facial expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 757-763, Jul. 1997. (Pubitemid 127762267)
39. M. Valstar, M. Pantic, and I. Patras, "Motion history for facial action detection in video," in Proc. IEEE Int. Conf. SMC, 2004, vol. 1, pp. 635-640. (Pubitemid 40415930)
40. S. Koelstra, M. Pantic, and I. Patras, "A dynamic texture based approach to recognition of facial actions and their temporal models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp. 1940-1954, Nov. 2010.
41. D. Chetverikov and R. Péteri, "A brief survey of dynamic texture description and recognition," in Proc. Comput. Recognit. Syst., 2005, pp. 17-26.
42. G. Zhao and M. Pietikainen, "Dynamic texture recognition using local binary patterns with an application to facial expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 915-928, Jun. 2007. (Pubitemid 46667407)
43. Y. Tian, T. Kanade, and J. Cohn, "Facial expression analysis," in Handbook of Face Recognition. New York: Springer-Verlag, 2005, pp. 247-276.
44. Y. Zhang and Q. Ji, "Active and dynamic information fusion for facial expression understanding from image sequences," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 699-714, May 2005. (Pubitemid 40608403)
45. S. Chew, P. Lucey, S. Lucey, J. Saragih, J. Cohn, and S. Sridharan, "Person-independent facial expression detection using constrained local models," in Proc. IEEE FG/FERA, 2011, pp. 915-920.
46. T. Baltrusaitis, D. McDuff, N. Banda, M. Mahmoud, R. Kaliouby, P. Robinson, and R. Picard, "Real-time inference of mental states from facial expressions and upper body gestures," in Proc. IEEE FG/FERA, 2011, pp. 909-914.
47. T. Ojala, M. Pietikainen, and D. Harwood, "A comparative study of texture measures with classification based on featured distributions," Pattern Recognit., vol. 29, no. 1, pp. 51-59, Jan. 1996.
48. A. Sattar, Y. Aidarous, S. Le Gallou, and R. Seguier, "Face alignment by 2.5D active appearance model optimized by simplex," in Proc. ICVS, 2007, pp. 1-10.
49. A. Savran, N. Alyuz, H. Dibeklioglu, O. Celiktutan, B. Gokberk, B. Sankur, and L. Akarun, "Bosphorus database for 3D face analysis," in Biometrics Identity Management. New York: Springer-Verlag, 2008, pp. 47-56.
50. A. Ross, "Procrustes analysis," Dept. Computer Science Engineering, Univ. South Carolina, Columbia, SC, Course Rep., 2004.
51. B. Scholkopf and A. Smola, Learning With Kernels. Cambridge, MA: MIT Press, 2002.
52. G. Lanckriet, N. Cristianini, P. Bartlett, L. Ghaoui, and M. Jordan, "Learning the kernel matrix with semidefinite programming," J. Mach. Learn. Res., vol. 5, pp. 27-72, Dec. 2004.
53. T. Senechal, K. Bailly, and L. Prevost, "Automatic facial action detection using histogram variation between emotional states," in Proc. ICPR, 2010, pp. 3752-3755.
54. T. Kanade, J. Cohn, and Y. Tian, "Comprehensive database for facial expression analysis," in Proc. IEEE Int. Conf. AFGR, 2000, pp. 46-53.