Multilayer architectures for facial action unit recognition

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 42, Issue 4, 2012, Pages 1027-1038

  Wu, Tingfan ^a   Butko, Nicholas J ^a   Ruvolo, Paul ^a   Whitehill, Jacob ^a   Bartlett, Marian S ^a   Movellan, Javier R ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Action unit recognition; facial expression recognition; Gabor energy filters (GEFs); local binary patterns (LBPs)

Indexed keywords

ACTION UNIT; COMPUTER VISION APPLICATIONS; DATA SETS; DUAL LAYER; EMPIRICAL ANALYSIS; EXPRESSION RECOGNITION; FACIAL ACTION UNIT RECOGNITION; FACIAL EXPRESSION RECOGNITION; GABOR ENERGY; HIDDEN LAYERS; LOCAL BINARY PATTERNS; MULTILAYER ARCHITECTURE; MULTIPLE RESOLUTIONS; NON-LINEAR TEXTURES; NONLINEAR FILTER; RECOGNITION PERFORMANCE; SECOND LAYER; SINGLE LAYER;

COMPUTER VISION; CONTENT BASED RETRIEVAL; FEATURE EXTRACTION; GESTURE RECOGNITION; TEXTURES;

FACE RECOGNITION;

EID: 84864139496     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2012.2195170     Document Type: Article

References (48)

1. X. Sun, H. Xu, C. Zhao, and J. Yang, "Facial expression recognition based on histogram sequence of local Gabor binary patterns," in Proc. IEEE Conf. Cybern. Intell. Syst., 2008, pp. 158-163.
2. T. Marks, J. R. Hershey, and J. R. Movellan, "Tracking motion, deformation, and texture using conditionally Gaussian processes," Pattern Anal. Mach. Intell., vol. 32, no. 2, pp. 348-363, Feb. 2010.
3. A. Ryan, J. Cohn, S. Lucey, J. Saragih, P. Lucey, F. De la Torre, and A. Rossi, "Automated facial expression recognition system," in Proc. 43rd IEEE Annu. Int. Carnahan Conf. Secur. Technol., 2009, pp. 172-177.
4. A. Ashraf, S. Lucey, J. Cohn, T. Chen, Z. Ambadar, K. Prkachin, and P. Solomon, "The painful face-pain expression recognition using active appearance models," Image Vis. Comput., vol. 27, no. 12, pp. 1788-1796, Nov. 2009.
5. P. Lucey, S. Lucey, and J. Cohn, "Registration invariant representations for expression detection," in Proc. IEEE Int. Conf. Digit. Image Comput., Tech. Appl., 2010, pp. 255-261.
6. M. F. Valstar and M. Pantic, "Fully automatic recognition of the temporal phases of facial actions," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 1, pp. 28-43, Feb. 2012.
7. G. Littlewort, J. Whitehill, T. Wu, I. Fasel, M. Frank, J. Movellan, and M. Bartlett, "The computer expression recognition toolbox (CERT)," in Proc. Autom. Face Gesture Recogn., 2008, pp. 1-2.
8. C. Shan, S. Gong, and P. McOwan, "Facial expression recognition based on local binary patterns: A comprehensive study," Image Vis. Comput., vol. 27, no. 6, pp. 803-816, May 2009.
9. G. Zhao and M. Pietikainen, "Dynamic texture recognition using local binary patterns with an application to facial expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 915-928, Jun. 2007. (Pubitemid 46667407)
10. P. Michel and R. El Kaliouby, "Real time facial expression recognition in video using support vector machines," in Proc. 5th ACM Int. Conf. Multimodal Interf., 2003, pp. 258-264. (Pubitemid 40681880)
11. G. Littlewort, M. Bartlett, I. Fasel, J. Susskind, and J. Movellan, "Dynamics of facial expression extracted automatically from video," Image Vis. Comput., vol. 24, no. 6, pp. 615-625, Jun. 2006. (Pubitemid 43867835)
12. F. De la Torre and J. F. Cohn, "Facial expression analysis," in Guide to Visual Analysis of Humans: Looking at People. Berlin, Germany: Springer-Verlag, 2011.
13. G. Donato, M. Bartlett, J. Hager, P. Ekman, and T. Sejnowski, "Classifying facial actions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 10, pp. 974-989, Oct. 1999. (Pubitemid 129304131)
14. J. Whitehill, G. Littlewort, I. Fasel, M. Bartlett, and J. R. Movellan, "Toward practical smile detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 11, pp. 2106-2111, Nov. 2009.
15. J. R. Movellan, "Optimal control of dynamic Bayes nets," UCSD, La Jolla, CA, 2005, vol. MPLab Tutorials.
16. G. Littlewort, M. Bartlett, I. Fasel, J. Susskind, and J. Movellan, "An automatic system for measuring facial expression in video," Image Vision Computing, vol. 24, no. 6, pp. 615-625.
17. M. Bartlett, G. Donato, J. R.Movellan, and J. Hager, "Face image analysis for expression measurement and detection of deceit," in Proc. 6th Symp. Neural Comput., 1999, pp. 8-16.
18. M. S. Bartlett, G. Littlewort, C. Lainscsek, I. Fasel, and J. Movellan, "Recognition of facial actions in spontaneous expressions," J. Multimedia, vol. 1, no. 6, pp. 22-35, 2006.
19. V. Vapnik, Statistical Learning Theory. Hoboken, NJ: Wiley, 1998.
20. Y. Freund and R. Schapire, "A decision-Theoretic generalization of online learning and an application to boosting," in Computational Learning Theory. New York: Springer-Verlag, 1995, pp. 23-37.
21. J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting," Ann. Stat., vol. 28, no. 2, pp. 337-407, 2000. (Pubitemid 33227445)
22. G. Hinton, S. Osindero, and Y. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006. (Pubitemid 44024729)
23. Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Comput., vol. 1, no. 4, pp. 541-551, 1989.
24. J. Susskind, G. Hinton, J. Movellan, and A. Anderson, "Generating facial expressions with deep belief nets," Affect. Comput., Emotion Model., Synth. Recogn., pp. 421-440, 2009.
25. D. C. Cirȩsan, U. Meier, J. Masci, L. M. Gambardella, and J. Schmidhuber, "High-performance neural networks for visual object classification," IDSIA/USI-SUPSI, Manno, Switzerland, Tech. Rep. IDSIA-01-11, Feb. 2011.
26. M. F. Valstar, B. Jiang, M. Méhu, M. Pantic, and K. Scherer, "The first facial expression recognition and analysis challenge," in Proc. IEEE Int. Conf. Autom. Face Gesture Recogn., 2011, pp. 921-926.
27. P. Ekman and W. Friesen, Facial Action Coding System (FACS): A Technique for the Measurement of Facial Action. Palo Alto, CA: Consulting, 1978.
28. T. Banziger and K. R. Scherer, Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) Corpus, K. R. Scherer, T. Banziger, and E. B. Roesch, Eds. London, U.K.: Oxford Univ. Press, 2010, ser. Blueprint for Affective Computing: A Sourcebook, Series in Affective Science, ch. 6.1, pp. 271-294.
29. T. Kanade, J. Cohn, and Y. L. Tian, "Comprehensive database for facial expression analysis," in Proc. IEEE Int. Conf. Autom. Face Gesture Recogn., 2000, p. 46.
30. M. Pantic, M. Valstar, R. Rademaker, and L. Maat, "Web-based database for facial expression analysis," in Proc. Int. Conf. Multimedia Expo., 2005, pp. 317-321. (Pubitemid 44668868)
31. in prep M. Frank, M. Bartlett, and J. Movellan, The M3 Database of Spontaneous Emotion Expression (University of Buffalo), 2010.
32. P. Lucey, J. Cohn, T. Kanade, J. SAragih, and Z. Ambadar, "The extended cohn-kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression," in Proc. CVPR Workshop Hum. Commun. Behav. Anal., 2010, pp. 94-101.
33. M. Eckhardt, I. Fasel, and J. Movellan, "Towards practical facial feature detection," Int. J. Pattern Recogn. Artif. Intell., vol. 23, no. 3, pp. 379-400, 2009.
34. P. Viola and M. Jones, "Robust real-Time face detection," Int. J. Comput. Vis., vol. 57, no. 2, pp. 137-154, May 2004.
35. C. Goodall (1991). Procrustes methods in the statistical analysis of shape. J. R. Stat. Soc.Ser. B,Methodological [Online]. 53(2), pp. 285-339. Available: http://www.jstor.org/stable/2345744
36. T. Ojala, M. Pietikäinen, and T. Mäenpää, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971-987, Jul. 2002. (Pubitemid 34835471)
37. C. Burges, "A tutorial on support vectormachines for pattern recognition," Data Mining Knowl. Disc., vol. 2, no. 2, pp. 121-167, 1998. (Pubitemid 128695475)
38. C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines, 2001. [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm
39. D. Green and J. Swets, Signal Detection Theory and Psychophysics. Los Altos, CA: Peninsula, 1966.
40. S. Mason and N. Graham, "Areas beneath the relative operating characteristics (roc) and relative operating levels (rol) curves: Statistical significance and interpretation," Q. J. Roy. Meteorol. Soc., vol. 128, no. 584, pp. 2145-2166, 2002. (Pubitemid 34995960)
41. S. Keerthi and C. Lin, "Asymptotic behaviors of support vector machines with Gaussian kernel," Neural Comput., vol. 15, no. 7, pp. 1667-1689, 2003. (Pubitemid 37035565)
42. T. Simon, M. Nguyen, F. De La Torre, and J. Cohn, "Action unit detection with segment-based svms," in Proc. IEEE CVPR, 2010, pp. 2737-2744.
43. T. Senechal, V. Rapp, H. Salam, R. Seguier, K. Bailly, and L. Prevost, "Combining aam coefficients with lgbp histograms in the multi-kernel svm framework to detect facial action units," in Proc. IEEE FG, 2011, pp. 860-865.
44. T. Wu, N. Butko, P. Ruvolo, J. Whitehill, M. Bartlett, and J. Movellan, "Action unit recognition transfer across datasets," in Proc. IEEE FG, 2011, pp. 889-896.
45. S. Moore, R. Bowden, and U. Guildford, "The effects of pose on facial expression recognition," in Proc. BMVC, London, U.K., 2009.
46. J. De Boner and P. Viola, "Structure driven image database retrieval," in Proc. Adv. Neural Inf. Process. Syst., 1998, pp. 866-872.
47. W. Zhang, S. Shan,W. Gao, X. Chen, and H. Zhang, "Local Gabor binary pattern histogram sequence (LGBPHS): A novel non-statistical model for face representation and recognition," in Proc. 10th IEEE ICCV, 2005, pp. 786-791. (Pubitemid 44055035)
48. R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, "Multi-pie," Image Vis. Comput., vol. 28, no. 5, pp. 807-813, 2010.