Multimodal learning for facial expression recognition

Pattern Recognition

Volumn 48, Issue 10, 2015, Pages 3191-3202

  Zhang, Wei ^a   Zhang, Youmei ^a   Ma, Lin ^b   Guan, Jingwei ^a   Gong, Shijie ^a  

^a SHANDONG UNIVERSITY   (China)

^b HUAWEI NOAH S ARK LAB   (China)

Author keywords

Facial expression recognition; Landmark; Multimodal learning; Texture

Indexed keywords

LEARNING SYSTEMS; TEXTURES;

FACIAL EXPRESSION RECOGNITION; FACIAL EXPRESSIONS; FACIAL IMAGES; LANDMARK; MULTI-MODAL LEARNING; MULTIMODAL INPUTS;

FACE RECOGNITION;

EID: 84931567876     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2015.04.012     Document Type: Article

References (31)

1. G.E. Hinton, and R.R. Salakhutdinov Reducing the dimensionality of data with neural networks Science 313 2006 504 507
2. A. Lorincz, L.A. Jeni, Emotional expression classification using time-series kernels, in: IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 889-895.
3. F. Long, T. Wu, J.R. Movellan, M.S. Bartlett, and G. Littlewort Learning spatiotemporal features by using independent component analysis with application to facial expression recognition Neurocomputing 93 2012 126 132
4. L.A. Jeni, J.M. Girard, J.F. Cohn, F.D.L. Torre, Continuous AU intensity estimation using localized, sparse facial feature space, in: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, 2013, pp. 1-7.
5. T. Wu, M.S. Bartlett, J.R. Movellan, Facial expression recognition using Gabor motion energy filters, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2010, pp. 42-47.
6. P. Yang, Q. Liu, D.N. Metaxas, Facial expression recognition using encoded dynamic features, in: IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1-8.
7. M. Liu, S. Shan, R. Wang, X. Chen, Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition, in: IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 1749-1756.
8. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, A.Y. Ng, Multimodal Deep Learning, in: International Conference on Machine Learning, 2011, pp. 689-696.
9. H. Hu, B. Liu, B. Wang, M. Liu, X. Wang, Multimodal DBN for predicting high-quality answers in cQA portals, in: Association for Computational Linguistics, 2013.
10. N. Srivastava, R. Salakhutdinov, Learning representations for multimodal data with deep belief nets, in: International Conference on Machine Learning, 2012.
11. N. Srivastava, R. Salakhutdinov, Multimodal learning with deep boltzmann machines, in: Neural Information Processing Systems, 2013, pp. 2222-2230.
12. M. Chen, Z. Xu, K. Weinberger, Marginalized denoising autoencoders for domain adaptation, in: International Conference on Machine Learning, 2012.
13. A. Krizhevsky, G.E. Hinton, Learning Multiple Layers of Features From Tiny Images, Technical Report, University of Toronto, 2009.
14. Y. Wanf, and Y. Zhang Non-negative matrix factorization a comprehensive review IEEE Trans. Knowl. Data Eng. 29 2013 1336 1353
15. T. Kanade, J.F. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in: IEEE International Conference on Automatic Face and Gesture Recognition, 2000, pp. 46-53.
16. G. Zhao, and M. Pietickaine Dynamic texture recognition using local binary patterns with an application to facial expression IEEE Trans. Pattern Anal. Mach. Intell. 29 2007 915 928
17. I. Lenz, H. Lee, and A. Saxena Deep learning for detecting robtic grasps Int. J. Robotics Res. 2014
18. A. Jalali, P. Ravikumar, S. Sanghavi, and C. Ruan A dirty model for multi-task learning Neural Inf. Process. Syst. 2010 964 972
19. M. Black, and Y. Yacoob Recognizing facial expressions in image sequences using local parameterized models of image motion Int. J. Comput. Vis. 25 1997 23 48
20. Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, in: Proceedings of the IEEE, 1998, pp. 2278-2324.
21. M. S. Bartlett, G. Littlewort, I. Fasel, J.R. Movellan, Real time face detection and facial expression recognition: development and applications to human computer interaction, in: IEEE Conference on Computer Vision and Pattern Recognition, 2003, p. 53.
22. K. Berns, J. Hirth, Control of Facial Expressions of the Humanoid Robot Head ROMAN, in: IEEE International Conference on Intelligent Robots and Systems, 2006, pp. 3119-3124.
23. E. Vural, M. Cetin, A. Ercil, G. Littlewort, M. Bartlett, J. Movellan, Automated drowsiness detection for improved driver safety-comprehensive databases for facial expression analysis, in: International Conference on Automotive Technologies, 2008.
24. P. Lucey, J. Cohn, S. Lucey, I. Matthews, S. Sridharan, K. Prkachin, Automatically detecting pain using facial actions, in: IEEE International Conference on Affective Computing and Intelligent Interaction, 2009, pp. 1-8.
25. N. Perveen, S. Gupta, K. Verma, Facial expression recognition using facial characteristic points and Gini index, in: IEEE Students Conference on Engineering and Systems, 2012, pp. 1-6.
26. L. Deng, Three classes of deep learning architectures and their applications: a tutorial survey, in: APSIPA Transactions on Signal and Information Processing, 2012.
27. L. Kang, P. Ye, Y. Li, D. Doermann, Convolutional neural networks for no-reference image quality assessment, in: IEEE Conference on Computer Vision and Pattern Recognition, 2014.
28. C. Strobl, A.L. Boulesteix, and T. Augustin Unbiased split selection for classification trees based on the Gini index Comput. Stat. Data Anal. 52 2007 483 501
29. D. Ciresan, U. Meier, J. Masci, and J. Schmidhuber Multi-column deep neural network for traffic sign classification Neural Netw. 32 2012 333 338
30. S. Wang, Z. Liu, and S. Lv A natural visible and infrared facial expression database for expression recognition and emotion inference IEEE Trans. Multimed. 12 2010 682 691
31. S. Wang, Y. Lv. Spontaneous facial expression recognition based on feature point tracking, in: IEEE 2011 Sixth International Conference on Image and Graphics (ICIG), 2011, pp. 760-765.