Automatic facial expression recognition using features of salient facial patches

IEEE Transactions on Affective Computing

Volumn 6, Issue 1, 2015, Pages 1-12

  Happy, S L ^a   Routray, Aurobinda ^a  

^a INDIAN INSTITUTE OF TECHNOLOGY   (India)

Author keywords

Facial expression analysis; facial landmark detection; feature selection; low resolution image; salient facial patches

Indexed keywords

CLASSIFICATION (OF INFORMATION); FEATURE EXTRACTION; GESTURE RECOGNITION;

AUTOMATIC FACIAL EXPRESSION RECOGNITION; DISCRIMINATIVE FEATURES; EXPRESSION RECOGNITION; FACIAL EXPRESSION ANALYSIS; FACIAL EXPRESSION RECOGNITION; FACIAL LANDMARK DETECTION; LOW RESOLUTION IMAGES; SALIENT FACIAL PATCHES;

FACE RECOGNITION;

EID: 84924061133     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAFFC.2014.2386334     Document Type: Article

References (63)

1. R. A. Calvo and S. D'Mello, "Affect detection: An interdisciplinary review of models, methods, and their applications," IEEE Trans. Affective Comput., vol. 1, no. 1, pp. 18-37, Jan. 2010.
2. C. Izard, "Innate and universal facial expressions: Evidence from developmental and cross-cultural research," Psychological Bull., vol. 115, pp. 288-299, 1994.
3. P. Ekman, W. V. Friesen, and J. C. Hager, "FACS Manual," Salt Lake City, UT, USA: A Human Face, May 2002.
4. J. Whitehill, M. S. Bartlett, and J. Movellan, "Automatic facial expression recognition," in Social Emotions in Nature and Artifact. London, U.K.: Oxford Univ. Press, 2013.
5. Y. Chang, C. Hu, and M. Turk, "Manifold of facial expression," in Proc. IEEE Int. Workshop Anal. Modeling Faces Gestures, 2003, pp. 28-35.
6. M. Pantic and I. Patras, "Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences," IEEE Trans. Syst., Man, Cybern., vol. 36, no. 2, pp. 433-449, Apr. 2006.
7. M. Pantic and L. Rothkrantz, "Facial action recognition for facial expression analysis from static face images," IEEE Trans. Syst., Man, Cybern., vol. 34, no. 3, pp. 1449-1461, Jun. 2004.
8. I. Cohen, N. Sebe, A. Garg, L. Chen, and T. Huang, "Facial expression recognition from video sequences: Temporal and static modeling,"Comput. Vis. Image Understanding, vol. 91, pp. 160-187, 2003.
9. S. M. Lajevardi and Z. M. Hussain, "Automatic facial expression recognition: Feature extraction and selection," Signal, Image Video Process., vol. 6, no. 1, pp. 159-169, 2012.
10. G. Zhao and M. Pietikainen, "Dynamic texture recognition using local binary patterns with an application to facial expressions,"IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 915-928, Jun. 2007.
11. T. Jabid, M. Kabir, and O. Chae, "Robust facial expression recognition based on local directional pattern," ETRI J., vol. 32, pp. 784-794, 2010.
12. th IEEE Int. Conf. Adv. Video Signal Based Surveillance, 2010, pp. 526-532.
13. C. Shan and R. Braspenning, "Recognizing facial expressions automatically from video," in Handbook of Ambient Intelligence and Smart Environments, New York, NY, USA: Springer-Verlag, 2010, pp. 479-509.
14. C. Shan, S. Gong, and P. W. McOwan, "Robust facial expression recognition using local binary patterns," in Proc. IEEE Int. Conf. Image Process., 2005, pp. II-370.
15. C. Shan and T. Gritti, "Learning discriminative LBP-histogram bins for facial expression recognition," in Proc. Brit. Mach. Vis. Conf., 2008, pp. 1-10.
16. M. Song, D. Tao, Z. Liu, X. Li, and M. Zhou, "Image ratio features for facial expression recognition application," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 40, no. 3, pp. 779-788, Jun. 2010.
17. L. Zhang and D. Tjondronegoro, "Facial expression recognition using facial movement features," IEEE Trans. Affective Comput., vol. 2, no. 4, pp. 219-229, Oct. 2011.
18. Y. Zhang and Q. Ji, "Active and dynamic information fusion for facial expression understanding from image sequences," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 699-714, May 2005.
19. Y. Tian, T. Kanade, and J. F. Cohn, "Recognizing lower face action units for facial expression analysis," in Proc. IEEE Int. Conf. Autom. Face Gesture Recog., 2000, pp. 484-490.
20. M. Z. Uddin, J. J. Lee, and T.-S. Kim, "An enhanced independent component-based human facial expression recognition from video," IEEE Trans. Consum. Electron., vol. 55, no. 4, pp. 2216-2224, Nov. 2009.
21. M. F. Valstar and M. Pantic, "Combined support vector machines and hidden Markov models for modeling facial action temporal dynamics," in Proc. IEEE Int. Conf. Human-Comput. Interaction, 2007, pp. 118-127.
22. T. Cootes, G. Edwards, and C. Taylor, "Active appearance models,"IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp. 681-685, Jun. 2001.
23. th Int. Conf. Autom. Face Gesture Recog., 2006, pp. 155-160.
24. A. B. Ashraf, S. Lucey, T. C. Jeffrey F. Cohn, Z. Ambadar, K. M. Prkachin, and P. E. Solomon, "The painful face-pain expression recognition using active appearance models," Image Vis. Comput., vol. 27, no. 12, pp. 1788-1796, 2009.
25. B. Abboud, F. Davoine, and M. Dang, "Facial expression recognition and synthesis based on an appearance model," Signal Process.: Image Commun., vol. 19, no. 8, pp. 723-740, 2004.
26. rd Int. Conf. Affective Comput. Intell. Interaction Workshops, 2009, pp. 1-8.
27. th IEEE Int. Conf. Autom. Face Gesture Recog., 2008, pp. 1-6.
28. D. Cristinacce and T. F. Cootes, "Feature detection and tracking with constrained local models," in Proc. Brit. Mach. Vis. Conf., 2006, pp. 929-938.
29. J. M. Saragih, S. Lucey, and J. F. Cohn, "Deformable model fitting by regularized landmark mean-shift," Int. J. Comput. Vis., vol. 91, no. 2, pp. 200-215, 2011.
30. A. Asthana, S. Zafeiriou, S. Cheng, and M. Pantic, "Robust discriminative response map fitting with constrained local models,"in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 3444-3451.
31. S. W. Chew, P. Lucey, S. Lucey, J. Saragih, J. F. Cohn, I. Matthews, and S. Sridharan, "In the pursuit of effective affective computing: The relationship between features and registration," IEEE Trans. Syst., Man, Cybernet., Part B: Cybern., vol. 42, no. 4, pp. 1006-1016, Aug. 2012.
32. M. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Recognizing facial expression: Machine learning and application to spontaneous behavior," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., 2005, pp. 568-573.
33. M. Lyons, J. Budynek, and S. Akamatsu, "Automatic classification of single facial images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 12, pp. 1357-1362, Dec. 1999.
34. A. Hadid, M. Pietikainen, and T. Ahonen, "A discriminative feature space for detecting and recognizing faces," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., 2004, pp. 797-804.
35. th Int. Conf. Intell. Human Comput. Interaction, 2012, pp. 1-5.
36. A. Dhall, A. Asthana, R. Goecke, and T. Gedeon, "Emotion recognition using PHOG and LPQ features," in Proc. IEEE Int. Conf. Autom. Face Gesture Recog. Workshops, 2011, pp. 878-883.
37. C. Shan, S. Gong, and P. W. McOwan, "Facial expression recognition based on local binary patterns: A comprehensive study,"Image Vis. Comput., vol. 27, no. 6, pp. 803-816, 2009.
38. K. I. Kim, K. Jung, and H. J. Kim, "Face recognition using kernel principal component analysis," IEEE Signal Process. Lett., vol. 9, no. 2, pp. 40-42, Feb. 2002.
39. A. J. Calder, A. M. Burton, P. Miller, A. W. Young, and S. Akamatsu, "A principal component analysis of facial expressions," Vis. Res., vol. 41, no. 9, pp. 1179-1208, 2001.
40. H.-B. Deng, L.-W. Jin, L.-X. Zhen, and J.-C. Huang, "A new facial expression recognition method based on local Gabor filter bank and PCA plus LDA," Int. J. Inform. Technol., vol. 11, no. 11, pp. 86-96, 2005.
41. th IEEE Int. Conf. Image Process., 2013, pp. 1192-1196.
42. C. Shan, S. Gong, and P. W. McOwan, "A comprehensive empirical study on linear subspace methods for facial expression analysis,"in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshop, 2006, pp. 153-153.
43. S.-K. Oh, S.-H. Yoo, and W. Pedrycz, "Design of face recognition algorithm using PCA-LDA combined for hybrid data pre-processing and polynomial-based RBF neural networks: Design and its application," Expert Syst. Appl., vol. 40, no. 5, pp. 1451-1466, 2013.
44. Y. Rahulamathavan, R.-W. Phan, J. A. Chambers, and D. J. Parish, "Facial expression recognition in the encrypted domain based on local fisher discriminant analysis," IEEE Trans. Affective Comput., vol. 4, no. 1, pp. 83-92, Jan.-Mar. 2013.
45. H. Mohammadzade and D. Hatzinakos, "Projection into expression subspaces for face recognition from single sample per person,"IEEE Trans. Affective Comput., vol. 4, no. 1, pp. 69-82, Jan.-Mar. 2013.
46. S. Moore and R. Bowden, "Local binary patterns for multi-view facial expression recognition," Comput. Vis. Image Understanding, vol. 115, no. 4, pp. 541-558, 2011.
47. L. Zhong, Q. Liu, P. Yang, B. Liu, J. Huang, and D. N. Metaxas, "Learning active facial patches for expression analysis," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 2562-2569.
48. L. Unzueta, W. Pimenta, J. Goenetxea, L. P. Santos, and F. Dornaika, "Efficient generic face model fitting to images and videos," Image Vis. Comput., vol. 32, no. 5, pp. 321-334, 2014.
49. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2001, pp. 511-518.
50. D. Nguyen, D. Halupka, P. Aarabi, and A. Sheikholeslami, "Real-time face detection and lip feature extraction using field-programmable gate arrays," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 36, no. 4, pp. 902-912, Aug. 2006.
51. rd ed. Pearson Education, New Jersey, US, 2008.
52. N. Otsu, "A threshold selection method from gray-level histograms,"IEEE Trans. Syst., Man Cybern., vol. 9, no. 1, pp. 62-66, Jan. 1979.
53. T. Ojala, M. Pietikainen, and T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971-987, Jul. 2002.
54. P. N. Belhumeur, J. P. Hespanha, and D. Kriegman, "Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,"IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711-720, Jul. 1997.
55. C. Cortes and V. Vapnik, "Support-vector networks," Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995.
56. C.-W. Hsu and C.-J. Lin, "A comparison of methods for multiclass support vector machines," IEEE Trans. Neural Netw., vol. 13, no. 2, pp. 415-425, Mar. 2002.
57. M. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, "Coding facial expressions with gabor wavelets," In Proc. Third IEEE Int. Conf. Automat. Face Gesture Recog., 1998, pp. 200-205.
58. rd IEEE Workshop CVPR Human Communicative Behavior Anal., 2010, pp. 1-8.
59. Y. Tian, T. Kanade, and J. F. Cohn, "Facial expression recognition,"in Handbook of Face Recognition. London, U.K.: Springer, 2011, pp. 487-519.
60. M. Sokolova and G. Lapalme, "A systematic analysis of performance measures for classification tasks," Inform. Process. Manage., vol. 45, no. 4, pp. 427-437, 2009.
61. P. Ahmad, N. A. Hossein, G. Marina, and Y. N. Svetlana, "Gauss-Laguerre wavelet textural feature fusion with geometrical information for facial expression identification," EURASIP J. Image Video Process., vol. 2012, no. 1, pp. 1-13, 2012.
62. Z. Zhang, C. Fang, and X. Ding, "Facial expression analysis across databases," in Proc. IEEE Int. Conf. Multimedia Technol., 2011, pp. 317-320.
63. rd Int. Conf. Audio-Video-Based Biometric Person Authentication, 2001, pp. 90-95.