Human Facial Expression Recognition Using Stepwise Linear Discriminant Analysis and Hidden Conditional Random Fields

IEEE Transactions on Image Processing

Volumn 24, Issue 4, 2015, Pages 1386-1398

  Siddiqi, Muhammad Hameed ^a   Ali, Rahman ^a   Khan, Adil Mehmood ^b   Park, Young Tack ^c   Lee, Sungyoung ^a  

^a Kyung Hee University   (South Korea)

^b INNOPOLIS UNIVERSITY   (Russian Federation)

^c Soongsil University   (South Korea)

Author keywords

Facial Expressions; Hidden Conditional Random Fields; Hidden Markov Models; Stepwise Linear Discriminant Analysis

Indexed keywords

DISCRIMINANT ANALYSIS; FEATURE EXTRACTION; GESTURE RECOGNITION; HIDDEN MARKOV MODELS; IMAGE SEGMENTATION; MARKOV PROCESSES; RANDOM PROCESSES; TRELLIS CODES;

FACIAL EXPRESSION RECOGNITION; FACIAL EXPRESSIONS; HIDDEN CONDITIONAL RANDOM FIELDS; HIERARCHICAL RECOGNITION; HUMAN FACIAL EXPRESSIONS; MIXTURE OF GAUSSIANS; STEPWISE LINEAR DISCRIMINANT ANALYSIS; WITHIN-CLASS VARIANCES;

FACE RECOGNITION;

ADOLESCENT; ADULT; ALGORITHM; ANATOMY AND HISTOLOGY; AUTOMATED PATTERN RECOGNITION; BIOMETRY; COMPUTER ASSISTED DIAGNOSIS; DISCRIMINANT ANALYSIS; FACE; FACIAL EXPRESSION; FEMALE; HUMAN; IMAGE ENHANCEMENT; MALE; PHOTOGRAPHY; PROBABILITY; PROCEDURES; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; YOUNG ADULT;

ADOLESCENT; ADULT; ALGORITHMS; BIOMETRIC IDENTIFICATION; DISCRIMINANT ANALYSIS; FACE; FACIAL EXPRESSION; FEMALE; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MALE; MARKOV CHAINS; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; YOUNG ADULT;

EID: 84924605599     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2015.2405346     Document Type: Article

References (75)

1. th Conf. Image Vis. Comput. New Zealand, 2012, pp. 464-468.
2. th IEEE Can. Conf. Elect. Comput. Eng., 2013.
3. M. S. Bartlett, J. C. Hager, P. Ekman, and T. J. Sejnowski, "Measuring facial expressions by computer image analysis," Psychophysiology, vol. 36, no. 2, pp. 253-263, 1999.
4. A. Mehrabian, "Communication without words," Psychol. Today, vol. 2, no. 9, pp. 52-55, 1968.
5. V. Bettadapura. (2012). "Face expression recognition and analysis: The state of the art." [Online]. Available: http://arxiv.org/abs/1203.6722
6. th IEEE Hawaii Int. Conf. Syst. Sci., Hawaii, USA, Jan. 2004, pp. 348-363.
7. th Int. Conf. Natural Comput. (ICNC), vol. 3. Aug. 2010, pp. 1212-1216.
8. S. Moore and R. Bowden, "The effects of pose on facial expression recognition," in Proc. Brit. Mach. Vis. Conf., 2009, pp. 1-11.
9. Z. Zhu and Q. Ji, "Robust real-time face pose and facial expression recovery," in Proc. Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 1. Jun. 2006, pp. 681-688.
10. M. S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Recognizing facial expression: Machine learning and application to spontaneous behavior," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), vol. 2. Jun. 2005, pp. 568-573.
11. Y.-L. Tian, T. Kanade, and J. F. Cohn, "Facial expression analysis," in Handbook of Face Recognition. New York, NY, USA: Springer-Verlag, 2005, pp. 247-275.
12. C. Shan, S. Gong, and P. W. McOwan, "Facial expression recognition based on local binary patterns: A comprehensive study," Image Vis. Comput., vol. 27, no. 6, pp. 803-816, 2009.
13. C. Shan and R. Braspenning, "Recognizing facial expressions automatically from video," in Handbook of Ambient Intelligence and Smart Environments . New York, NY, USA: Springer-Verlag, 2010, pp. 479-509.
14. D. J. Krusienski, E. W. Sellers, D. J. McFarland, T. M. Vaughan, and J. R. Wolpaw, "Toward enhanced P300 speller performance," J. Neurosci. Methods, vol. 167, no. 1, pp. 15-21, 2008.
15. M. Ager, Z. Cvetkovic, and P. Sollich. (2013). "Phoneme classification in high-dimensional linear feature domains." [Online]. Available: http://arxiv.org/abs/1312.6849
16. M. Nusseck, D. W. Cunningham, C. Wallraven, and H. H. Bülthoff, "The contribution of different facial regions to the recognition of conversational expressions," J. Vis., vol. 8, no. 8, 2008, Art. ID 1.
17. K. Kaulard, D. W. Cunningham, H. H. Bülthoff, and C. Wallraven, "The MPI facial expression database - A validated database of emotional and conversational facial expressions," PloS One, vol. 7, no. 3, p. e32321, 2012.
18. K. L. Schmidt and J. F. Cohn, "Human facial expressions as adaptations: Evolutionary questions in facial expression research," Amer. J. Phys. Anthropol., vol. 116, no. S33, pp. 3-24, 2001.
19. X. Wang and K. K. Paliwal, "Feature extraction and dimensionality reduction algorithms and their applications in vowel recognition,"Pattern Recognit., vol. 36, no. 10, pp. 2429-2439, 2003.
20. Y. Pang, Y. Yuan, and X. Li, "Iterative subspace analysis based on feature line distance," IEEE Trans. Image Process., vol. 18, no. 4, pp. 903-907, Apr. 2009.
21. G. Aguilar-Torres, K. Toscano-Medina, G. Sanchez-Perez, M. Nakano-Miyatake, and H. Perez-Meana, "Eigenface-Gabor algorithm for feature extraction in face recognition," Int. J. Comput., vol. 3, no. 1, pp. 20-30, 2009.
22. S. R. V. Kittusamy and V. Chakrapani, "Facial expression recognition using eigenspaces," J. Comput. Sci., vol. 8, no. 10, pp. 1674-1679, 2012.
23. J. Kalita and K. Das, "Recognition of facial expression using eigenvector based distributed features and Euclidean distance based decision making technique," Int. J. Adv. Comput. Sci. Appl., vol. 4, no. 2, pp. 196-202, 2013.
24. Z. Abidin and A. Harjoko, "A neural network based facial expression recognition using Fisherface," Int. J. Comput. Appl., vol. 59, no. 3, pp. 30-34, 2012.
25. V. J. Mistry and M. M. Goyani, "A literature survey on facial expression recognition using global features," Int. J. Eng. Adv. Technol., vol. 2, no. 4, pp. 653-657, 2013.
26. F. Long, T. Wu, J. R. Movellan, M. S. Bartlett, and G. Littlewort, "Learning spatiotemporal features by using independent component analysis with application to facial expression recognition,"Neurocomputing, vol. 93, no. 1, pp. 126-132, 2012.
27. nd Int. Conf. Digit. Inf. Commun. Technol. Appl. (DICTAP), May 2012, pp. 99-103.
28. M. Kaur, R. Vashisht, and N. Neeru, "Recognition of facial expressions with principal component analysis and singular value decomposition,"Int. J. Comput. Appl., vol. 9, no. 12, pp. 36-40, 2010.
29. A. Halder, A. Jati, G. Singh, A. Konar, A. Chakraborty, and R. Janarthanan, "Facial action point based emotion recognition by principal component analysis," in Proc. Int. Conf. Soft Comput. Problem Solving (SocProS), Dec. 2011, pp. 721-733.
30. M. Z. Uddin, T.-S. Kim, and B. C. Song, "An optical flow feature-based robust facial expression recognition with HMM from video," Int. J. Innovative Comput., Inf., Control, vol. 9, no. 4, pp. 1409-1421, 2013.
31. L. Ma, "Facial expression recognition using 2D DCT of binarized edge images and constructive feedforward neural networks," in Proc. Int. Joint Conf. Neural Netw. (IJCNN) (IEEE World Congr. Comput. Intell.), Jun. 2008, pp. 4083-4088.
32. M. Kumbhar, A. Jadhav, and M. Patil, "Facial expression recognition based on image feature," Int. J. Comput. Commun. Eng., vol. 1, no. 2, pp. 117-119, 2012.
33. S. Joseph, "Ecological and biochemical studies on cyanobacteria of Cochin estuary and their application as source of antioxidants,"Ph.D. dissertation, Dept. Marine Biol., Microbiol., Biochem., Cochin Univ. Sci. Technol., Kochi, India, 2005.
34. S. Chitra and G. Balakrishnan, "A survey of face recognition on feature extraction process of dimensionality reduction techniques,"J. Theoretical Appl. Inf. Technol., vol. 36, no. 1, pp. 92-100, 2012.
35. S. C. Christabel, M. Annalakshmi, and D. P. Winston, "Facial feature extraction based on local color and texture for face recognition using neural network," Int. J. Sci. Eng. Appl., vol. 2, no. 4, pp. 78-82, 2013.
36. L. Zhang and D. Tjondronegoro, "Feature extraction and representation for face recognition," in Face Recognition, M. Oravec, Ed. Rijeka, Croatia: InTech, 2010, pp. 1-20.
37. S. Z. Li, X. Hou, H. Zhang, and Q. Cheng, "Learning spatially localized, parts-based representation," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), vol. 1. Dec. 2001, pp. I-207-I-212.
38. W. Gu, C. Xiang, Y. V. Venkatesh, D. Huang, and H. Lin, "Facial expression recognition using radial encoding of local Gabor features and classifier synthesis," Pattern Recognit., vol. 45, no. 1, pp. 80-91, 2012.
39. th Int. Conf. Pattern Recognit. (ICPR), vol. 1. Aug. 2004, pp. 288-291.
40. S. Zhang, X. Zhao, and B. Lei, "Facial expression recognition based on local binary patterns and local Fisher discriminant analysis," WSEAS Trans. Signal Process., vol. 8, no. 1, pp. 21-31, 2012.
41. T. Ahsan, T. Jabid, and U.-P. Chong, "Facial expression recognition using local transitional pattern on Gabor filtered facial images," IETE Tech. Rev., vol. 30, no. 1, pp. 47-52, 2013.
42. T. Jabid, M. H. Kabir, and O. Chae, "Robust facial expression recognition based on local directional pattern," ETRI J., vol. 32, no. 5, pp. 784-794, 2010.
43. th IEEE Int. Conf. Adv. Video Signal Based Surveill. (AVSS), Aug./Sep. 2010, pp. 526-532.
44. A. R. Rivera, J. R. Castillo, and O. Chae, "Local directional number pattern for face analysis: Face and expression recognition," IEEE Trans. Image Process., vol. 22, no. 5, pp. 1740-1752, May 2013.
45. Y. Rahulamathavan, R. C.-W. Phan, J. A. Chambers, and D. J. Parish, "Facial expression recognition in the encrypted domain based on local Fisher discriminant analysis," IEEE Trans. Affective Comput., vol. 4, no. 1, pp. 83-92, Jan./Mar. 2013.
46. Z.Wang and X. Sun, "Manifold adaptive kernel local Fisher discriminant analysis for face recognition," J. Multimedia, vol. 7, no. 6, pp. 387-393, 2012.
47. C. S. Patil and A. J. Patil, "A review paper on facial detection technique using pixel and color segmentation," Int. J. Comput. Appl., vol. 62, no. 1, pp. 21-24, 2013.
48. M. Gargesha and P. Kuchi, "Facial expression recognition using artificial neural networks," EEE 511-Artif. Neural Comput. Syst., pp. 1-6, Spring 2002.
49. J. V. Tu, "Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes,"J. Clin. Epidemiol., vol. 49, no. 11, pp. 1225-1231, 1996.
50. H.-H. Tsai, Y.-S. Lai, and Y.-C. Zhang, "Using SVM to design facial expression recognition for shape and texture features," in Proc. Int. Conf. Mach. Learn. Cybern. (ICMLC), vol. 5. Jul. 2010, pp. 2697-2704.
51. (2011). Support Vector Machines (SVMs). [Online]. Available: http://scikit-learn.org/stable/modules/svm.html, accessed Dec. 15, 2013.
52. W. Liu, J.-L. Lu, Z.-F. Wang, and H.-J. Song, "An expression space model for facial expression analysis," in Proc. Congr. Image Signal Process. (CISP), vol. 2. May 2008, pp. 680-684.
53. th Int. Conf. Pattern Recognit. (ICPR), Aug. 2010, pp. 4251-4254.
54. T. Bouwmans and F. El Baf, "Modeling of dynamic backgrounds by type-2 fuzzy Gaussians mixture models," MASAUM J. Basic Appl. Sci., vol. 1, no. 2, pp. 265-276, 2009.
55. M. Z. Uddin, J. J. Lee, and T.-S. Kim, "An enhanced independent component-based human facial expression recognition from video,"IEEE Trans. Consum. Electron., vol. 55, no. 4, pp. 2216-2224, Nov. 2009.
56. th Int. Conf. Mach. Learn., 2001, pp. 282-289.
57. S. Kumar and M. Hebert, "Discriminative fields for modeling spatial dependencies in natural images," in Advances in Neural Information Processing Systems 16. Cambridge, MA, USA: MIT Press, 2003.
58. A. Quattoni, S. Wang, L. Morency, M. Collins, and T. Darrell, "Hidden conditional random fields," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1848-1852, Oct. 2007.
59. A. Gunawardana, M. Mahajan, A. Acero, and J. C. Platt, "Hidden conditional random fields for phone classification," in Proc. Int. Conf. INTERSPEECH, 2005, vol. 2, no. 1, pp. 1117-1120.
60. M. Mahajan, A. Gunawardana, and A. Acero, "Training algorithms for hidden conditional random fields," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), vol. 1. May 2006, pp. I-273-I-276.
61. S. Reiter, B. Schuller, and G. Rigoll, "Hidden conditional random fields for meeting segmentation," in Proc. IEEE Int. Conf. Multimedia Expo, Jul. 2007, pp. 639-642.
62. S. Mika, G. Ratsch, J. Weston, B. Scholkopf, and K. Muller, "Fisher discriminant analysis with kernels," in Proc. IEEE Signal Process. Soc. Workshop Neural Netw. Signal Process. IX, Aug. 1999, pp. 41-48.
63. G. Baudat and F. Anouar, "Generalized discriminant analysis using a kernel approach," Neural Comput., vol. 12, no. 10, pp. 2385-2404, 2000.
64. S. Mika, "Kernel Fisher discriminants," Ph.D. dissertation, Fakultät Elektrotech. Informatik, Universitätsbibliothek, Bern, Switzerland, 2002.
65. P. N. Belhumeur, J. P. Hespanha, and D. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection,"IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711-720, Jul. 1997.
66. P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, "The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2010, pp. 94-101.
67. rd IEEE Int. Conf. Autom. Face Gesture Recognit., Apr. 1998, pp. 200-205.
68. K.-C. Lee, J. Ho, and D. Kriegman, "Acquiring linear subspaces for face recognition under variable lighting," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 684-698, May 2005.
69. M. Pantic, M. Valstar, R. Rademaker, and L. Maat, "Web-based database for facial expression analysis," in Proc. IEEE Int. Conf. Multimedia Expo (ICME), Jul. 2005, p. 5.
70. D. Ghimire and J. Lee, "Geometric feature-based facial expression recognition in image sequences using multi-class AdaBoost and support vector machines," Sensors, vol. 13, no. 6, pp. 7714-7734, 2013.
71. S. Du and R. K. Ward, "Adaptive region-based image enhancement method for robust face recognition under variable illumination conditions,"IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 9, pp. 1165-1175, Sep. 2010.
72. P. Forczmański, G. Kukharev, and N. Shchegoleva, "Simple and robust facial portraits recognition under variable lighting conditions based on two-dimensional orthogonal transformations," in Image Analysis and Processing-ICIAP. Berlin, Germany: Springer-Verlag, 2013, pp. 602-611.
73. L. Zhong, Q. Liu, P. Yang, B. Liu, J. Huang, and D. N. Metaxas, "Learning active facial patches for expression analysis," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 2562-2569.
74. X. Zhao and S. Zhang, "Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding," EURASIP J. Adv. Signal Process., vol. 2012, no. 1, pp. 1-9, 2012.
75. Y. Rahulamathavan, R. C.-W. Phan, S. Veluru, K. Cumanan, and M. Rajarajan, "Privacy-preserving multi-class support vector machine for outsourcing the data classification in cloud," IEEE Trans. Dependable Secure Comput., vol. 11, no. 5, pp. 467-479, Sep./Oct. 2014.