Speaking effect removal on emotion recognition from facial expressions based on eigenface conversion

IEEE Transactions on Multimedia

Volumn 15, Issue 8, 2013, Pages 1732-1744

  Wu, Chung Hsien ^a   Wei, Wen Li ^a   Lin, Jen Chun ^a   Lee, Wei Yu ^a  

^a NATIONAL CHENG KUNG UNIVERSITY   (Taiwan)

Author keywords

Arousal valence emotion plane; Articulatory attribute; Conversion function; Emotion recognition; Facial expression

Indexed keywords

AROUSAL-VALENCE EMOTION PLANE; ARTICULATORY ATTRIBUTES; CONVERSION FUNCTION; EMOTION RECOGNITION; FACIAL EXPRESSIONS;

DECISION TREES;

FACE RECOGNITION;

EID: 84888365943     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2013.2272917     Document Type: Article

References (45)

1. I. Kotsia, I. Buciu, and I. Pitas, "An analysis of facial expression recognition under partial facial image occlusion," J. Image Vision Comput., vol. 26, no. 7, pp. 1052-1067, 2008.
2. P. Yang, Q. Liu, X. Cui, and D. N. Metaxas, "Facial expression recognition using encoded dynamic features," in Proc. IEEE Conf. Comput. Vision and Pattern Recognition, 2008.
3. M. Yeasin, B. Bullot, and R. Sharma, "Recognition of facial expressions and measurement of levels of interest from video," IEEE Trans. Multimedia, vol. 8, no. 3, pp. 500-508, 2006.
4. Y. Chang, C. Hu, R. Feris, and M. Turk, "Manifold based analysis of facial expression," Image Vision Comput., vol. 24, no. 6, pp. 605-614, 2006. (Pubitemid 43867834)
5. I. Cohen, N. Sebe, L. Chen, A. Garg, and T. S. Huang, "Facial expression recognition from video sequences: temporal and static modeling," Comput. Vision Image Understand., vol. 91, no. 1-2, pp. 160-187, 2003.
6. J. A. Russell, "A circumplex model of affect," J. Personal. Social Psychol., vol. 39, no. 6, pp. 1161-1178, 1980.
7. R. E. Thayer, The Biopsychology of Mood and Arousal. New York, NY, USA: Oxford Univ. Press, 1989.
8. H. Gunes and M. Pantic, "Automatic, dimensional and continuous emotion recognition," Int. J. Synthetic Emotions, vol. 1, no. 1, pp. 68-99, 2010.
9. M. A. Nicolaou, H. Gunes, and M. Pantic, "Amulti-layer hybrid framework for dimensional emotion classification," in Proc. ACM Multimedia, 2011, pp. 933-936.
10. L. Zhang, D. Tjondronegoro, and V. Chandran, "Evaluation of texture and geometry for dimensional facial expression recognition," in Proc. Int. Conf. Digital Image Computing: Techniques and Applicat., 2011, pp. 620-626.
11. D. Wu, T. D. Parsons, E. Mower, and S. Narayanan, "Speech emotion estimation in 3D space," in Proc. IEEE Int. Conf. Multimedia & Expo (ICME), 2010, pp. 737-742.
12. Y.-H. Yang, Y.-C. Lin, Y.-F. Su, and H.-H. Chen, "A regression approach to music emotion recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 2, pp. 448-457, 2008.
13. X. Zhao and S. Zhang, "Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding," EURASIP J. Adv. Signal Process., vol. 2012, no. 20, pp. 1-9, 2012.
14. S. Park and D. Kim, "Subtle facial expression recognition usingmotion magnification," Pattern Recognit. Lett., vol. 30, no. 7, pp. 708-716, 2009.
15. G. U. Kharat and S. V. Dudul, "Human emotion recognition system using optimally designed SVM with different facial feature extraction techniques," WSEAS Trans. Comput., vol. 7, no. 6, pp. 650-659, 2008.
16. R. W. Picard, Affective Computing. Cambridge, MA, USA: MIT Press, 1997.
17. Z. Zeng, J. Tu, M. Liu, T. S. Huang, B. Pianfetti, D. Roth, and S. Levinson, "Audio-visual affect recognition," IEEE Trans. Multimedia, vol. 9, no. 2, pp. 424-428, 2007. (Pubitemid 46183104)
18. Z. Zeng, J. Tu, M. Liu, T. Zhang, N. Rizzolo, Z. Zhang, T. Huang, D. Roth, and S. Levinson, "Bimodal HCI-related affect recognition," in Proc. 6th Int. Conf. Multimodal Interfaces, 2004, pp. 137-143. (Pubitemid 40371902)
19. D. Datcu and L. J. M. Rothkrantz, "Semantic audio-visual data fusion for automatic emotion recognition," Euromedia, 2008.
20. J.-C. Lin, C.-H. Wu, and W.-L. Wei, "Error weighted semi-coupled hidden Markov model for audio-visual emotion recognition," IEEE Trans. Multimedia, vol. 14, no. 1, pp. 142-156, 2012.
21. Y. Zhu, Y. Chen, and Y. Zhan, "Expression recognition method of image sequence in audio-video," in Proc. 3rd Int. Symp. Intelligent Inf. Technol. Applicat., 2009, pp. 513-516.
22. S. M. Siniscalchi, T. Svendsen, and C.-H. Lee, "Toward a detectorbased universal phone recognizer," in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing, 2008, pp. 4261-4264.
23. J. Li, Y. Tsao, and C.-H. Lee, "A study on knowledge source integration for rescoring in automatic speech recognition," in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing, 2005, pp. 837-840.
24. Handbook of the International Phonetic Association. Cambridge, U.K.: Cambridge Univ. Press, 1999.
25. C.-H. Wu, C.-H. Lee, and Z.-J. Chung, "Co-articulation generation using maximum direction change and apparent motion for Chinese visual speech synthesis," in Proc. Int. Comput. Symp., 2010.
26. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. Int. Conf. Comput. Vision Pattern Recognition, 2001, pp. 511-518.
27. T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active appearance models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp. 681-685, 2001. (Pubitemid 32585279)
28. T. F. Cootes, D. H. Cooper, C. J. Taylor, and J. Graham, "Trainable method of parametric shape description," Image Vision Comput., vol. 10, no. 5, pp. 289-294, 1992.
29. A. Lanitis, C. J. Taylor, and T. F. Cootes, "Automatic tracking, coding and reconstruction of human faces using flexible appearance models," IEEE Electron. Lett., vol. 30, no. 19, pp. 1587-1588, 1994.
30. F. Tang and B. Deng, "Facial expression recognition using AAM and local facial features," in Proc. Int. Conf. Natural Computation, 2007, pp. 632-635.
31. H. C. Choi and S. Y. Oh, "Realtime facial expression recognition using active appearance model and multilayer perceptron," in Proc. SICEICASE Int. Joint Conf., 2006, pp. 5924-5927.
32. I. T. Jolliffe, Principal Component Analysis. New York, NY, USA: Springer, 1986.
33. M. Turk and A. Pentland, "Eigenfaces for recognition," J. Cogn. Neurosci., vol. 3, no. 1, pp. 71-86, 1991.
34. J. Yang, D. Zhang, A. F. Frangi, and J.-Y. Yang, "Two-dimensional PCA: a new approach to appearance-based face representation and recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 1, pp. 131-137, 2004.
35. A. Lima, H. Zen, Y. Nankaku, C. Miyajima, K. Tokuda, and T. Kitamura, "On the use of kernel PCA for feature extraction in speech recognition," IEICE Trans. Inf. Syst., vol. 87-D, no. 12, pp. 2802-2811, 2004. (Pubitemid 40021352)
36. T. Takiguchi and Y. Ariki, "PCA-based speech enhancement for distorted speech recognition," J. Multimedia, vol. 2, no. 5, pp. 13-18, 2007.
37. S. Dubuisson, F. Davoine, and M. Masson, "A solution for facial expression representation and recognition," Signal Process.: Image Commun., vol. 17, no. 9, pp. 657-673, 2002. (Pubitemid 35214475)
38. Y. Stylianou, O. Cappé, and E. Moulines, "Continuous probabilistic transform for voice conversion," IEEE Trans. Speech Audio Process., vol. 6, no. 2, pp. 131-142, 1998. (Pubitemid 128720639)
39. J. Tao, Y. Kang, and A. Li, "Prosody conversion from neutral speech to emotional speech," IEEE Trans. Speech Audio Process., vol. 14, no. 4, pp. 1145-1154, 2006.
40. A. Kain and M. W. Macon, "Spectral voice conversion for text-tospeech synthesis," in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing, 1998, pp. 285-288.
41. C.-H. Lee, C.-H. Wu, and J.-C. Guo, "Pronunciation variation generation for spontaneous speech synthesis using state-based voice transformation," in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing, 2010, pp. 4826-4829.
42. K. P. Truong, D. A. v. Leeuwen, M.A. Neerincx, and F. M. G. d. Jong, "Arousal and valence prediction in spontaneous emotional speech: felt versus perceived emotion," in Proc. Interspeech, 2009, pp. 2027-2030.
43. A. Sen and M. Srivastava, Regression Analysis: Theory, Methods, and Applications. New York, NY, USA: Springer, 1990.
44. A. J. Smola and B. Schölkopf, "A tutorial on support vector regression," Statist. Comput., vol. 14, no. 3, pp. 199-222, 2004. (Pubitemid 39063488)
45. P. J. Lang, Behavioral Treatment and bio-Behavioral Assessment: Computer Applications: Technology in Mental Health Care Delivery Systems, J. B. Sidowski, J.H. Johnson, and T.A Williams, Eds. Norwood, MA, USA: Ablex, 1980, pp. 119-137.