Emotion recognition from facial expressions and its control using fuzzy logic

IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

Volumn 39, Issue 4, 2009, Pages 726-743

  Chakraborty, Aruna ^a,b   Konar, Amit ^b   Chakraborty, Uday Kumar ^c   Chatterjee, Amita ^b  

^a ST THOMAS COLLEGE OF ENGINEERING AND TECHNOLOGY   (India)

^b JADAVPUR UNIVERSITY   (India)

^c UNIVERSITY OF MISSOURI ST LOUIS   (United States)

Author keywords

Emotion control; Emotion modeling; Emotion recognition; Fuzzy logic

Indexed keywords

EMOTION CONTROL; EMOTION MODELING; EMOTION RECOGNITION; EXTERNAL STIMULUS; FACIAL EXPRESSIONS; FACIAL FEATURE; HUMAN EMOTION RECOGNITION; HUMAN SUBJECTS; MAMDANI; REGIONS OF INTEREST; RELATIONAL MODEL;

FACE RECOGNITION; FUZZY CONTROL; FUZZY SETS;

FUZZY LOGIC;

EID: 67650675840     PISSN: 10834427     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCA.2009.2014645     Document Type: Article

References (46)

1. J. C. Bezdek, "Fuzzy mathematics in pattern classification," Ph.D. dissertation, Appl. Math. Center, Cornell Univ., Ithaca, NY, 1973.
2. B. Biswas, A. K. Mukherjee, and A. Konar, "Matching of digital images using fuzzy logic," AMSE Publication, vol. 35, no. 2, pp. 7-11, 1995.
3. M. T. Black and Y. Yacoob, "Recognizing facial expressions in image sequences using local parameterized models of image motion," Int. J. Comput. Vis., vol. 25, no. 1, pp. 23-48, Oct. 1997.
4. C. Busso and S. Narayanan, "Interaction between speech and facial gestures in emotional utterances: A single subject study," IEEE Trans. Audio, Speech Language Process., vol. 15, no. 8, pp. 2331-2347, Nov. 2007.
5. I. Cohen, "Facial expression recognition from video sequences," M.S. thesis, Univ. Illinois Urbana-Champaign, Dept. Elect. Eng., Urbana, IL, 2000.
6. I. Cohen, N. Sebe, A. Garg, L. S. Chen, and T. S. Huang, "Facial expression recognition from video sequences: Temporal and static modeling," Comput. Vis. Image Underst., vol. 91, no. 1/2, pp. 160-187, Jul. 2003.
7. C. Conati, "Probabilistic assessment of user's emotions in educational games," J. Appl. Artif. Intell., Special Issue Merging Cognition Affect HCT, vol. 16, no. 7/8, pp. 555-575, Aug. 2002.
8. G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, and T. J. Sejnowski, "Classifying facial actions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 10, pp. 974-989, Oct. 1999.
9. P. Ekman and W. V. Friesen, Unmasking the Face: A Guide to Recognizing Emotions From Facial Clues. Englewood Cliffs, NJ: Prentice-Hall, 1975.
10. I. A. Essa and A. P. Pentland, "Coding, analysis, interpretation and recognition of facial expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 757-763, Jul. 1997.
11. W. A. Fellenz, J. G. Taylor, R. Cowie, E. Douglas-Cowie, F. Piat, S. Kollias, C. Orovas, and B. Apolloni, "On emotion recognition of faces and of speech using neural networks, fuzzy logic and the ASSESS systems," in Proc. IEEE -INNS-ENNS Int. Joint Conf. Neural Netw., 2000, pp. 93-98.
12. J. M. Fernandez-Dols, H. Wallbotl, and F. Sanchez, "Emotion category accessibility and the decoding of emotion from facial expression and context," J. Nonverbal Behav., vol. 15, no. 2, pp. 107-123, Jun. 1991.
13. Y. Gao, M. K. H. Leung, S. C. Hui, and M.W. Tananda, "Facial expression recognition from line-based caricatures," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 33, no. 3, pp. 407-412, May 2003.
14. R. N. Gordon, The Structure of Emotions: Investigations in Cognitive Philosophy, ser. Cambridge Studies in Philosophy. Cambridge, U.K.: Cambridge Univ. Press, 1990.
15. K. Izumitani, T. Mikami, and K. Inoue, "A model of expression grade for face graphs using fuzzy integral," Syst. Control, vol. 28, no. 10, pp. 590-596, 1984.
16. F. Kawakami, S. Morishima, H. Yamada, and H. Harashima, "Construction of 3-D emotion space using neural network," in Proc. 3rd Int. Conf. Fuzzy Logic, Neural Nets Soft Comput., Iizuka, Japan, 1994, pp. 309-310.
17. G. J. Klir and B. Yuan, Fuzzy Sets and Fuzzy Logic: Theory and Applications. Englewood Cliffs, NJ: Prentice-Hall, 1995.
18. H. Kobayashi and F. Hara, "The recognition of basic facial expressions by neural network," Trans. Soc. Instrum. Contr. Eng., vol. 29, no. 1, pp. 112-118, 1993.
19. H. Kobayashi and F. Hara, "Measurement of the strength of six basic facial expressions by neural network," Trans. Jpn. Soc. Mech. Eng. (C), vol. 59, no. 567, pp. 177-183, 1993.
20. H. Kobayashi and F. Hara, "Recognition of mixed facial expressions by neural network," Trans. Jpn. Soc. Mech. Eng. (C), vol. 59, no. 567, pp. 184-189, 1993.
21. A. Konar, Computational Intelligence: Principles, Techniques and Applications. Heidelberg, Germany: Springer-Verlag, 2005.
22. A. F. Kramer, E. J. Sirevaag, and R. Braune, "A psycho-physiological assessment of operator workload during simulated flight missions," Hum. Factors, vol. 29, no. 2, pp. 145-160, Apr. 1987.
23. A. Lanitis, C. J. Taylor, and T. F. Cootes, "Automatic interpretation and coding of face images using flexible models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 743-756, Jul. 1997.
24. H. Li, P. Roivainen, and R. Forchheimer, "3D motion estimation in model-based facial image coding," IEEE Trans. Pattern Anal. Mach. Intell., vol. 15, no. 6, pp. 545-555, Jun. 1993.
25. X. Li and Q. Ji, "Active affective state detection and user assistance with dynamic Bayesian networks," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 35, no. 1, pp. 93-105, Jan. 2005.
26. K. Mase, "Recognition of facial expression from optical flow," Proc. IEICE Trans., Special Issue Comput. Vis. Appl., vol. 74, no. 10, pp. 3474-3483, 1991.
27. K. Ogata, Modern Control Engineering. Englewood Cliffs, NJ: Prentice-Hall, 1990.
28. W. Pedrycz and J. Valente de Oliveira, "A development of fuzzy encoding and decoding through fuzzy clustering," IEEE Trans. Instrum. Meas. vol. 57, no. 4, pp. 829-837, Apr. 2008.
29. W. Pedrycz and F. Gomide, An Introduction to Fuzzy Sets: Analysis and Design. Cambridge, MA: MIT Press, 1998.
30. R. Picard, Affective Computing. Cambridge, MA: MIT Press, 1997.
31. R. W. Picard, E. Vyzas, and J. Healey, "Toward machine emotional intelligence: Analysis of affective psychological states," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 10, pp. 1175-1191, Oct. 2001.
32. P. Rani, N. Sarkar, and J. Adams, "Anxiety-based affective communication for implicit human-machine interaction," Adv. Eng. Inf., vol. 21, no. 3, pp. 323-334, Jul. 2007.
33. P. Rani, N. Sarkar, C. Smith, and L. Kirby, "Anxiety detecting robotic systems-Towards implicit human-robot collaboration," Robotica, vol. 22, no. 1, pp. 83-93, 2004.
34. M. Rosenblum, Y. Yacoob, and L. Davis, "Human expression recognition from motion using a radial basis function network architecture," IEEE Trans. Neural Netw., vol. 7, no. 5, pp. 1121-1138, Sep. 1996.
35. J. Scheirer, R. Fernadez, J. Klein, and R. Picard, "Frustrating the user on purpose: A step toward building an affective computer," Interact. Comput., vol. 14, no. 2, pp. 93-118, Feb. 2002.
36. H. Simon, Motivational and Emotional Control of Cognition, Models of Thought. New Haven, CT: Yale Univ. Press, 1979, pp. 29-38.
37. D. Terzopoulos and K. Waters, "Analysis and synthesis of facial image sequences using physical and anatomical models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 15, no. 6, pp. 569-579, Jun. 1993.
38. Y. Tian, T. Kanade, and J. Cohn, "Recognizing action units for facial expression analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 2, pp. 97-115, Feb. 2001.
39. N. Ueki, S. Morishima, and H. Harashima, "Expression analysis/synthesis system based on emotion space constructed by multilayered neural network," Syst. Comput. Jpn., vol. 25, no. 13, pp. 95-103, 1994.
40. O. A. Uwechue and S. A. Pandya, Human Face Recognition Using Third-Order Synthetic Neural Networks. Boston, MA: Kluwer, 1997.
41. P. Vanger, R. Honlinger, and H. Haykin, "Applications of synergetic in decoding facial expressions of emotions," in Proc. Int. Workshop Autom. Face Gesture Recog., Zurich, Switzerland, 1995, pp. 24-29.
42. A. Vasilakos and W. Pedrycz, Ambient Intelligence, Wireless Net-working and Ubiquitous Computing. Norwood, MA: Artech House, Jun. 2006.
43. Y. Yacoob and L. Davis, "Computing spatio-temporal representations of human faces," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., Jun. 1994, pp. 70-75.
44. Y. Yacoob and L. Davis, "Recognizing human facial expression from long image sequences using optical flow," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 6, pp. 636-642, Jun. 1996.
45. H. Yamada, "Visual information for categorizing facial expression of emotion," Appl. Cogn. Psychol., vol. 7, no. 3, pp. 257-270, 1993.
46. Z. Zeng, Y. Fu, G. I. Roisman, Z. Wen, Y. Hu, and T. S. Huang, "Spontaneous emotional facial expression detection," J. Multimedia vol. 1, no. 5, pp. 1-8, Aug. 2006.