A hierarchical positive and negative emotion understanding system based on integrated analysis of visual and brain signals

Neurocomputing

Volumn 73, Issue 16-18, 2010, Pages 3264-3272

  Zhang, Qing ^a   Lee, Minho ^a  

^a Kyungpook National University   (South Korea)

Author keywords

EEG; Emotion understanding; Facial expression; GIST; IAPS; Neuro fuzzy inference model

Indexed keywords

EEG; EMOTION UNDERSTANDING; FACIAL EXPRESSIONS; GIST; IAPS; NEURO-FUZZY;

BRAIN; ELECTROENCEPHALOGRAPHY; ELECTROPHYSIOLOGY; FEATURE EXTRACTION;

FUZZY INFERENCE;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BRAIN DEPTH STIMULATION; CONTROL SYSTEM; CONTROLLED STUDY; ELECTROENCEPHALOGRAM; EMOTION; EMOTIONALITY; FACIAL EXPRESSION; FUZZY SYSTEM; HUMAN; MENTAL CAPACITY; NEUROIMAGING; NORMAL DISTRIBUTION; NUCLEAR MAGNETIC RESONANCE IMAGING; PATTERN RECOGNITION; PRIORITY JOURNAL; SIGNAL TRANSDUCTION; TASK PERFORMANCE; TIME PERCEPTION; VISUAL INFORMATION; VISUAL STIMULATION;

EID: 78649967728     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2010.04.001     Document Type: Article

References (33)

1. Colombo C., Bimbo A., Pala P. Semantics in visual information retrieval. IEEE Multimedia 1999, 6(3):38-53.
2. Y. Iwadate, et al., MIC Interactive Dance System-an emotional interaction system, in: Proceedings of the International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, vol. 1, September 2000, pp. 95-98.
3. Assfalg J., Bertini M., Colombo C., Bimbo A. Semantic annotation of sports videos. IEEE Multimedia 2002, 9(2):52-60.
4. C. Yu, L. Xu, An emotion-based approach to decision making and self learning in autonomous robot control, in: The Fifth World Congress on Intelligent Control and Automation, vol. 3, June 2004, pp. 2386-2390.
5. Güntekin B., Basar E. Emotional face expressions are differentiated with brain oscillations. International Journal of Psychophysiology 2007, 64:91-100.
6. Minsky M. The Emotion Machine 2006, Simon & Schuster, USA, New York.
7. E. Douglas-Cowie, WP5 members, HUMAINE, IST FP6 Contract no. 507422, 2004.
8. Picard R. Affective Computing 1997, MIT Press, London.
9. Zhang Q., Lee M. Analysis of positive and negative emotions in natural scene using brain activity and gist. Neurocomputing 2009, 72(4-6):1302-1306.
10. G. Ball, J. Breese, Modeling the emotional state of computer users, in: Workshop on Attitude, Personality and Emotions in User-Adapted Interaction, UM '99, 1999.
11. E. Hudlika, Increasing SIA architecture realism by modeling and adapting to affect and personality, in: Fall AAAI Symposium on Socially Intelligent Agents: the Human in the Loop, North Falmouth, MS, USA.
12. Dubois D., Prade H. Possibility theory, probability theory and multi-valued logics: a classification. Annals of Mathematics and Artificial Intelligence 2001, 32:35-66.
13. Bloom P. Descartes' Baby: How the Science of Child Development Explains What Makes us Human. 2005, Basic Books, New York, NY.
14. Jang J.S.R., Sun C.T., Mizutani E. Neuo-Fuzzy and Soft Computing. Matlab Curriculum Series 1997, Prentice Hall.
15. Lang P.J., Bradley M.M., Cuthbert B.N. International affective picture system (IAPS): affective ratings of pictures and instruction manual. Technical Report A-6 2005, University of Florida, Gainesville, FL.
16. Ekman P. Basic emotions. Handbook of Cognition and Emotion 1999, 45-60.
17. Plutchik R. Emotion: Theory, Research, and Experience, 1. Theories of Emotion, 1 1980, Academic, New York.
18. Lang P.J. The emotion probe: studies of motivation and attention. American Psychologist 1995, 50(5):372-385.
19. Choi J.-H., Sohn J.-I., Yang H.-S., Chung Y.-R., Lee M., Koh S.-C. Modeling of recycling oxic and anoxic treatment system for swine wastewater using neural networks. Biotechnology Bioprocess Engineering 2000, 5:355-361.
20. Batur Aziz Umit, Hayes Monson H. Adaptive active appearance models. IEEE Transaction on Image Processing 2005, 14:1707-1721.
21. Kobayashi H., Hara F. Facial Interaction between Animated 3D Face Robot and Human Beings. 1997, IEEE Computer Society Press, 3732-3737.
22. Krages B. Photography: The Art of Composition 2005, Allworth Press.
23. A. Oliva, Neurobiology of Attention. Preprint, ch41, 2006.
24. Marcelja S. Mathematical description of the responses of simple cortical cells. Journal of the Optical Society of America 1980, 70:1297-1300.
25. Oliva A., Torralba A. Building the gist of a scene: the role of global image features in recognition. Progress in Brain Research 2006, 155:23-36.
26. J. Bezdek, Fuzzy Mathematics in Pattern Classification, Ph.D. Thesis, Applied Math Center, Cornell University, Ithaca, 1973.
27. Dubois D., Prade H. Possibility theory is not fully compositional!-A comment on a short note by H.J. Greenberg. Fuzzy Sets and Systems 1988, 95:131-134.
28. Bersini H., Bontempi G. Now comes the time to defuzzify the neuro-fuzzy models. Fuzzy Sets and Systems 1997, 90(2):161-170.
29. Sugeno M., Kang G.T. Structure identification of fuzzy model. Fuzzy Sets and Systems 1988, 28:15-33.
30. T. Takagi, M. Sugeno, Derivation of fuzzy control rules from human operator's control actions, in: Proceedings of the IFAC Symposium on Fuzzy Information, Knowledge Representation and Decision Analysis, July 1983, pp. 55-60.
31. Takagi T., Sugeno M. Fuzzy identification of systems and its applications to modeling and control. IEEE Transaction System, Man, and Cybernatics 1985, 15:116-132.
32. G. Castellano, A Neurofuzzy methodology for predictive modeling, Ph.D. Thesis, Dep. Computer Science, Univ. of Bari, November 2000.
33. Q. Zhang, M. Lee, Emotion recognition in natural scene images based on brain activity and gist, in: IEEE International Joint Conference on Neural Networks, 2008, pp. 3050-3057.