Action-perception cycle learning for incremental emotion recognition in a movie clip using 3D fuzzy GIST based on visual and EEG signals

Integrated Computer-Aided Engineering

Volumn 21, Issue 3, 2014, Pages 295-310

  Lee, Giyoung ^a   Kwon, Mingu ^a,b   Kavuri, Swathi ^a   Lee, Minho ^a  

^a Kyungpook National University   (South Korea)

^b LG Electronics Inc   (South Korea)

Author keywords

3D fuzzy GIST; action perception (AP) cycle; Autonomous emotion understanding; electroencephalography (EEG); Incremental adaptive neuro fuzzy inference system (ANFIS); Independent Component Analysis (ICA)

Indexed keywords

ELECTROENCEPHALOGRAPHY; ELECTROPHYSIOLOGY; FUZZY SYSTEMS; INDEPENDENT COMPONENT ANALYSIS; MOTION PICTURES; TRACKING (POSITION);

3D FUZZY GISTS; ACTION-PERCEPTION (AP) CYCLE; ADAPTIVE NEURO-FUZZY INFERENCE SYSTEM; EMOTION UNDERSTANDING; INDEPENDENT COMPONENT ANALYSIS(ICA);

BEHAVIORAL RESEARCH;

EID: 84902195820     PISSN: 10692509     EISSN: 18758835     Source Type: Journal    
DOI: 10.3233/ICA-140464     Document Type: Article

References (43)

1. U.R. Acharya, S.V. Sree, S. Chattopadhyay, W. Yu and P.C.A. Ang, Application of recurrence quantification analysis for the automated identification of epileptic EEG signals, International Journal of Neural Systems 21 (2011), 199-211.
2. M. Ahmadlou, H. Adeli and A. Adeli, Fractality analysis of frontal brain in major depressive disorder, International Journal of Psychophysiology 85 (2012), 206-211.
3. M. Ahmadlou, H. Adeli and A. Adeli, Fuzzy synchronization likelihood-wavelet methodology for diagnosis of autism spectrum disorder, Journal of Neuroscience Methods 211 (2012), 203-209.
4. M. Ahmadlou, H. Adeli and A. Adeli, Spatiotemporal analysis of relative convergence of EEGs reveals differences between brain dynamics of depressive women and men, Clinical EEG and Neuroscience 44 (2013), 175-181.
5. J. Assfalg,M. Bertini, C. Colombo and A. Del Bimbo, Semantic annotation of sports videos, IEEE Multimedia 9 (2002), 52-60.
6. J.C. Bezdek, Fuzzy mathematics in pattern classification, (1973).
7. N.-Y. Bian, B. Wang, Y. Cao and L. Zhang, Automatic removal of artifacts from EEG data using ICA and exponential analysis, Advances in Neural Networks-ISNN 2006, Springer, 2006, pp. 719-726.
8. C. Colombo, A. Del Bimbo and P. Pala, Semantics in visual information retrieval, IEEE Multimedia 6 (1999), 38-53.
9. P.J. Corr, Reinforcement sensitivity theory and personality, Neuroscience & Biobehavioral Reviews 28 (2004), 317-332.
10. A. Damasio, Neural basis of emotions, Scholarpedia 6 (2011), 1804.
11. R.J. Davidson, Anterior cerebral asymmetry and the nature of emotion, Brain and cognition 20 (1992), 125-151.
12. H.G. Feichtinger and T. Strohmer, Gabor Analysis and Algorithms: Theory and Applications, Springer, 1998.
13. E. Gilet, J. Diard and P. Bessière, Bayesian action-perception computational model: Interaction of production and recognition of cursive letters, PloS One 6 (2011), e20387.
14. H. Gunes, B. Schuller, M. Pantic and R. Cowie, Emotion representation, analysis and synthesis in continuous space: A survey, Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, IEEE, 2011, pp. 827-834.
15. G.R. Harik, F.G. Lobo and D.E. Goldberg, The compact genetic algorithm, Evolutionary Computation, IEEE Transactions on 3 (1999), 287-297.
16. U. Hasson, R. Malach and D.J. Heeger, Reliability of cortical activity during natural stimulation, Trends in Cognitive Sciences 14 (2010), 40-48.
17. J. Itten and E. Van Haagen, The Art of Color: The Subjective Experience and Objective Rationale of Color, Van Nostrand Reinhold, New York, NY, USA, 1973.
18. Y. Iwadate, M. Inoue, R. Suzuki, N. Hikawa, M. Makino and Y. Kanemoto, MIC Interactive Dance System-an emotional interaction system, Knowledge-Based Intelligent Engineering Systems and Allied Technologies, 2000. Proceedings. Fourth International Conference on, IEEE, 2000, pp. 95-98.
19. J.-S.R. Jang and C.-T. Sun, Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Prentice-Hall, Inc., 1996.
20. X. Jie, R. Cao and L. Li, Emotion recognition based on the sample entropy of EEG, Bio-Medical Materials and Engineering 23 (2013), S1237-S1244.
21. T.P. Jung, S. Makeig, C. Humphries, T.W. Lee, M.J. Mckeown, V. Iragui and T.J. Sejnowski, Removing electroencephalographic artifacts by blind source separation, Psychophysiology 37 (2000), 163-178.
22. M. Kwon, J.-S. Kang and M. Lee, Emotion classification in movie clips based on 3d fuzzy gist and eeg signal analysis, Brain-Computer Interface (BCI), 2013 International Winter Workshop on, IEEE, 2013, pp. 67-68.
23. M. Kwon and M. Lee, 3D fuzzy GIST to analyze emotional features in movies, Intelligent Data Engineering and Automated Learning-IDEAL 2012, Springer, 2012, pp. 192-199.
24. J. Lee andW.P. Dos Santos, An adaptive fuzzy-based system to simulate, quantify and compensate color blindness, Integrated Computer-Aided Engineering 18 (2011), 29-40.
25. M.M. Müler, A. Keil, T. Gruber and T. Elbert, Processing of affective pictures modulates right-hemispheric gamma band EEG activity, Clinical Neurophysiology 110 (1999), 1913-1920.
26. M. Mottaghi-Kashtiban and M.G. Shayesteh, A new window function for signal spectrum analysis and FIR filter design, Electrical Engineering (ICEE), 2010 18th Iranian Conference on, IEEE, 2010, pp. 215-219.
27. A. Oliva and A. Torralba, Modeling the shape of the scene: A holistic representation of the spatial envelope, International Journal of Computer Vision 42 (2001), 145-175.
28. L.C. Ou, M.R. Luo, A. Woodcock and A. Wright, A study of colour emotion and colour preference. Part I: Colour emotions for single colours, Color Research & Application 29 (2004), 232-240.
29. M.V. Peelen and P.E. Downing, The neural basis of visual body perception, Nature Reviews Neuroscience 8 (2007), 636-648.
30. P. Philippot, C. Baeyens, C. Douilliez and B. Francart, Cognitive regulation of emotion: Application to clinical disorders, The Regulation of Emotion (2004), 71-97.
31. J. Piater, S. Jodogne, R. Detry, D. Kraft, N. Krüger, O. Kroemer and J. Peters, Learning visual representations for perception-action systems, The International Journal of Robotics Research 30 (2011), 294-307.
32. R.W. Picard, Affective computing, MIT press, 2000.
33. G. Pourtois, A. Schettino and P. Vuilleumier, Brain mechanisms for emotional influences on perception and attention: What is magic and what is not, Biological psychology 92 (2013), 492-512.
34. U. Reuter, A fuzzy approach for modelling non-stochastic heterogeneous data in engineering based on cluster analysis, Integrated Computer-Aided Engineering 18 (2011), 281-289.
35. V. Schwämmle and O.N. Jensen, A simple and fast method to determine the parameters for fuzzy c-means cluster analysis, Bioinformatics 26 (2010), 2841-2848.
36. D. Sherman, N. Zhang, S. Garg, N.V. Thakor, M.A. Mirski, M.A. White and M.J. Hinich, Detection of nonlinear interactions of EEG alpha waves in the brain by a new coherence measure and its application to epilepsy and anti-epileptic drug therapy, International Journal of Neural Systems 21 (2011), 115-126.
37. M. Soleymani, M. Pantic and T. Pun, Multimodal emotion recognition in response to videos, Affective Computing, IEEE Transactions on 3 (2012), 211-223.
38. W. Wang, Y. Yu and J. Zhang, A new SVM based emotional classification of image, Journal of Electronics (China) 22 (2005), 98-104.
39. W. Wei-Ning, Y. Ying-lin and J. Sheng-ming, Image retrieval by emotional semantics: A study of emotional space and feature extraction, Systems, Man and Cybernetics, 2006. SMC'06. IEEE International Conference on, IEEE, 2006, pp. 3534-3539.
40. C. Yu and L. Xu, An emotion-based approach to decision making and self learning in autonomous robot control, Intelligent Control and Automation, 2004. WCICA 2004. Fifth World Congress on, IEEE, 2004, pp. 2386-2390.
41. C. Zhang, H. Wang and M.-H. Wu, EEG-based expert system using complexity measures and probability density function control in alpha sub-band, Integrated Computer-Aided Engineering 20 (2013), 391-405.
42. Q. Zhang, S. Jeong and M. Lee, Autonomous emotion development using incremental modified adaptive neuro-fuzzy inference system, Neurocomputing 86 (2012), 33-44.
43. Q. Zhang and M. Lee, Emotion development system by interacting with human EEG and natural scene understanding, Cognitive Systems Research 14 (2012), 37-49.