On assisting a visual-facial affect recognition system with keyboard-stroke pattern information

Knowledge-Based Systems

Volumn 23, Issue 4, 2010, Pages 350-356

  Stathopoulou, I O ^a   Alepis, E ^a   Tsihrintzis, G A ^a   Virvou, M ^a  

^a UNIVERSITY OF PIRAEUS   (Greece)

Author keywords

Emotion perception; Emotion recognition; Facial expression; Human computer interaction; Multimodal affective systems; Pattern recognition

Indexed keywords

AFFECT RECOGNITION; EMOTION RECOGNITION; EMOTIONAL STATE; EVALUATION PROCESS; FACIAL EXPRESSION RECOGNITION; FACIAL EXPRESSIONS; MULTI-MODAL; PATTERN INFORMATION; USER MODELS;

FACE RECOGNITION; GESTURE RECOGNITION; HUMAN COMPUTER INTERACTION; KNOWLEDGE MANAGEMENT;

PATTERN RECOGNITION SYSTEMS;

EID: 77950594466     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2009.11.007     Document Type: Article

References (39)

1. E. Alepis, M. Virvou, K. Kabassi, Affective student modeling based on microphone and keyboard user actions, in: ICALT '06: Proceedings of the Sixth IEEE International Conference on Advanced Learning Technologies, IEEE Computer Society, Washington, DC, USA, 2006, pp. 139-141.
2. Berthouze B.N., and Kleinsmith A. A categorical approach to affective gesture recognition. Connection Science 15 4 (2003) 259-269.
3. C. Busso, Z. Deng, S. Yildirim, M. Bulut, C.M. Lee, A. Kazemzadeh, S. Lee, U. Neumann, S. Narayanan, Analysis of emotion recognition using facial expressions, speech and multimodal information, in: ICMI '04: Proceedings of the Sixth International Conference on Multimodal Interfaces, ACM, New York, NY, USA, 2004, pp. 205-211. .
4. Camurri A., Lagerlöf I., and Volpe G. Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. International Journal of Human-Computer Studies 59 1-2 (2003) 213-225.
5. L.S. Chen, T.S. Huang, T. Miyasato, R. Nakatsu, Multimodal human emotion/expression recognition, in: Proceedings of International Conference on Automatic Face and Gesture Recognition, 1998, pp. 366-371.
6. R. Cowie, E. Douglas-cowie, Automatic statistical analysis of the signal and prosodic signs of emotion in speech, 1989.
7. Damasio A.R. Emotion in the perspective of an integrated nervous system. Brain Research Reviews 26 (1998) 83-86
8. Damasio A.R. Fundamental feelings. Nature 413 (2001) 781
9. Davidson R., Pizzagalli D., Nitschke J., and Kalin N. Parsing the Subcomponents of Emotion and Disorders of Emotion: Perspectives from Affective Neuroscience, Handbook of Affective Sciences (2003), Oxford University Press, USA
10. Davidson R., Scherer K., and Goldsmith H. Handbook of Affective Sciences (2003), Oxford, USA
11. L. De Silva, T. Miyasato, R. Nakatsu, Facial emotion recognition using multimodal information, in: Proceedings of IEEE International Conference on Information, Communications and Signal Processing - ICICS. Singapore, Thailand, 1997.
12. D. Goleman, Emotional Intelligence, Bantam Books, New York, USA.
13. H. Graf, E. Cosatto, V. Strom, F. Huang, Visual prosody: facial movements accompanying speech, in: 5th IEEE International Conference on Automatic Face and Gesture Recognition, 2002, pp. 381-386.
14. H. Gunes, M. Piccardi, A bimodal face and body gesture database for automatic analysis of human nonverbal affective behavior, in: ICPR'06: Proceedings of the 18th International Conference on Pattern Recognition, IEEE Computer Society, Washington, DC, USA, 2006, pp. 1148-1153. .
15. Isbister K., and Hook K. Evaluating affective interactions (introduction to special issue). International Journal of Human-Computer Studies 65 4 (2007) 273-274
16. R. Kaliouby, P. Robinson, Generalization of a vision-based computational model of mind-reading (2005) 582-589. .
17. Liao W., Zhang W., Zhu Z., Ji Q., and Gray W.D. Toward a decision-theoretic framework for affect recognition and user assistance. International Journal of Human-Computer Studies 64 9 (2006) 847-873.
18. S. Oviatt, User-centered modeling and evaluation of multimodal interfaces, in: IEEE Proceedings, vol. 91(B), 2003, pp. 1457-1468.
19. Pantic M., and Rothkrantz L.J.M. Automatic analysis of facial expressions: the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (2000) 1424-1445
20. M. Pantic, L.J.M. Rothkrantz, Toward an affect-sensitive multimodal human-computer interaction, in: Proceedings of the IEEE, 2003, pp. 1370-1390.
21. Pantic M., and Rothkrantz L.J.M. Toward an affect-sensitive multimodal human-computer interaction. Proceedings of the IEEE 91 9 (2003) 1370-1390 10.1109/JPROC.2003.817122
22. Picard R. Affective computing: challenges. International Journal of Human-Computer Studies 59 1-2 (2003) 55-64 10.1016/S1071-5819(03)00052-1
23. Picard R.W., Vyzas E., and Healey J. Toward machine emotional intelligence: analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (2001) 1175-1191
24. Pierrakos D., Papatheodorou G.P.C., and Spyropoulos C. Web usage mining as a tool for personalization: a survey. User Modeling and User Adapted Interaction 13 4 (2003) 311-372
25. K.R. Scherer, Adding the affective dimension: a new look in speech analysis and synthesis (1996) 1808-1811.
26. I.O. Stathopoulou, G. Tsihrintzis, A neural network-based facial analysis system, in: Proceedings of the 5th International Workshop on Image Analysis for Multimedia Interactive Services. Lisboa, Portugal, 2004.
27. I.O. Stathopoulou, G. Tsihrintzis, An improved neural network-based face detection and facial expression classification system, in: IEEE International Conference on Systems, Man, and Cybernetics. The Hague, Netherlands, 2004.
28. I.O. Stathopoulou, G. Tsihrintzis, Detection and expression classification systems for face images (FADECS), in: Proceedings of the IEEE Workshop on Signal Processing Systems (SiPS'05). Athens, Greece, 2005.
29. I.O. Stathopoulou, G. Tsihrintzis, Evaluation of the discrimination power of features extracted from 2-D and 3-D facial images for facial expression analysis, in: Proceedings of the 13th European Signal Processing Conference, Antalya, Turkey, 2005.
30. I.O. Stathopoulou, G. Tsihrintzis, Pre-processing and expression classification in low quality face images, in: Proceedings of 5th EURASIP Conference on Speech and Image Processing, Multimedia Communications and Services, 2005.
31. I.O. Stathopoulou, G. Tsihrintzis, An accurate method for eye detection and feature extraction in face color images, in: Proceedings of the 13th International Conference on Signals, Systems, and Image Processing. Budapest, Hungary, 2006.
32. Stathopoulou I.O., and Tsihrintzis G. Facial expression classification: specifying requirements for an automated system. Proceedings of the 10th International Conference on Knowledge-Based and Intelligent Information and Engineering Systems. LNAI vol. 4252 (2006), Springer-Verlag, Heidelberg, Berlin 1128-1135. http://dx.doi.org/10.1007/11893004
33. I.O. Stathopoulou, G.A. Tsihrintzis, Neu-faces: a neural network-based face image analysis system, in: ICANNGA '07: Proceedings of the 8th International Conference on Adaptive and Natural Computing Algorithms, Part II, LNCS, vol. 4432, Springer-Verlag, Berlin, Heidelberg, 2007, pp. 449-456. .
34. I.O. Stathopoulou, G.A. Tsihrintzis, Comparative performance evaluation of artificial neural network-based vs. human facial expression classifiers for facial expression recognition, in: KES-IMSS 2008: 1st International Symposium on Intelligent Interactive Multimedia Systems and Services, SCI, vol. 142, Springer-Verlag, Berlin, Heidelberg, 2008, pp. 55-65. http://dx.doi.org/10.1007/978-3-540-68127-4.
35. M. Virvou, G.A. Tsihrintzis, E. Alepis, I.O. Stathopoulou, K. Kabassi, Combining empirical studies of audio-lingual and visual-facial modalities for emotion recognition, in: KES'07: Knowledge-Based Intelligent Information and Engineering Systems and the XVII Italian Workshop on Neural Networks on Proceedings of the 11th International Conference, LNAI, vol. 4693, Springer-Verlag, Berlin, Heidelberg, 2007, pp. 1130-1137. .
36. E. Alepis, M. Virvou, Emotional intelligence: constructing user stereotypes for affective bi- modal interaction, in: Lecture Notes in Computer Science: Knowledge-based Intelligent Information and Engineering System, LNAI - I, vol. 4251, Springer-Verlag, Berlin, Heidelberg, 2006, pp. 435-442.
37. Alepis E., Virvou M., and Kabassi K. Requirements analysis and design of an affective bi-modal intelligent tutoring system: the case of keyboard and microphone. Studies in Computational Intelligence, Intelligent Interactive Systems in Knowledge-Based Environments 104 (2008) 9-24
38. Fishburn P.C. Additive utilities with incomplete product set: applications to priorities and assignments. Operations Research (1967)
39. Hwang C.L., and Yoon K. Multiple attribute decision making: methods and applications. Lecture Notes in Economics and Mathematical Systems vol. 186 (1981), Springer, Berlin/Heidelberg/New York