Automatic detection of learner's affect from gross body language

Applied Artificial Intelligence

Volumn 23, Issue 2, 2009, Pages 123-150

  D'Mello, Sidney ^a   Graesser, Art ^a  

^a University of Memphis   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTER AIDED INSTRUCTION; EDUCATION COMPUTING;

AFFECTIVE STATE; AUTOMATIC DETECTION; AUTOTUTOR; BODY LANGUAGE; BODY POSITIONS; DATA SET; INTELLIGENT TUTORING; SETS OF FEATURES; TRAINING DATA; TUTORING SYSTEM;

LEARNING SYSTEMS;

EID: 60649093489     PISSN: 08839514     EISSN: 10876545     Source Type: Journal    
DOI: 10.1080/08839510802631745     Document Type: Article

References (56)

1. Aist, G., B. Kort, R. Reilly, R.W. Picard, and J. Mostow. 2002. Experimentally augmenting an intelligent tutoring system with human supplied capabilities: Adding human-provided emotional scaffolding to an automated reading tutor that listens. In: Proceedings of the 6th International Conference on Intelligent Tutoring Systems (TTS2002), Biarritz, France and San Sebastian, Spain, June 2-7, 2002.
2. Ang, J., R. Dhillon, A. Krupski, E. Shriberg, and A. Stolcke. 2002. Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In: Proceedings of the International Conference on Spoken Language Processing 2037-2039.
3. Barrett, L. F. 2006. Are emotions natural kinds? Perspectives on Psychological Science 1:28-58.
4. Bernstein, N. 1967. The Co-Ordination and Regulation of Movement. London: Pergamon Press.
5. Bianchi-Berthouze, N. and C. L. Lisetti. 2002. Modeling multimodal expression of users affective subjective experience. User Modeling and User-Adapted. Interaction 12(1):49-84.
6. Bianchi-Berthouze, N., P. Cairns, A. Cox, C. Jennett, and W. Kim. 2006. On posture as a modality for expressing and recognizing emotions. In: Emotion and HCI Workshop, British Computer Society HCI September 12, 2006, London.
7. Bull, E. P. 1987. Posture and Gesture. Pergamon Press: Oxford.
8. Campbell, D. T. and D. W. Fiske. 1959. Convergent And discriminant validation by the multitraitmultimethod matrix. Psychological Bulletin 56: 81-105.
9. Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20:37-46.
10. Conati, C. 2002. Probabilistic assessment of user's emotions in educational games. Journal of Applied Artificial Intelligence 16:555-575.
11. Coulson, M. 2004. Attributing emotion to static body postures: Recognition accuracy, confusions, and viewpoint dependence. Journal of Nonverbal Behavior 28:117-139.
12. Craig, S. D., A. C. Graesser, J. Sullins, and B. Gholson. 2004. Affect and learning: An exploratory look into the role of affect in learning. Journal of Educational Media 29:241-250.
13. de Meijer, M. 1989. The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior 13:247-268.
14. Dempster, A. P., N. M. Laird, and D. B. Rubin. 1977. Maximum Likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society 39:1-38.
15. de Rosis, F. 2002. Toward merging cognition and affect in HCI. Applied Artificial Intelligence 16:487-494.
16. De Vicente, A. and H. Pain. 2002. Informing the detection of students' motivational state: An empirical study. In: Proceedings of the 6th International Conference on Intelligent Tutoring Systems (TTS2002), Biarritz, France and San Sebastian, Spain, June 2-7, 2002.
17. D'Mello, S. K., S. D. Craig, B. Gholson, S. Franklin, R. Picard, and A. C. Graesser. 2005. Integrating affect sensors in an intelligent tutoring system. Affective interactions: The computer. In: The Affective Loop Workshop at the 2005 International Conference on Intelligent User Interfaces. New York: AMC Press.
18. D'Mello, S. K., S. D. Craig, J. Sullins, and A. C. Graesser. 2006. Predicting affective states through an emote-aloud procedure from autotutor's mixed-initiative dialogue. International Journal of Artificial Intelligence in Education 16:3-28.
19. D'Mello, S. K., R. Picard, and A. C. Graesser. 2007. Towards an affect sensitive autotutor. IEEE Intelligent Systems 22:53-61.
20. Dweck, C. S. 2002. Messages that motivate: How praise molds students' beliefs, motivation, and performance (in surprising ways). In: Improving Academic Achievement: Impact of Psychological Factors on Education, ed. J. Aronson, pp. 61-87. Orlando, FL: Academic Press.
21. Ekman, P. and W. V. Friesen. 1969. Nonverbal leakage and clues to deception. Psychiatry 32:88-105.
22. Ekman, P. and W. V. Friesen. 1978. The Facial Action Coding System: A Technique for the Measurement, of Facial Movement Palo Alto, CA: Consulting Psychologists Press.
23. Graesser, A C., S. Lu, G. T. Jackson, H. Mitchell, M. Ventura, A. Olney, and M. M. Louwerse. 2004. AutoTulor: A tutor with dialogue in natural language. Behavioral Research Methods, Instruments, and Computers 36:180-193.
24. Graesser, A. C., B. McDaniel, P. Chipman, A. Witherspoon, S. D'Mello, and B. Gholson. 2006. Detection of emotions during learning with autotutor. In: Proceedings of the 28th Annual Conference of the Cognitive Science Society. Mahwah, NJ: Erlbaum.
25. Graesser, A. C., N. Person, D. Harter, and Tutoring Research Group. 2001. Teaching tactics and dialogue in reply to: AutoTutor. International Journal of Artificial Intelligence in Education 12:257-279.
26. Grimm, M., E. Mower, K. Kroschel, and S. Narayan. 2006. Combining categorical and primitives-based emotion recognition. In: 14th European Signal Processing Conference (EUSIPCO), September 4-8, 2008, Florence, Italy.
27. Kim, Y. 2005. Empathetic virtual peers enhanced learner interest and self-efficacy. In: Workshop on. Motivation and Affect in Educational Software at the, 12th International Conference on. Artificial, Intelligence in Education, July 18-22, 2005, Amsterdam, the Netherlands.
28. Klein, J., Y. Moon, and R. Picard. 2002. This computer responds to user frustration - Theory, design, and results. Interacting with. Computers 14:19-140.
29. Kort, B., R. Reilly, and R. Picard. 2001. An affective model of interplay between emotions and learning: reengineering educational pedagogy building a learning companion. In: Proceedings IEEE International Conference on Advanced Learning Technology: Issues, Achievements and Challenges, Madison, WI, pp. 43-48.
30. Linnenbrink, E. A. and P. Pintrich. 2002. The role of motivational beliefs in conceptual change. In: Reconsidering Conceptual Change: Issues in Theory and Practice, eds. M. Limon and L. Mason. Dordrecht, the Netherlands: Kluwer Academic Publishers.
31. Liscombe, J., G. Riccardi, and D. Hakkani-Tür. 2005. Using context to improve emotion detection in Spoken dialog systems. In: EUROSPEECH'05, 9th European Conference on Speech Communication and Technology, Lisbon, Portugal.
32. Lisetti, C. L. and P. Gmytrasiewicz. 2002. Can a rational agent afford to be affectless? A formal approach. Applied Artificial Intelligence, An International journal 16:577-609.
33. Litman, D. J. and K. Forbes-Riley. 2004. Predicting student emotions in computer-human tutoring dialogues. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. East Stroudsburg, PA: Association for Computational Linguistics.
34. Mandler, G. 1984. Mind and Body: Psychology of Emotion and Stress. New York: Norton.
35. Matsubara, Y. and M. Nagamachi. 1996. Motivation systems and motivation models for intelligent tutoring. In: Proceedings of the Third, International Conference in Intelligent Tutoring Systems, June 12-14, 1996, Montreal, Canada.
36. Mehrabian, A. 1972. Nonverbal Communication. Chicago, IL: Aldine-Atherton.
37. Montepare, J., E. Koff, D. Zaitchik, and M. Albert. 1999. The use of body movements and gestures as cues to emotions in younger and older adults, Journal of Nonverbal Behavior 23:133-152.
38. Morimoto, C., D. Koons, A. Amir, and M. Flickner. 1998. Pupil detection and tracking using multiple light sources. Technical Report, IBM, Almaden Research Center, San Francisco, CA, USA
39. Mota, S. and R. W. Picard. 2003. Automated posture analysis for detecting learner's interest level. In: Workshop on Computer Vision and Pattern Recognition far Human-Computer Interaction (CVPR HCI), June 16-22, 2003, Madison, WI, USA.
40. Norman, D. A. 1994. How might people interact with agents? Communication of the ACM 37:68-71.
41. Paiva, A., R. Prada, and R. W. Picard. (eds.). 2007. Affective Computing and Intelligent. Interaction. Springer.
42. Pantic, M. and L.J. M. Rothkrantz. 2003. Towards an affect-sensitive multimodal human-computer interaction. In: Proceedings of the IEEE, Special Issue on Multimodal Human-Computer Interaction (HCI) 91(9):1370-1390.
43. Picard, R. W. 1997. Affective Computing. Boston, MA: MIT Press.
44. Picard, R. W., E. Vyzas, and J. Healey. 2001. Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions Pattern Analysis and Machine Intelligence 23:1175-1191.
45. Prendinger, H. and M. Ishizuka. 2005. The empathic companion: A character-based interface that addresses users' affective states. International journal of Applied Artificial Intelligence 19:267-285.
46. Rani, P., N. Sarkar, and C. A. Smith. 2003. An affect-sensitive human-robot cooperation - Theory and experiments. In: Proceedings of the IEEE Conference on Robotics and Automation. Taipei, Taiwan: IEEE, pp. 2382-2387.
47. Robson, C. 1993. Real Word Research: A Resource for Social Scientist and Practitioner Researchers. Oxford: Blackwell.
48. Selfridge, O. G. 1959. Pandemonium: A paradigm for learning. In: Symposium on the Mechanization of Thought Processes. London: Her Majesty's Stationary Office, pp. 511-531.
49. Shafran, I., M. Riley, and M. Mohri. 2003. Voice signatures. In: Proceedings IEEE Automatic Speech Recognition and Understanding Workshop, Piscataway, +: IEEE, pp. 31-36.
50. Shneiderman, B. and C. Plaisant. 2005. Designing the User Interface: Strategies for Effective Human-Computer Interaction. 4th ed. Reading, MA: Addison-Wesley.
51. Tan, H. Z., I. Lu, and A. Pendand. 1997. The chair as a novel haptic user interface. In: Proc. Workshop of Perceptual User Interface, pp. 56-57.
52. Tekscan. 1997. Tekscan Body Pressure Measurement System User's Manual. South Boston, MA Tekscan Inc.
53. VanLehn, K., A. C. Graesser, G. T. Jackson, P. Jordan, A. Olney, and C. P. Rose. 2007. When are tutorial dialogues more effective than reading? Cognitive Science 31:3-62.
54. Walk, R. D. and K. L. Walters. 1988. Perception of the smile and other emotions of the body and face at different distances. Bulletin of the Psychonomic. Society 26:510-510.
55. Whang, M. C., J. S. Lim, and W. Boucsein. 2003. Preparing computers for affective communication: A psychophysiological concept and preliminary results. Human Factors 45:623-634.
56. Witten, I. H. and E. Frank. 2005. Data Mining: Practical Machine Learning Tools and Techniques. 2nd ed. San Francisco, CA: Morgan Kaufmann.