Multimodal analysis of the implicit affective channel in computer-mediated textual communication

ICMI'12 - Proceedings of the ACM International Conference on Multimodal Interaction

Volumn , Issue , 2012, Pages 145-152

  Grafsgaard, Joseph F ^a   Fulton, Robert M ^b   Boyer, Kristy Elizabeth ^a   Wiebe, Eric N ^b   Lester, James C ^a  

^a North Carolina State University   (United States)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

Affect; Computer mediated communication; Depth images; Gesture; Kinect; Posture; Textual dialogue

Indexed keywords

AFFECT; COMPUTER-MEDIATED COMMUNICATION; DEPTH IMAGE; GESTURE; KINECT; POSTURE; TEXTUAL DIALOGUE;

COMMUNICATION; INTERACTIVE COMPUTER SYSTEMS; SPEECH PROCESSING; STUDENTS;

BEHAVIORAL RESEARCH;

EID: 84870184623     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2388676.2388708     Document Type: Conference Paper

References (29)

1. Baltrusaitis, T. et al. 2011. Real-Time Inference of Mental States from Facial Expressions and Upper Body Gestures. Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, 909-914.
2. Calvo, R.A. and D'Mello, S.K. 2010. Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications. IEEE Transactions on Affective Computing. 1, 1, 18-37.
3. Csikszentmihalyi, M. 1990. Flow: The Psychology of Optimal Experience. Harper-Row.
4. Derks, D. et al. 2008. The Role of Emotion in Computer-Mediated Communication: A Review. Computers in Human Behavior. 24, 3, 766-785.
5. D'Mello, S.K. and Graesser, A.C. 2010. Mining Bodily Patterns of Affective Experience during Learning. Proceedings of the International Conference on Educational Data Mining, 31-40.
6. Forbes-Riley, K. and Litman, D. 2011. Benefits and Challenges of Real-Time Uncertainty Detection and Adaptation in a Spoken Dialogue Computer Tutor. Speech Communication. 53, 9-10, 1115-1136.
7. Graesser, A.C. and Olde, B.A. 2003. How Does One Know Whether a Person Understands a Device? The Quality of the Questions the Person Asks When the Device Breaks Down. Journal of Educational Psychology. 95, 3, 524-536.
8. Hart, S.G. and Staveland, L.E. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Human Mental Workload. P.A. Hancock and N. Meshkati, eds. Elsevier Science. 139-183.
9. Kapoor, A. et al. 2007. Automatic Prediction of Frustration. International Journal of Human-Computer Studies. 65, 8, 724-736.
10. Kiesler, S. et al. 1984. Social Psychological Aspects of Computer-Mediated Communication. American Psychologist. 39, 10, 1123-1134.
11. Kort, B. et al. 2001. An Affective Model of Interplay between Emotions and Learning: Reengineering Educational Pedagogy-Building a Learning Companion. Proceedings of the IEEE International Conference on Advanced Learning Technologies, 43-46.
12. Mahmoud, M. et al. 2011. 3D Corpus of Spontaneous Complex Mental States. Proceedings of the International Conference on Affective Computing and Intelligent Interaction, 205-214.
13. Mahmoud, M. and Robinson, P. 2011. Interpreting Hand-Over-Face Gestures. Proceedings of the International Conference on Affective Computing and Intelligent Interaction, 248-255.
14. Marcoccia, M. and Atifi, H. 2008. Text-Centered Versus Multimodal Analysis of Instant Messaging Conversation. Language@Internet. 5.
15. McDuff, D. et al. 2011. Crowdsourced Data Collection of Facial Responses. Proceedings of the 13th International Conference on Multimodal Interfaces, 11-18.
16. McNeill, D. 2005. Gesture & Thought. The University of Chicago Press.
17. Neviarouskaya, A. et al. 2011. Affect Analysis Model: Novel Rule-Based Approach to Affect Sensing from Text. Natural Language Engineering. 17, 01, 95-135.
18. Neviarouskaya, A. et al. 2007. Analysis of Affect Expressed through the Evolving Language of Online Communication. Proceedings of the 12th International Conference on Intelligent User Interfaces, 278-281.
19. Neviarouskaya, A. et al. 2007. Textual Affect Sensing for Sociable and Expressive. Proceedings of the International Conference on Affective Computing and Intelligent Interaction, 220-231.
20. O'Brien, H.L. and Toms, E.G. 2010. The Development and Evaluation of a Survey to Measure User Engagement. Journal of the American Society for Information Science and Technology. 61, 1, 50-69.
21. Pantic, M. et al. 2006. Human Computing and Machine Understanding of Human Behavior: A Survey. Proceedings of the 8th International Conference on Multimodal Interaction, 239-248.
22. Perry, M.S. and Werner-Wilson, R.J. 2011. Couples and Computer-Mediated Communication: A Closer Look at the Affordances and Use of the Channel. Family and Consumer Sciences Research Journal. 40, 2, 120-134.
23. Quek, F. et al. 2002. Multimodal Human Discourse : Gesture and Speech. ACM Transactions on Computer-Human Interaction. 9, 3, 171-193.
24. Sanghvi, J. et al. 2011. Automatic Analysis of Affective Postures and Body Motion to Detect Engagement with a Game Companion. Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, 305-311.
25. Schuller, B. et al. 2011. Recognising Realistic Emotions and Affect in Speech: State of the Art and Lessons Learnt from the First Challenge. Speech Communication. 53, 9-10, 1062-1087.
26. Walther, J.B. 1992. Interpersonal Effects in Computer-Mediated Interaction: A Relational Perspective. Communication Research. 19, 1, 52-90.
27. Walther, J.B. 2007. Selective Self-Presentation in Computer-Mediated Communication: Hyperpersonal Dimensions of Technology, Language, and Cognition. Computers in Human Behavior. 23, 5, 2538-2557.
28. Woolf, B.P. et al. 2009. Affect-Aware Tutors: Recognising and Responding to Student Affect. International Journal of Learning Technology. 4, 3-4, 129-164.
29. Zeng, Z. et al. 2009. A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence. 31, 1, 39-58.