Automatically recognizing facial indicators of frustration: A learning-centric analysis

Proceedings - 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII 2013

Volumn , Issue , 2013, Pages 159-165

  Grafsgaard, Joseph F ^a   Wiggins, Joseph B ^a   Boyer, Kristy Elizabeth ^a   Wiebe, Eric N ^b   Lester, James C ^a  

^a North Carolina State University   (United States)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

Affect; Computer mediated tutoring; Facial action units; Facial expression recognition; Frustration; Intensity; Learning

Indexed keywords

AFFECT; COMPUTER-MEDIATED TUTORING; FACIAL ACTION; FACIAL EXPRESSION RECOGNITION; FRUSTRATION; INTENSITY; LEARNING;

ASSOCIATION REACTIONS; GESTURE RECOGNITION; HUMAN COMPUTER INTERACTION;

FACE RECOGNITION;

EID: 84893321453     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ACII.2013.33     Document Type: Conference Paper

References (28)

1. R. S. J. d. Baker, S. K. D'Mello, M. M. T. Rodrigo, and A. C. Graesser, "Better to Be Frustrated than Bored: The Incidence, Persistence, and Impact of Learners' Cognitive-Affective States during Interactions with Three Different Computer-Based Learning Environments," International Journal of Human-Computer Studies, vol. 68, no. 4, pp. 223-241, Apr. 2010.
2. S. K. D'Mello, S. D. Craig, and A. C. Graesser, "Multi-Method Assessment of Affective Experience and Expression during Deep Learning," International Journal of Learning Technology, vol. 4, no. 3/4, pp. 165-187, 2009.
3. B. P. Woolf, W. Burleson, I. Arroyo, T. Dragon, D. G. Cooper, and R. W. Picard, "Affect-Aware Tutors: Recognising and Responding to Student Affect," International Journal of Learning Technology, vol. 4, no. 3-4, pp. 129-164, 2009.
4. J. F. Grafsgaard, K. E. Boyer, and J. C. Lester, "Toward a Machine Learning Framework for Understanding Affective Tutorial Interaction," in Proceedings of the 11th International Conference on Intelligent Tutoring Systems, 2012, pp. 52-58.
5. J. F. Grafsgaard, K. E. Boyer, and J. C. Lester, "Predicting Facial Indicators of Confusion with Hidden Markov Models," in Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction, 2011, pp. 97-106.
6. S. W. McQuiggan, S. Lee, and J. C. Lester, "Early Prediction of Student Frustration," in Proceedings of the Second International Conference on Affective Computing and Intelligent Interaction, 2007, pp. 698-709.
7. A. Kapoor, W. Burleson, and R. W. Picard, "Automatic Prediction of Frustration," International Journal of Human-Computer Studies, vol. 65, no. 8, pp. 724-736, Aug. 2007.
8. P. Ekman and W. V. Friesen, Facial Action Coding System. Palo Alto, CA: Consulting Psychologists Press, 1978.
9. P. Ekman, W. V. Friesen, and J. C. Hager, Facial Action Coding System: Investigator's Guide. Salt Lake City, USA: A Human Face, 2002.
10. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
11. R. A. Calvo and S. K. D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications," IEEE Transactions on Affective Computing, vol. 1, no. 1, pp. 18-37, 2010.
12. R. El Kaliouby and P. Robinson, "The Emotional Hearing Aid: an Assistive Tool for Children with Asperger Syndrome," Universal Access in the Information Society, vol. 4, no. 2, pp. 121-134, Aug. 2005.
13. T. Ruf, A. Ernst, and C. Kublbeck, "Face Detection with the Sophisticated High-speed Object Recognition Engine (SHORE)," in Microelectronic Systems, 2011, pp. 243-252.
14. M. J. den Uyl and H. van Kuilenburg, "The FaceReader: Online Facial Expression Recognition," in Proceedings of Measuring Behavior 2005, 2008, pp. 589-590.
15. G. Littlewort, J. Whitehill, T. Wu, I. Fasel, M. Frank, J. Movellan, and M. Bartlett, "The Computer Expression Recognition Toolbox (CERT)," in Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, 2011, pp. 298-305.
16. T. Wu, N. J. Butko, P. Ruvolo, J. Whitehill, M. S. Bartlett, and J. R. Movellan, "Multi-Layer Architectures for Facial Action Unit Recognition," IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, vol. 42, no. 4, pp. 1027-1038, 2012.
17. J. Hamm, C. G. Kohler, R. C. Gur, and R. Verma, "Automated Facial Action Coding System for Dynamic Analysis of Facial Expression in Neuropsychiatric Disorders," Journal of Neuroscience Methods, vol. 200, no. 2, pp. 224-238, 2011.
18. J. F. Grafsgaard, J. B. Wiggins, K. E. Boyer, E. N. Wiebe, and J. C. Lester, "Automatically Recognizing Facial Expression: Predicting Engagement and Frustration," in Proceedings of the 6th International Conference on Educational Data Mining, 2013.
19. S. Afzal and P. Robinson, "Natural Affect Data - Collection & Annotation in a Learning Context," in Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction, 2009, pp. 1-7.
20. I. Arroyo, D. G. Cooper, W. Burleson, B. P. Woolf, K. Muldner, and R. M. Christopherson, "Emotion Sensors Go To School," in 14th International Conference on Artificial Intelligence in Education, 2009, pp. 17-24.
21. G. Littlewort, M. S. Bartlett, L. P. Salamanca, and J. Reilly, "Automated Measurement of Children's Facial Expressions during Problem Solving Tasks," in Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, 2011, pp. 30-35.
22. J. Whitehill, Z. Serpell, A. Foster, Y.-C. Lin, B. Pearson, M. Bartlett, and J. Movellan, "Towards an Optimal Affect-Sensitive Instructional System of Cognitive Skills," in Proceedings of the Computer Vision and Pattern Recognition Workshop on Human Communicative Behavior, 2011, pp. 20-25.
23. H. L. O'Brien and E. G. Toms, "The Development and Evaluation of a Survey to Measure User Engagement," Journal of the American Society for Information Science and Technology, vol. 61, no. 1, pp. 50-69, 2010.
24. S. G. Hart and L. E. Staveland, "Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research," in Human Mental Workload, P. A. Hancock and N. Meshkati, Eds. Amsterdam: Elsevier Science, 1988, pp. 139-183.
25. S. G. Hart, "NASA-Task Load Index (NASA-TLX); 20 Years Later," in Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, 2006, vol. 50, no. 9, pp. 904-908.
26. M. Pantic, A. Pentland, A. Nijholt, and T. Huang, "Human Computing and Machine Understanding of Human Behavior: A Survey," in Proceedings of the 8th International Conference on Multimodal Interaction, 2006, pp. 239-248.
27. J. A. Harrigan and D. M. O'Connell, "How Do You Look When Feeling Anxious? Facial Displays of Anxiety," Personality and Individual Differences, vol. 21, no. 2, pp. 205-212, 1996.
28. D. Matsumoto and P. Ekman, "The Relationship Among Expressions, Labels, and Descriptions of Contempt," Journal of Personality and Social Psychology, vol. 87, no. 4, pp. 529-540, Oct. 2004.