Automatically recognizing facial expression: Predicting engagement and frustration

Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013

Volumn , Issue , 2013, Pages

  Grafsgaard, Joseph F ^a   Wiggins, Joseph B ^a   Boyer, Kristy Elizabeth ^a   Wiebe, Eric N ^b   Lester, James C ^a  

^a North Carolina State University   (United States)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

Affect; Computer mediated tutoring; Engagement; Facial expression recognition; Frustration

Indexed keywords

DATA MINING;

AFFECT; AUTOMATED ANALYSIS; AUTOMATED TRACKING; COMPUTER EXPRESSIONS; ENGAGEMENT; FACIAL EXPRESSION RECOGNITION; FACIAL EXPRESSIONS; FRUSTRATION;

FACE RECOGNITION;

EID: 85084013878     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Baker, R.S.J. d., D’Mello, S.K., Rodrigo, M.M.T. and Graesser, A.C. 2010. Better to Be Frustrated than Bored: The Incidence, Persistence, and Impact of Learners’ Cognitive-Affective States during Interactions with Three Different Computer-Based Learning Environments. International Journal of Human-Computer Studies. 68, 4, 223–241.
2. Baker, R.S.J. d., Gowda, S.M., Wixon, M., Kalka, J., Wagner, A.Z., Salvi, A., Aleven, V., Kusbit, G.W., Ocumpaugh, J. and Rossi, L. 2012. Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra. Proceedings of the 5th International Conference on Educational Data Mining, 126–133.
3. Baltrusaitis, T., Robinson, P. and Morency, L.-P. 2012. 3D Constrained Local Model for Rigid and Non-Rigid Facial Tracking. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2610–2617.
4. Calvo, R.A. and D’Mello, S.K. 2010. Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications. IEEE Transactions on Affective Computing. 1, 1, 18–37.
5. Cooper, D.G., Muldner, K., Arroyo, I., Woolf, B.P. and Burleson, W. 2010. Ranking Feature Sets for Emotion Models used in Classroom Based Intelligent Tutoring Systems. Proceedings of the 18th International Conference on User Modeling, Adaptation, and Personalization, 135–146.
6. D’Mello, S.K. and Calvo, R.A. 2011. Significant Accomplishments, New Challenges, and New Perspectives. New Perspectives on Affect and Learning Technologies. R.A. Calvo and S.K. D’Mello, eds. Springer. 255–271.
7. D’Mello, S.K., Craig, S.D. and Graesser, A.C. 2009. Multi-Method Assessment of Affective Experience and Expression during Deep Learning. International Journal of Learning Technology. 4, 3/4, 165–187.
8. D’Mello, S.K. and Graesser, A. 2010. Multimodal Semi-automated Affect Detection From Conversational Cues, Gross Body Language, and Facial Features. User Modeling and User-Adapted Interaction. 20, 2, 147–187.
9. D’Mello, S.K., Lehman, B., Pekrun, R. and Graesser, A.C. Confusion Can Be Beneficial for Learning. Learning & Instruction. (in press)
10. Ekman, P. and Friesen, W. V. 1978. Facial Action Coding System. Consulting Psychologists Press.
11. Ekman, P., Friesen, W. V. and Hager, J.C. 2002. Facial Action Coding System: Investigator’s Guide. A Human Face.
12. Grafsgaard, J.F., Boyer, K.E. and Lester, J.C. 2011. Predicting Facial Indicators of Confusion with Hidden Markov Models. Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction, 97–106.
13. Grafsgaard, J.F., Boyer, K.E. and Lester, J.C. 2012. Toward a Machine Learning Framework for Understanding Affective Tutorial Interaction. Proceedings of the 11th International Conference on Intelligent Tutoring Systems, 52–58.
14. Grafsgaard, J.F., Fulton, R.M., Boyer, K.E., Wiebe, E.N. and Lester, J.C. 2012. Multimodal Analysis of the Implicit Affective Channel in Computer-Mediated Textual Communication. Proceedings of the 14th ACM International Conference on Multimodal Interaction, 145–152.
15. Harrigan, J.A. and O’Connell, D.M. 1996. How Do You Look When Feeling Anxious? Facial Displays of Anxiety. Personality and Individual Differences. 21, 2, 205–212.
16. Hart, S.G. and Staveland, L.E. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Human Mental Workload. P.A. Hancock and N. Meshkati, eds. Elsevier Science. 139–183.
17. Kaliouby, R. and Robinson, P. 2005. Generalization of a Vision-Based Computational Model of Mind-Reading. Proceedings of the First International Conference on Affective Computing and Intelligent Interfaces, 582–589.
18. Littlewort, G., Bartlett, M.S., Salamanca, L.P. and Reilly, J. 2011. Automated Measurement of Children’s Facial Expressions during Problem Solving Tasks. Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, 30–35.
19. Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J. and Bartlett, M. 2011. The Computer Expression Recognition Toolbox (CERT). Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, 298–305.
20. O’Brien, H.L. and Toms, E.G. 2010. The Development and Evaluation of a Survey to Measure User Engagement. Journal of the American Society for Information Science and Technology. 61, 1, 50–69.
21. Picard, R.W., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., Machover, T., Resnick, M., Roy, D. and Strohecker, C. 2004. Affective Learning — A Manifesto. BT Technology Journal. 22, 4, 253–269.
22. Rodrigo, M.M.T. and Baker, R.S.J.d. 2011. Comparing Learners’ Affect while using an Intelligent Tutor and an Educational Game. Research and Practice in Technology Enhanced Learning. 6, 1, 43–66.
23. Ruf, T., Ernst, A. and Kublbeck, C. 2011. Face Detection with the Sophisticated High-speed Object Recognition Engine (SHORE). Microelectronic Systems. 243–252.
24. den Uyl, M.J. and van Kuilenburg, H. 2008. The FaceReader: Online Facial Expression Recognition. Proceedings of Measuring Behavior 2005, 589–590.
25. Whitehill, J., Bartlett, M. and Movellan, J. 2008. Automatic Facial Expression Recognition for Intelligent Tutoring Systems. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 1–6.
26. Wu, T., Butko, N.J., Ruvolo, P., Whitehill, J., Bartlett, M.S. and Movellan, J.R. 2012. Multi-Layer Architectures for Facial Action Unit Recognition. IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, 42, 4, 1027-1038.
27. Zeng, Z., Pantic, M., Roisman, G.I. and Huang, T.S. 2009. A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence. 31, 1, 39–58.