Designing for automatic affect inference in learning environments

Educational Technology and Society

Volumn 14, Issue 4, 2011, Pages 21-34

  Afzal, Shazia ^a   Robinson, Peter ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Affective computing; Computer based learning; Emotions in learning; Facial affect analysis

Indexed keywords

EID: 84856821090     PISSN: 11763647     EISSN: 14364522     Source Type: Journal    
DOI: None     Document Type: Article

References (71)

1. Afzal, S., & Robinson, P. (2009). Natural Affect Data - Collection and Annotation in a Learning Context. Affective Computing & Intelligent Interaction. Amsterdam.
2. Alexander, S. T., Hill, S., & Sarrafzadeh, A. (2005). How do Human Tutors Adapt to Affective State? Proceedings of User Modelling. Edinburgh, Scotland.
3. Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005). Deciphering the enigmatic face: The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16 (5), 403-410.
4. Amershi, S., Conati, C., & Maclaren, H. (2006). Using Feature Selection and Unsupervised Clustering to Identify Affective Expressions in Educational Games. Workshop on Motivational and Affective Issues in ITS. Intelligent Tutoring Systems, (pp. 21-28).
5. Baker, R. S. (2007). Modeling and Understanding Students' Off-Task Behaviour in Intelligent Tutoring Systems. CHI (pp. 1059-1068). San Jose, USA: ACM.
6. Baron-Cohen, S., Golan, O., Wheelwright, S., & Hill, J. (2004). Mind Reading: The Interactive Guide to Emotions. London: Jessica Kingsley Publishers.
7. Bloom, B. S. (1984). The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educational Researcher, 13 (6), 4-16.
8. Boekaerts, M. (1992). The adaptable learning process: Initiating and maintaining behavioural change. Journal of Applied Psychology: An International Review, 41, 377-397.
9. Boekaerts, M. (2002). The online motivation questionnaire: A self-report instrument to assess students' context sensitivity. New Directions in Measures and Methods, 77-120.
10. Boekaerts, M. (2003). Towards a model that integrates motivation, affect and learning. British Journal of Educational Psychology Monograph. BJEP Monograph Series II, Number 2 - Development and Motivation, 1 (1), 173-189.
11. Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How People Learn: Brain, Mind, Experience and School. Washington, DC: National Academy Press.
12. Cohen, I., Sebe, N., Garg, A., Chen, L. S., & Huang, T. S. (2003). Facial Expression Recognition from Video Sequences: Temporal and Static Modelling. Computer Vision and Image Understanding, 91, 160-187.
13. Cohn, J. F. (2006). Foundations of Human Computing: Facial Expression and Emotion. International Conference on Multimodal Interfaces (ICMI). Banff, Canada: ACM.
14. Conati, C. (2002). Probabilistic Assessment of User's Emotions in Educational Games. Journal of Applied Artificial Intelligence, 16, 555-575.
15. Conati, C., & Maclaren, H. (2004). Evaluating a Probabilistic Model of Student Affect. 7th International Conference on Intelligent Tutoring Systems (ITS). Maceio, Brazil.
16. Conati, C., & Zhou, X. (2002). Modelling Students' Emotions from Cognitive Appraisal in Educational Games. Paper presented at the International Conference on Intelligent Tutoring Systems, June 2-7, Biarritz, France and San Sebastian, Spain.
17. Cowie, R. (2009). Perceiving emotion: towards a realistic understanding of the task. Philosophical Transactions of the Royal Society B: Biological Science, 364 (1535), 3515.
18. Cowie, R., & Cornelius, R. (2003). Describing the emotional states that are expressed in speech. Speech Communication, 40, 5-32.
19. Cowie, R., & McKeown, G. (2009). The challenges of dealing with distributed signs of emotion: theory and empirical evidence. Affective Computing and Intelligent Interaction (ACII) (pp. 351-356). Amsterdam: IEEE.
20. Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., et al. (2001). Emotion Recognition in Human-Computer Interaction. IEEE Signal Processing Magazine, 18 (1), 32-80.
21. Craig, S. D., Graesser, A. C., Sullins, J., & Gholson, B. (2004). Affect and learning: an exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29, 241-250
22. de Vicente, A., & Pain, H. (2002). Informing the Detection of the Students' Motivational State: An Empirical Study. Paper presented at the International Conference on Intelligent Tutoring Systems, June 2-7, Biarritz, France and San Sebastian, Spain.
23. de Vicente, A., & Pain, H. (1998). Motivation Diagnosis in Intelligent Tutoring Systems. In B. P. Goettl, C. Halff, C. L. Redfield, & V. J. Shute (Ed.), Intelligent Tutoring Systems, (pp. 86-95). Texas.
24. Douglas-Cowie, E., Cowie, R., Sneddon, I., Cox, C., Lowry, O., McRorie, M., et al. (2007). The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data. Affective Computing and Intelligent Interaction (pp. 488-500). Springer.
25. D'Mello, S. K., Craig, S. D., Gholson, B., Franklin, S., Picard, R. W., & Graesser, A. C. (2005). Integrating Affect Sensors in an Intelligent Tutoring System. Retrieved May 1, 2011 from http://affect.media.mit.edu/pdfs/05.dmello-etal.pdf.
26. D'Mello, S., & Graesser, A. (2007). Mind and Body: Dialogue and Posture for Affect Detection in Learning Environments. International Conference on Artificial Intelligence in Education. Los Angeles.
27. D'Mello, S., Picard, R. W., & Graesser, A. (2007). Towards An Affect-Sensitive Auto-Tutor. IEEE Intelligent Systems, 22 (4), 53.
28. D'Mello, S., Taylor, R., Davidson, K., & Graesser, A. (2008). Self Versus Teacher Judgements of Learner Emotions During a Tutoring Session with AutoTutor. Intelligent Tutoring Systems (ITS), (pp. 9-18).
29. du Boulay, B., & Luckin, R. (2001). Modelling Human Teaching Tactics and Strategies for Tutoring Systems. International Journal of Artificial Intelligence in Education, 12, 235-256.
30. Ekman, P., & Friesen, W. V. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement. Palo Alto, CA: Consulting Psychologists Press.
31. Essa, I. A. (1997). Coding, Analysis, Interpretation, and Recognition of Facial Expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (7), 757-763.
32. Har-Peled, S., Roth, D., & Zimak, D. (2003). Constraint Classification for Multiclass Classification and Ranking. (S. Becker, & K. Obermayer, Eds.) Advances in Neural Information Processing Systems, 15.
33. Heylen, D., Ghijsen, M., Nijholt, A., & Akker, R. (2005). Facial Signs of Affect During Tutoring Sessions. In J. Tao J, & R. W. Picard (Ed.), Affective Computing and Intelligent Interaction. Lecture Notes in Computer Science, 3784, 24-31.
34. Hudlicka, E. (2003). To feel or not to feel: The role of affect in human-computer interaction. International Journal of Human-Computer Studies, 59, 1-32.
35. Jaques, P. A., & Vicari, R. M. (2007). A BDI approach to infer students' emotions in an intelligent learning environment. Computers & Education, 49, 360-384.
36. Jarvenoja, H., & Jarvela, S. (2005). How students describe the sources of their emotional and motivational experiences during the learning process: A qualitative approach. Learning and Instruction, 15 (5), 465-480.
37. Immordino-Yang, M. H., & Damasio, A. (2007). We Feel, Therefore We Learn: The Relevance of Affective and Social Neuroscience to Education, Mind, Brain and Education. Mind, Brain and Education, 1 (1), 3-10.
38. Kapoor, A., Burleson, W., & Picard, R. W. (2007). Automatic Prediction of Frustration. Journal of Human-Computer Studies, 65 (8), 724-736.
39. Kapoor, A., & Picard, R. W. (2005). Multimodal Affect Recognition in Learning Environments. 13th Annual ACM International Conference on Multimedia. Singapore.
40. Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: Reengineering educational pedagogy-building a learning companion. IEEE International Conference on Advanced Learning Technology: Issues, Achievements and Challenges, (pp. 43-48). Madison.
41. Lepper, M. R., Woolverton, M., Mumme, D. L., & Gurtner, J. (1993). Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. (S. P. Lajoie, & S. J. Derry, Eds.) Computers as Cognitive Tools, 75-105.
42. Liao W, W., Zhang, W., Zhu, Z., Ji, Q., & Gray, W. D. (2006). Toward a decision-theoretic framework for affect recognition and user assistance. International Journal of Human-Computer Studies, 64, 847-873.
43. Lisetti, C., & Schiano, D. (2000). Facial expression recognition: Where Human Computer Interaction, Artificial Intelligence and Cognitive Science Intersect. Pragmatics and Cognition, 8 (1), 185-235.
44. Litman, D., & Forbes, K. (2003). Recognising emotions from student speech in tutoring dialogues. Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), New York: IEEE, 25-30.
45. Littlewort, G., Bartlett, M. S., Fasel, I., Susskind, J., & Movellan, J. (2006). Dynamics of Facial Expression Extracted Automatically from Video. Image and Vision Computing, 24 (6), 615.
46. Mavrikis, M., Maciocia, A., & Lee, J. (2007). Towards Predictive Modelling of Student Affect from Web-Based Interactions. Proceedings of the International Conference on Artificial Intelligence in Education. Amsterdam: IOS Press, 169-176.
47. Merill, D. C., Reiser, B. J., Trafton, J. G., & Ranney, M. (1992). Effective Tutoring Techniques: A Comparison of Human Tutors and Intelligent Tutoring Systems. Journal of the Learning Sciences, 2, 277-305.
48. Meyer, D. K., & Turner, J. C. (2002). Discovering emotion in classroom motivation research. Educational Psychologist, 37, 107-114.
49. O'Regan, K. (2003). Emotion and e-learning. Journal of Asynchronous Learning Networks, 7 (3), 78-92.
50. Oliver, N., & Horvitz, E. (2005). A Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities. Lecture Notes in Computer Science, 3538, 199-208.
51. Ortony, A., Clore, G. L., & Collins, A. (1998). The Cognitive Structure of Emotions. Cambridge: Cambridge University Press.
52. Pantic, M., & Patras, I. (2006). Dynamics of Facial Expression: Recognition of Facial Actions and Their Temporal Segments From Face Profile Image Sequences. IEEE Transactions on Systems, Man, and Cybernetics, 36 (2), 433-449.
53. Pantic, M., & Rothkrantz, L. J. (2003). Toward an Affect-Sensitive Multimodal Human-Computer Interaction. Proceedings of the IEEE, 91 (9), 1370-1390.
54. Park, S.H., & Furnkranz, J. (2007). Efficient Pairwise Classification. Lecture Notes in Computer Science, 4701, 658-665.
55. Pekrun, R. (2005). Progress and open problems in educational emotion research. Learning and Instruction, 15, 497-506.
56. Pekrun, R. (1992). The Impact of Emotions on Learning and Achievement: Towards a Theory of Cognitive/Motivational Mediators. Applied Psychology, 41, 359-376.
57. Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students' self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37, 91-105.
58. Peter, C., & Herbon, A. (2006). Emotion representation and physiology assignments in digital systems. Interacting with Computers, 18, 139-170.
59. Picard, R. W., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., et al. (2004). Affective Learning-a manifesto. BT Technology Journal, 22, 253-269.
60. Rabiner, L. R. (1989). A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE, 77 (2), 257-285.
61. Rifkin, R., & Klautau, A. (2004). In Defense of One-Vs-All Classification. Journal of Machine Learning Research, 5, 101-141.
62. Sarrafzadeh, A., Fan, C., Dadgostar, F., Alexander, S., & Messom, C. (2004). Frown gives game away: Affect sensitive tutoring systems for elementary mathematics. IEEE Conference on Systems, Man and Cybernetics. The Hague.
63. Scherer, K. L. (2005). What are emotions? And how can they be measured? Social Science Information, 44 (4), 695-729.
64. Schutz, P. A., Hong, J. Y., Cross, D. I., & Obson, J. N. (2006). Reflections on Investigating Emotion in Educational Activity Settings. Educational Psychology Review, 18, 343-360.
65. Tong, Y., Liao, W., & Ji, Q. (2007). Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29 (10), 1683-1699.
66. Turner, J. E., Husman, J., & Schallert, D. L. (2002). The importance of students'-goals in their emotional experience of academic failure: Investigating the precursors and the consequences of shame. Educational Psychologist, 92, 548-573.
67. van Vuuren, S. (2006). Technologies that power pedagogical agents and visions for the future. Retrieved May 1, 2011 from http://www.bltek.com/images/workshops/2004/ed_tech_pedagogical_agents_022206.pdf.
68. Whitehall, J., Bartlett, M., & Movellan, J. (2008). Automatic Facial Expression Recognition for Intelligent Tutoring Systems. Paper presented at the IEEE Computer Vision and Pattern Recognition Conference, June 23-28, Anchorage, Alaska, USA.
69. Wosnitza, M., & Volet, S. (2005). Origin, direction and impact of emotions in social online learning. Learning and Instruction, 15 (5), 440-464.
70. Zakharov, K., Mitrovic, A., & Johnston, L. (2008). Towards Emotionally-Intelligent Pedagogical Agents. Lecture Notes in Computer Science, 5091, 19-28.
71. Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009). A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Transactions on Pattern Analysis & Machine Intelligence, 31 (1), 39-58.