A hierarchical probabilistic framework for recognizing learners' interaction experience trends and emotions

Advances in Human-Computer Interaction

Volumn 2014, Issue , 2014, Pages

  Jraidi, Imène ^a   Chaouachi, Maher ^a   Frasson, Claude ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84900000295     PISSN: 16875893     EISSN: 16875907     Source Type: Journal    
DOI: 10.1155/2014/632630     Document Type: Article

References (80)

1. Alben L., Quality of experience: defining the criteria for effective interaction design. Interactions 1996 3 3 11 15 10.1145/235008.235010
2. Berka C., Levendowski D. J., Lumicao M. N., Yau A., Davis G., Zivkovic V. T., Olmstead R. E., Tremoulet P. D., Craven P. L., EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviation Space and Environmental Medicine 2007 78 5 B231 B244 2-s2.0-34248205762
3. Brave S., Nass C., Jacko J., Sears A., Emotion in human-computer interaction. Handbook of Human-Computer Interaction 2002 Elsevier Science 81 96
4. Cowie R., Douglas-Cowie E., Tsapatsoulis N., Votsis G., Kollias S., Fellenz W., Taylor J. G., Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine 2001 18 1 32 80 2-s2.0-0035067995 10.1109/79.911197
5. Hudlicka E., To feel or not to feel: the role of affect in human-computer interaction. International Journal of Human Computer Studies 2003 59 1-2 1 32 2-s2.0-0038209645 10.1016/S1071-5819(03)00047-8
6. Matthias S. H., Uwe M., Thomas K., Adaptive User Interfaces: Principles and Practice 1993 New York, NY, USA Elsevier Science
7. Picard R., Affective Computing 1997 MIT Press
8. Bechara A., Damasio H., Damasio A. R., Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex 2000 10 3 295 307 2-s2.0-0034059124
9. Damasio A., Descartes' Error: Emotion, Reason and the Human Brain 1994 New York, NY, USA Grosset and Putnam
10. Goleman D., Emotional Intelligence 1995 New York, NY, USA Bantam Books
11. Isen A. M., Lewis M., Haviland-Jones J. M., Positive affect and decision making. Handbook of Emotions 2000 New York, NY, USA Guilford Press 417 435
12. Arroyo I., Cooper D. G., Burleson W., Woolf B. P., Muldner K., Christopherson R., Emotion sensors go to school. Frontiers in Artificial Intelligence and Applications 2009 200 1 17 24 2-s2.0-73149108640 10.3233/978-1-60750-028-5-17
13. Conati C., Maclaren H., Empirically building and evaluating a probabilistic model of user affect. User Modelling and User-Adapted Interaction 2009 19 3 267 303 2-s2.0-68149100832 10.1007/s11257-009-9062-8
14. D'Mello S. K., Craig S. D., Witherspoon A., McDaniel B., Graesser A., Automatic detection of learner's affect from conversational cues. User Modelling and User-Adapted Interaction 2008 18 1-2 45 80 2-s2.0-38749099104 10.1007/s11257-007-9037-6
15. Forbes-Riley K., Litman D., Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system. Computer Speech and Language 2011 25 1 105 126 2-s2.0-77955427299 10.1016/j.csl.2009.12.002
16. Kapoor A., Burleson W., Picard R. W., Automatic prediction of frustration. International Journal of Human Computer Studies 2007 65 8 724 736 2-s2.0-34249276773 10.1016/j.ijhcs.2007.02.003
17. McQuiggan S. W., Lee S., Lester J. C., Early prediction of student frustration. Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction 2007 Lisbon, Portugal Springer
18. Woolf B., Burleson W., Arroyo I., Dragon T., Cooper D., Picard R., Affect aware tutors: recognising and responding to student affect. International Journal of Learning Technology 2009 4 3-4 129 164 10.1504/IJLT.2009.028804
19. Allanson J., Fairclough S. H., A research agenda for physiological computing. Interacting with Computers 2004 16 5 857 878 2-s2.0-9544239374 10.1016/j.intcom.2004.08.001
20. Cacioppo J. T., Berntson G. C., Poehlmann K. M., Ito T. A., Lewis M., Haviland-Jones J. M., The psychophysiology of emotions. Handbook of Emotions 2000 2nd The Guilford Press
21. Fairclough S. H., Fundamentals of physiological computing. Interacting with Computers 2009 21 1-2 133 145 10.1016/j.intcom.2008.10.011
22. Baker R. S. J. D., D'Mello S. K., Rodrigo M. M. T., Graesser A. C., Better to be frustrated than bored: the incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human Computer Studies 2010 68 4 223 241 2-s2.0-76649103360 10.1016/j.ijhcs.2009.12.003
23. Craig S., Graesser A., Sullins J., Gholson B., Affect and learning: an exploratory look into the role of affect in learning with autotutor. Journal of Educational Media 2004 29 3 241 250
24. Picard R. W., Papert S., Bender W., Blumberg B., Breazeal C., Cavallo D., Machover T., Resnick M., Roy D., Strohecker C., Affective learning-a manifesto. BT Technology Journal 2004 22 4 253 269 2-s2.0-8644249082 10.1023/B:BTTJ. 0000047603.37042.33
25. Graesser A. C., D'mello S. K., Chipman P., King B., McDaniel B., Exploring relationships between affect and learning with autotutor. Proceedings of the 13th International Conference on Artificial Intelligence in Education 2007 16 23
26. Hara N., Student distress in a web-based distance education course. Information, Communication & Society 2000 3 4 557 579 10.1080/ 13691180010002297
27. O'Regan K., Emotion and E-learning. Journal of Asynchronous Learning Network 2003 7 3 78 92 2-s2.0-3042536233
28. Muse L. A., Harris S. G., Feud H. S., Has the inverted-u theory of stress and job performance had a fair test? Human Performance 2003 16 4 349 364 2-s2.0-0141907723
29. Sullivan S. E., Bhagat R. S., Organizational stress, job satisfaction and job performance: where do we go from here? Journal of Management 1992 18 2 353 374 10.1177/014920639201800207
30. Graesser A. C., Olde B. A., How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. Journal of Educational Psychology 2003 95 3 524 536 2-s2.0-0141484686 10.1037/0022-0663.95.3.524
31. VanLehn K., Siler S., Murray C., Yamauchi T., Baggett W. B., Why do only some events cause learning during human tutoring? Cognition and Instruction 2003 21 3 209 249 2-s2.0-0041760343
32. Hone K., Empathic agents to reduce user frustration: the effects of varying agent characteristics. Interacting with Computers 2006 18 2 227 245 2-s2.0-33144473090 10.1016/j.intcom.2005.05.003
33. Klein J., Moon Y., Picard R. W., This computer responds to user frustration: theory, design, and results. Interacting with Computers 2002 14 2 119 140 2-s2.0-0036473090
34. Healey J., Picard R., Smartcar: detecting driver stress. 4 Proceedings of the 15th International Conference on Pattern Recognition 2000 218 221
35. Picard R. W., Liu K. K., Relative subjective count and assessment of interruptive technologies applied to mobile monitoring of stress. International Journal of Human Computer Studies 2007 65 4 361 375 2-s2.0-33846827095 10.1016/j.ijhcs.2006.11.019
36. Zhai J., Barreto A., Stress detection in computer users based on digital signal processing of noninvasive physiological variables. Proceedings of the 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS '06) September 2006 1355 1358 2-s2.0-34047139992 10.1109/IEMBS.2006.259421
37. Jraidi I., Frasson C., Student's uncertainty modeling through a multimodal sensor-based approach. Educational Technology & Society 2013 16 1 219 230
38. Pon-Barry H., Schultz K., Bratt E. O., Clark B., Peters S., Responding to student uncertainty in spoken tutorial dialogue systems. International Journal of Artificial Intelligence in Education 2006 16 2 171 194
39. Caldwell J. A., Hall K. K., Erickson B. S., EEG data collected from helicopter pilots in flight are sufficiently sensitive to detect increased fatigue from sleep deprivation. The International Journal of Aviation Psychology 2002 12 1 19 32 2-s2.0-0012085093
40. Heitmann A., Guttkuhn R., Trutschel U., Moore-Ede M., Technologies for the monitoring and prevention of driver fatigue. Proceedings of the 1st International Driving Symposium on Human Factors in Driving Assessment, Training, and Vehicle Design 2001 81 86
41. Ji Q., Lan P., Looney C., A probabilistic framework for modeling and real-time monitoring human fatigue. IEEE Transactions on Systems, Man and Cybernetics A: Systems and Humans 2006 36 5 862 875 10.1109/tsmca.2005.855922
42. Kim K. H., Bang S. W., Kim S. R., Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing 2004 42 3 419 427 2-s2.0-2942731250 10.1007/BF02344719
43. Scherer K. R., Wranik T., Sangsue J., Tran V., Scherer U., Emotions in everyday life: probability of occurrence, risk factors, appraisal and reaction patterns. Social Science Information 2004 43 4 499 570 2-s2.0-10844265255 10.1177/0539018404047701
44. Pantic M., Rothkrantz L. J. M., Facial action recognition for facial expression analysis from static face images. IEEE Transactions on Systems, Man, and Cybernetics B: Cybernetics 2004 34 3 1449 1461 2-s2.0-2942612281 10.1109/TSMCB.2004.825931
45. Ashraf A. B., Lucey S., Cohn J. F., Chen T., Ambadar Z., Prkachin K. M., Solomon P. E., The painful face-pain expression recognition using active appearance models. Image and Vision Computing 2009 27 12 1788 1796 2-s2.0-70349283376 10.1016/j.imavis.2009.05.007
46. Ma L., Khorasani K., Facial expression recognition using constructive feedforward neural networks. IEEE Transactions on Systems, Man, and Cybernetics B: Cybernetics 2004 34 3 1588 1595 2-s2.0-2942522626 10.1109/TSMCB.2004.825930
47. Petrushin V. A., Emotion in speech: recognition and application to call centers. Proceedings of the Artificial Neural Networks in Engineering Conference (ANNIE '99) November 1999 1085 1092 2-s2.0-0033329296
48. Ang J., Dhillon R., Krupski A., Shriberg E., Stolcke A., Prosody-based automatic detection of annoyance and frustration in human-computer dialog. Proceedings of the 7th International Conference on Spoken Language Processing 2002 Denver, Colo, USA 2037 2040
49. Litman D. J., Forbes-Riley K., Predicting student emotions in computer-human tutoring dialogues. Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics 2004 Barcelona, Spain Association for Computational Linguistics
50. Kaliouby R. E., Robinson P., Real-time inference of complex mental states from facial expressions and head gestures. 10 Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop (CVPRW '04) 2004 154 10.1109/CVPR.2004.153
51. Liao W., Zhang W., Zhu Z., Ji Q., Gray W. D., Toward a decision-theoretic framework for affect recognition and user assistance. International Journal of Human Computer Studies 2006 64 9 847 873 2-s2.0-33745676838 10.1016/j.ijhcs. 2006.04.001
52. Csikszentmihalyi M., The Psychology of Optimal Experience 1990 New York, NY, USA Harper & Row
53. Burleson W., Picard R. W., Affective agents: sustaining motivation to learn through failure and a state of stuck. Proceedings of the Social and Emotional Intelligence in Learning Environments, Workshop in conjunction with the 7th International Conference on Intelligent Tutoring Systems 2004 Maceio, Alagoas, Brasil
54. Baker R. S. J. D., Corbett A. T., Roll I., Koedinger K. R., Developing a generalizable detector of when students game the system. User Modelling and User-Adapted Interaction 2008 18 3 287 314 2-s2.0-45449102857 10.1007/s11257-007-9045-6
55. Beck J. E., Engagement tracing: using response times to model student disengagement. Artificial Intelligence in Education: Supporting Learning Through Intelligent and Socially Informed Technology 2005 IOS Press
56. Rodrigo M. M. T., Baker R. S. J. D., Lagud M. C. V., Lim S. A. L., Macapanpan A. F., Pascua S. A. M. S., Santillano J. Q., Sevilla L. R. S., Sugay J. O., Tep S., Viehland N. J. B., Affect and usage choices in simulation problem-solving environments. Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work 2007 IOS Press 145 152
57. Cowley B., Charles D., Black M., Hickey R., Toward an understanding of flow in video games. Computers in Entertainment 2008 6 2, article 20 2-s2.0-49449083303 10.1145/1371216.1371223
58. Jegers K., Pervasive game flow: understanding player enjoyment in pervasive gaming. Computers in Entertainment 2007 5 1 2-s2.0-34147224241 10.1145/1236224.1236238 1236238
59. Sweetser P., Wyeth P., Gameflow: a model for evaluating player enjoyment in games. Computers in Entertainment 2005 3 3 3 10.1145/1077246.1077253
60. Murphy K. P., Dynamic bayesian networks: representation, inference and learning [Ph.D. thesis] 2002 Berkeley, Calif, USA Computer Science Division, University of California
61. Lang P. J., The emotion probe: studies of motivation and attention. American Psychologist 1995 50 5 372 385 2-s2.0-0029304079
62. Lang P. J., Greenwald M. K., Bradley M. M., Hamm A. O., Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 1993 30 3 261 273 2-s2.0-0027523286
63. Altimiras J., Understanding autonomic sympathovagal balance from short-term heart rate variations. Are we analyzing noise? Comparative Biochemistry and Physiology: A Molecular and Integrative Physiology 1999 124 4 447 460 2-s2.0-0033452920 10.1016/S1095-6433(99)00137-3
64. Levenson R. W., Autonomic nervous system differences among emotions. Psychological Science 1992 3 1 23 27
65. Berka C., Levendowski D. J., Cvetinovic M. M., Petrovic M. M., Davis G., Lumicao M. N., Zivkovic V. T., Popovic M. V., Olmstead R., Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. International Journal of Human-Computer Interaction 2004 17 2 151 170 2-s2.0-4043105547
66. Sterman M. B., Mann C. A., Concepts and applications of eeg analysis in aviation performance evaluation. Biological Psychology 1995 40 1-2 115 130 10.1016/0301-0511(95)05101-5
67. Stevens R., Galloway T., Berka C., Conati C., McCoy K., Paliouras G., EEG-related changes in cognitive workload, engagement and distraction as students acquire problem solving skills. User Modeling 2007 2007 4511 Berlin, Germany Springer 187 196
68. Pope A. T., Bogart E. H., Bartolome D. S., Biocybernetic system evaluates indices of operator engagement in automated task. Biological Psychology 1995 40 1-2 187 195 2-s2.0-0028997254 10.1016/0301-0511(95)05116-3
69. Freeman F. G., Mikulka P. J., Prinzel L. J., Scerbo M. W., Evaluation of an adaptive automation system using three EEG indices with a visual tracking task. Biological Psychology 1999 50 1 61 76 2-s2.0-0032920483 10.1016/S0301-0511(99)00002-2
70. Chaouachi M., Chalfoun P., Jraidi I., Frasson C., Affect and mental engagement: towards adaptability for intelligent systems. Proceedings of the 23rd International Florida Artificial Intelligence Research Society Conference (FLAIRS-23 '10) May 2010 Daytona Beach, Fla, USA 355 360 2-s2.0-77957888104
71. Pekrun R., Elliot A. J., Maier M. A., Achievement goals and discrete achievement emotions: a theoretical model and prospective test. Journal of Educational Psychology 2006 98 3 583 597 2-s2.0-33746911574 10.1037/0022-0663. 98.3.583
72. Chaouachi M., Jraidi I., Frasson C., Konstan J., Conejo R., Marzo J. L., Oliver N., Modeling mental workload using eeg features for intelligent systems. User Modeling, Adaption and Personalization 2011 6787 Berlin, Germany Springer 50 61
73. Thought Technology. 2007, http://www.thoughttechnology.com/
74. Jasper H. H., The ten-twenty electrode system of the international federation. Electroencephalography and clinical neurophysiology 1958 10 371 375
75. John O. P., Naumann L. P., Soto C. J., John O. P., Robins R. W., Pervin L. A., Paradigm shift to the integrative big-five trait taxonomy: history, measurement, and conceptual issues. Handbook of Personality: Theory and Research 2008 114 158
76. Lauritzen S. L., The EM algorithm for graphical association models with missing data. Computational Statistics & Data Analysis 1995 19 2 191 201 10.1016/0167-9473(93)E0056-A
77. Domingos P., Pazzani M., On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning 1997 29 2-3 103 130 2-s2.0-0031269184
78. Quinlan J. R., Induction of decision trees. Machine Learning 1986 1 1 81 106 2-s2.0-33744584654 10.1007/BF00116251
79. Platt J. C., Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods 1999 MIT Press 185 208
80. Jraidi I., Chalfoun P., Frasson C., Cerri S., Clancey W., Papadourakis G., Panourgia K., Implicit strategies for intelligent tutoring systems. Intelligent Tutoring Systems 2012 7315 Berlin, Germany Springer 1 10