Automatic gaze-based user-independent detection of mind wandering during computerized reading

User Modeling and User-Adapted Interaction

Volumn 26, Issue 1, 2016, Pages 33-68

  Bixler, Robert ^a   D’Mello, Sidney ^a  

^a University of Notre Dame   (United States)

Author keywords

Gaze tracking; Mind wandering; User modeling

Indexed keywords

ARTIFICIAL INTELLIGENCE; SUPERVISED LEARNING; TRACKING (POSITION);

FIXATION DURATION; GAZE TRACKING; IMPROVE PERFORMANCE; INTELLIGENT INTERFACE; MIND WANDERING; SUPERVISED MACHINE LEARNING; USER INDEPENDENTS; USER MODELING;

LEARNING SYSTEMS;

EID: 84944706244     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-015-9167-1     Document Type: Article

References (88)

1. Baayen, R., Harald, P., Richard, G.: The CELEX Lexical Database (Release 2) (1995)
2. Baer, R.A.: Mindfulness training as a clinical intervention: a conceptual and empirical review. Clin. Psychol. 10(2), 125–143 (2003)
3. Barbuceanu, F., Antonya, C., Duguleana, M., et al.: Attentive User Interface for Interaction Within Virtual Reality Environments Based on Gaze Analysis. Human-Computer Interaction, pp. 204–213. Springer, Interaction Techniques and Environments, New York (2011)
4. Ba, Sileye O., Odobez, J-M.: A Study on Visual Focus of Attention Recognition from Head Pose in a Meeting Room, pp. 75–87. Springer, In Machine Learning for Multimodal Interaction, New York (2006)
5. Bergasa, L.M., Nuevo, J., Sotelo, M.A., et al.: Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transp. Sys. 7(1), 63–77 (2006)
6. Bixler, R., D’Mello, S.: Toward fully automated person-independent detection of mind wandering. In: User Modeling, Adaptation, and Personalization, pp. 37–48. Springer, New York (2014)
7. Bixler, R., Kopp, K., D’Mello, S.: Evaluation of a Personalized Method for Proactive Mind Wandering Reduction. In: Proceedings of the 4th Workshop on Personalization Approaches for Learning Environments (PALE 2014), pp. 33, (2014)
8. Blanchard, N., Bixler, R., Joyce, T., et al.: Automated physiological-based detection of mind wandering during learning. In: Intelligent Tutoring Systems, pp. 55–60. Springer, New York (2014)
9. Börner, D., Kalz, M., Specht, M.: Lead me gently: facilitating knowledge gain through attention-aware ambient learning displays. Comput. & Educ. 78, 10–19 (2014)
10. Chawla, N.V., Bowyer, K. W., Hall, L.O., et al.: SMOTE: Synthetic Minority Over-Sampling Technique.’ J. Artif. Intell. Res. 16(1), 321–357 (2011)
11. Christoff, K., Gordon, A.M., Smallwood, J., et al.: experience Sampling during fMRI reveals default network and executive system contributions to mind wandering. Proc. Natl. Acad. Sci. 106(21), 8719–8724 (2009)
12. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)
13. Davidson, R.J.: Empirical explorations of mindfulness: conceptual and methodological conundrums. Emotion 10(1), 8–11 (2010)
14. D’Mello, S., Cobian, J., Hunter, M.: Automatic gaze-based detection of mind wandering during reading. Proceedings of the 6th International Conference on Educational Data Mining, International Educational Data Mining Society, pp. 364–365 (2013)
15. Dong, L., Di, H., Tao, L., et al.: Visual Focus of Attention Recognition in the Ambient Kitchen. In: Computer Vision–ACCV 2009, pp. 548–559. Springer, New York (2010)
16. Dong, Y., Hu, Z., Uchimura, K., et al.: Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans. Intell. Transp. Syst. 12(2), 596–614 (2011)
17. D’Orazio, T., Leo, M., Guaragnella, C., et al.: A visual approach for driver inattention detection. Pattern Recognit. 40(8), 2341–2355 (2007)
18. Drummond, J., Litman, D.: In the Zone: towards Detecting Student Zoning Out Using Supervised Machine Learning, pp. 306–308. Springer, New York, In Intelligent Tutoring Systems (2010)
19. Feng, S., D’Mello, S., Graesser, A.C.: Mind wandering while reading easy and difficult texts. Psychon. Bull. & Rev. 20(3), 586–592 (2013)
20. Fletcher, L., Zelinsky, A.: Driver inattention detection based on eye gaze-road event correlation. Int. J. Robot. Res. 28(6), 774–801 (2009)
21. Forster, S., Lavie, N.: Distracted by your mind? Individual differences in distractibility predict mind wandering. J. Exp. Psychol. 40(1), 251–260 (2014)
22. Foulsham, T., Farley, J., Kingstone, A.: Mind wandering in sentence reading: decoupling the link between mind and eye. Can. J. Exp. Psychol./Rev. Can de Psychol Exp 67(1), 51–59 (2013)
23. Frank, D.J., Nara, B., Zavagnin, M., et al.: Validating older adults’ reports of less mind-wandering: an examination of eye movements and dispositional influences. Psychol. Aging 30(2), 266–278 (2015)
24. Franklin, M.S., Broadway, J.M., Mrazek, M.D., et al.: Window to the wandering mind: pupillometry of spontaneous thought while reading. Q. J. Exp. Psychol. 66(12), 2289–2294 (2013)
25. Franklin, M.S., Smallwood, J., Schooler, J.W.: Catching the mind in flight: using behavioral indices to detect mindless reading in real time. Psychon. Bull. & Rev. 18(5), 992–997 (2011)
26. Furugori, S., Yoshizawa, N., Iname, C., et al.: Estimation of driver fatigue by pressure distribution on seat in long term driving. Rev. Automot. Eng. 26(1), 053–058 (2005)
27. Grandchamp, R., Braboszcz, C., Delorme, A.: Oculometric variations during mind wandering. Front. Psychol. 5, 1000–1078 (2014)
28. Gruberger, M., Ben-Simon, E., Levkovitz, Y., et al.: Towards a neuroscience of mind-wandering. Front. Hum. Neurosci. 5, 56–60 (2011)
29. Hall, M.A.: Correlation-based feature selection for machine learning. PhD Thesis, department of Computer Science, University of Waikato, New Zealand. (1999)
30. Hall, M., Frank, E., Holmes, G., et al.: The WEKA data mining software: an update. ACM SIGKDD Explor. Newsl. 11(1), 10–18 (2009)
31. Halpern, D.F., Millis, K., Graesser, A.C., et al.: Operation ARA: a computerized learning game that teaches critical thinking and scientific reasoning. Think. Skills Creat. 7(2), 93–100 (2012)
32. Holmqvist, K., Nyström, M., Andersson, R., et al.: Eye Tracking: a Comprehensive Guide to Methods and Measures. Oxford University Press, Oxford (2011)
33. Horvitz, E., Jacobs, A., Hovel D.: Attention-sensitive alerting’. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 305–313. Morgan Kaufmann Publishers Inc. (1999)
34. Horvitz, E., Kadie, C., Paek, T., et al.: Models of attention in computing and communication: from principles to applications. Commun. ACM 46(3), 52–59 (2003)
35. Jha, A.P., Krompinger, J., Baime, M.J.: Mindfulness training modifies subsystems of attention. Cogn. Affect. & Behav. Neurosci. 7(2), 109–119 (2007)
36. Just, M.A., Carpenter, P.A.: A theory of reading: from eye fixations to comprehension. Psychol. Rev. 87(4), 329 (1980)
37. Kane, M.J., Brown, L.H., McVay, J.C., et al.: For whom the mind wanders, and when an experience-sampling study of working memory and executive control in daily life. Psychol. Sci. 18(7), 614–621 (2007)
38. Killingsworth, M.A., Gilbert, D.T.: A wandering mind is an unhappy mind. Science 330(6006), 932–932 (2010)
39. Knipling, R.F., Wierwille, W.W.: America, IVHS: Vehicle-Based Drowsy Driver Detection: Current Status and Future Prospects. National Highway Traffic Safety Administration,Office of Crash Avoidance Research (1994)
40. Kopp, K., Bixler, R., D’Mello, S.: Identifying Learning Conditions that Minimize Mind Wandering by Modeling Individual Attributes, pp. 94–103. Springer, New York, In Intelligent Tutoring Systems (2014)
41. Lutz, A., Slagter, H.A., Rawlings, N.B., et al.: Mental training enhances attentional stability: neural and behavioral evidence. J. Neurosci. 29(42), 13418–13427 (2009)
42. Mason, M.F., Norton, M.I., Van Horn, J.D., et al.: Wandering minds: the default network and stimulus-independent thought. Science 315(5810), 393–395 (2007)
43. Matsumoto, Y., Ogasawara, T., Zelinsky, A.: Behavior recognition based on head pose and gaze direction measurement. In: Intelligent Robots and Systems, 2000. (IROS 2000). Proceedings. 2000 IEEE/RSJ International Conference on, IEEE pp. 2127–2132 (2000)
44. Mills, C., D’Mello, S.: In Press. Toward a real-time (Day) dreamcatcher: detecting mind wandering episodes during online reading. In: Proceedings of the 8th International Conference on Educational Data Mining, International Educational Data Mining Society pp. XX–XX
45. Mooneyham, B.W., Schooler, J.W.: The costs and benefits of mind-wandering: a review. Can. J. Exp. Psychol./Rev. Can. de Psychol. Exp. 67(1), 11–18 (2013)
46. Mrazek, M.D., Franklin, M.S., Phillips, D.T., et al.: Mindfulness training improves working memory capacity and gre performance while reducing mind wandering. Psychol. Sci. 24(5), 776–781 (2013)
47. Mrazek, M.D., Smallwood, J., Schooler, J.W.: Mindfulness and mind-wandering: finding convergence through opposing constructs. Emotion 12(3), 442–448 (2012)
48. Muir, M., Conati, C.: An analysis of attention to student—adaptive hints in an educational game. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., et al. (eds.) Intelligent Tutoring Systems, Lecture Notes in Computer Science, pp. 112–122. Springer, Heidelberg (2012)
49. Navalpakkam, V., Kumar, R., Li, L., et al.: Attention and selection in online choice tasks. In: Masthoff, J., Mobasher, B., Desmarais, M.C., et al. (eds.) User Modeling, Adaptation, and Personalization, Lecture Notes in Computer Science, pp. 200–211. Springer, Heidelberg (2012)
50. Pham, P., Wang, J.: Attentive learner: improving mobile MOOC learning via implicit heart rate tracking. In: Conati, C., Heffernan, N., Mitrovic, A., et al. (eds.) Artificial Intelligence in Education, pp. 367–376. Springer International Publishing, New York (2015)
51. Randall, J.G., Oswald, F.L., Beier, M.E.: Mind-wandering, cognition, and performance: a theory-driven meta-analysis of attention regulation. Psychol. Bull. 140(6), 1411–1431 (2014)
52. Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372 (1998)
53. Reichle, E.D., Pollatsek, A., Fisher, D.L., et al.: Toward a model of eye movement control in reading. Psychol. Rev. 105(1), 125 (1998)
54. Reichle, E.D., Reineberg, A.E., Schooler, J.W.: Eye movements during mindless reading. Psychol. Sci. 21(9), 1300–1310 (2010)
55. Robertson, L.H., Manly, T., Andrade, J., et al.: Oops!: performance correlates of everyday attentional failures in traumatic brain injured and normal subjects. Neuropsychologia 35(6), 747–758 (1997)
56. Roda, C., Nabeth, T.: Supporting attention in learning environments: attention support services, and information management. In: Creating New Learning Experiences on a Global Scale, pp. 277–291. Springer, New York (2007)
57. Roda, C., Thomas, J.: Attention aware systems. Encycl. Hum. Comput. Interact. 58, 38 (2006)
58. Schad, D.J., Nuthmann, A., Engbert, R.: Your mind wanders weakly, your mind wanders deeply: objective measures reveal mindless reading at different levels. Cognition 125(2), 179–194 (2012)
59. Schooler, J.W., Reichle, E.D., Halpern D.V.: Zoning out while reading: evidence for dissociations between experience and metaconsciousness. In: Daniel T.L. (ed.) Thinking and Seeing: Visual Metacognition in Adults and Children, pp. 203–226. Cambridge, Mass.: MIT Press (2004)
60. Schooler, J.W., Smallwood, J., Christoff, K., et al.: Meta-awareness, perceptual decoupling and the wandering mind. Trends Cognit. Sci. 15(7), 319–326 (2011)
61. Seibert, P.S., Ellis, H.C.: Irrelevant thoughts, emotional mood states, and cognitive task performance. Mem. & Cognit. 19(5), 507–513 (1991)
62. Selker, T.: Visual attentive interfaces. BT Technol. J. 22(4), 146–150 (2004)
63. Sewell, W., Komogortsev, O.: Real-time eye gaze tracking with an unmodified commodity webcam employing a neural network. In: CHI’10 Extended Abstracts on Human Factors in Computing Systems, ACM pp. 3739–3744, (2010)
64. Smallwood, J.: Mind-wandering while reading: attentional decoupling, mindless reading and the cascade model of inattention. Lang. Linguist. Compass 5(2), 63–77 (2011)
65. Smallwood, J., Beach, E., Schooler, J.W., et al.: Going AWOL in the brain: mind wandering reduces cortical analysis of external events. J. Cognit. Neurosci. 20(3), 458–469 (2008)
66. Smallwood, J., Brown, K.S., Tipper, C., et al.: Pupillometric evidence for the decoupling of attention from perceptual input during offline thought ’ed. Sam Gilbert. PLoS ONE 6(3), e18298 (2011)
67. Smallwood, J., Davies, J.B., Heim, D., et al.: Subjective experience and the attentional lapse: task engagement and disengagement during sustained attention. Consci. Cognit. 13(4), 657–690 (2004)
68. Smallwood, J., Fishman, D.J., Schooler, J.W.: Counting the cost of an absent mind: mind wandering as an underrecognized influence on educational performance. Psychon. Bull. & Rev. 14(2), 230–236 (2007)
69. Smallwood, J., McSpadden, M., Schooler, J.W.: When attention matters: the curious incident of the wandering mind. Mem. & Cognit. 36(6), 1144–1150 (2008)
70. Smallwood, J., Schooler, J.W.: The restless mind. Psychol. Bull. 132(6), 946–958 (2006)
71. Smilek, D., Carriere, J.S.A., Cheyne, J.A.: Out of mind, out of sight: eye blinking as indicator and embodiment of mind wandering. Psychol. Sci. 21(6), 786–789 (2010)
72. Stiefelhagen, R.: Tracking focus of attention in meetings. In: Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, IEEE Computer Society pp. 273, (2002)
73. Stiefelhagen, R., Yang, J., Waibel, A.: Estimating focus of attention based on gaze and sound. In: Proceedings of the 2001 workshop on Perceptive user interfaces, ACM pp. 1–9 (2001)
74. Stiefelhagen, R., Zhu, J.: Head orientation and gaze direction in meetings. In: CHI’02 Extended Abstracts on Human Factors in Computing Systems, ACM pp. 858–859 (2002)
75. Su, Mu-Chun, Hsiung, Chao-Yueh, Huang De-Yuan (2006) ’A Simple Approach to Implementing a System for Monitoring Driver Inattention’. In Systems, Man and Cybernetics, 2006. SMC’06. IEEE International Conference on, IEEE pp. 429–433
76. Sussman, E., Bishop, H., Madnick, B., et al.: Driver inattention and highway safety. Transp. Res. Rec. 1047, 40–48 (1985)
77. Tawari, A., Sivaraman, S., Trivedi, M.M. et al.: Looking-in and looking-out vision for urban intelligent assistance: estimation of driver attentive state and dynamic surround for safe merging and braking. In: Intelligent Vehicles Symposium Proceedings, 2014 IEEE, IEEE pp. 115–120 (2014)
78. Toet, A.: Gaze directed displays as an enabling technology for attention aware systems. Comput. Hum. Behav. 22(4), 615–647 (2006)
79. Torkkola, K., Massey, N., Wood. C.: Driver inattention detection through intelligent analysis of readily available sensors. In: Intelligent Transportation Systems, 2004. Proceedings. The 7th International IEEE Conference on, IEEE pp. 326–331 (2004)
80. Ugurlu, Y.: User attention analysis for e-learning systems using gaze and speech information. In: Information Science, Electronics and Electrical Engineering (ISEEE), 2014 International Conference on, IEEE pp. 1–5 (2014)
81. Uzzaman, S., Steve, J.: The eyes know what you are thinking: eye movements as an objective measure of mind wandering. Consci. Cognit. 20(4), 1882–1886 (2011)
82. Vertegaal, R., Shell, J.S., Chen, D., et al.: Designing for augmented attention: towards a framework for attentive user interfaces. Comput. Hum. Behav. 22(4), 771–789 (2006)
83. Voit, M., Stiefelhagen, R.: Deducing the visual focus of attention from head pose estimation in dynamic multi-view meeting scenarios. In: Proceedings of the 10th International Conference on Multimodal Interfaces, ACM pp. 173–180 (2008)
84. Voßkühler, A., Nordmeier, V., Kuchinke, L., et al.: open gaze and mouse analyzer: open-source software designed to analyze eye and mouse movements in slideshow study design. Behav. Res. Methods 40(4), 1150–1162 (2008)
85. Warren, T., White, S.J., Reichle, E.D.: Investigating the causes of wrap-up effects: evidence from eye movements and e-z reader. Cognition 111(1), 132–137 (2009)
86. Yeo, M.V.M., Li, X., Shen, K., et al.: ’Can SVM be used for automatic EEG detection of drowsiness during car driving? Saf. Sci. 47(1), 115–124 (2009)
87. Yonetani, R., Kawashima, H., Matsuyama, T.: Multi-mode saliency dynamics model for analyzing gaze and attention. Proceedings of the Symposium on Eye Tracking Research and Applications, ETRA ’12, New York, NY, pp. 115–122. ACM, New York (2012)
88. Zeidan, F., Johnson, S.K., Diamond, B.J., et al.: Mindfulness meditation improves cognition: evidence of brief mental training. Consci. Cognit. 19(2), 597–605 (2010)