Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management

ACM Transactions on Computer-Human Interaction

Volumn 14, Issue 4, 2008, Pages

  Bailey, Brian P ^a,b   Iqbal, Shamsi T ^a,b  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b University of Illinois at Urbana Champaign   (United States)

Author keywords

Attention; Interruption; Pupil size; Task models; User studies; Workload

Indexed keywords

COGNITIVE THEORISTS; TASK MODELS; USER STUDIES;

COGNITIVE SYSTEMS; JOB ANALYSIS; MANAGEMENT INFORMATION SYSTEMS;

HUMAN COMPUTER INTERACTION;

EID: 38749142911     PISSN: 10730516     EISSN: 15577325     Source Type: Journal    
DOI: 10.1145/1314683.1314689     Document Type: Article

References (75)

1. ADAMCZYK, P. D. AND BAILEY, B. P. 2004. If not now when? The effects of interruptions at different moments within task execution. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 271-278.
2. ANTTONEN, J. AND SURAKKA, V. 2005. Emotions and heart rate while sitting on a chair. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 491-499.
3. BAILEY, B. P., ADAMCZYK, P. D., CHANG, T. Y., AND CHILSON, N. A. 2006. A framework for specifying and monitoring user tasks. J. Comput. Human Behav. 22, 4, 685-708.
4. BAILEY, B. P. AND KONSTAN, J. A. 2006. On the need for attention aware systems: Measuring effects of interruption on task performance, error rate, and affective state. J. Comput. Human Behav. 22, 4, 709-732.
5. BEATTY, J. 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psych. Bull. 91, 2, 276-292.
6. BRADSHAW, J. L. 1967. Pupil size as a measure of arousal during information processing. Nature 216, 515-516.
7. CARD, S., MORAN, T., AND NEWELL, A. 1983. The psychology of human-computer interaction. Lawrence Erlbaum Associates, Hillsdale.
8. CHEN, D. AND VERTEGAAL, R. 2004. Using mental load for managing interruptions in a physiologically attentive user interface. In Proceedings of ACM Conference on Human Factors in Computing Systems. ACM, New York, 1513-1516.
9. CUTRELL, E., CZERWINSKI, M., AND HORVITZ, E. 2001. Notification, disruption and memory: Effects of messaging interruptions on memory and performance. In Proceedings of the IFIP TC.13 International Conference on Human-Computer Interaction (Tokyo, Japan). 263-269.
10. CZERWINSKI, M., CUTRELL, E., AND HORVITZ, E. 2000a. Instant messaging and interruption: Influence of task type on performance. In Proceedings of the Annual Conference of the Human Factors and Ergonomics Society of Australia (OZCHI) (Sydney, Australia). C. Paris, N. Ozkan, S. Howard and S. Lu, Eds, 356-361.
11. CZERWINSKI, M., CUTRELL, E., AND HORVITZ, E. 2000b. Instant messaging: Effects of relevance and timing. In People and Computers XIV: Proceedings of HCI, S. Turner and P. Turner, Eds, 71-76.
12. DABBISH, L. AND KRAUT, R. E. 2004. Controlling interruptions: Awareness displays and social motivation for coordination. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. ACM, New York, 182-191.
13. DEGANI, A. AND WIENER, E. 1993. Cockpit checklists: Concepts, design, and use. Human Factors 35, 2, 345-359.
14. DEY, A. K. AND ABOWD, G. D. 2000. CybreMinder: A context-aware system for supporting reminders. In Proceedings of 2nd International Symposium on Handheld and Ubiquitous Computing, 172-186.
15. DISMUKES, K., YOUNG, G., AND SUMWALT, R. 1998. Cockpit interruptions and distractions. ASRS Direct. 10.
16. DONCHIN, E., KRAMER, A. F., AND WICKENS, C. D. 1986. Applications of brain event related potentials to problems in engineering psychology. Guildford, New York.
17. FOGARTY, J., KO, A. J., AUNG, H. H., GOLDEN, E., TANG, K. P., AND HUDSON, S. E. 2005. Examining task engagement in sensor-based statistical models of human interruptibility. In Proceeding of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 331-340.
18. GALE, A. AND EDWARDS, J. 1983. The EEG and human behavior. Academic Press, New York.
19. GEVINS, A. AND SCHAFFER, R. 1980. A critical review of electroencephalographic (EEG) correlates of higher cortical functions. CRT Crit. Rev. Bioeng. 4, 113-164.
20. GUERLAIN, S. AND WILLIS, R. 2001. The tactical tomahawk weapons control system: Operator interface design project. In Human Systems Integration Symposium.
21. HART, S. G. AND STAVELAND, L. E. 1988. Development of a multi-dimensional workload rating scale: Results of empirical and theoretical research. In Human Mental Workload P. A. Hancock and N. Meshkati (Eds.), Elsevier, Amsterdam, The Netherlands, pp. 138-183.
22. HESS, E. H. 1972. Pupillometrics: A method of studying mental, emotional and sensory processes. In Handbook of Psychophysiology, N. S. Greenfield and R. A. Sternbach, (Eds.) Holt, Rinehart & Winston, New York, pp. 491-531.
23. HESS, E. H. AND POLT, J. M. 1964. Pupil size in relation to mental activity during simple problem solving. Science 132, 1190-1192.
24. HOECKS, B. AND LEVELT, W. 1993. Pupillary dilation as a measure of attention: A quantitative system analysis. Behav. Res. Meth. Instrum. Comput. 25, 16-26.
25. HORVITZ, E. AND APACIBLE, J. 2003. Learning and reasoning about interruption. In Proceedings of the 5th ACM International Conference on Multimodal Interfaces. ACM, New York, 20-27.
26. HORVITZ, E., JACOBS, A., AND HOVEL, D. 1999. Attention-sensitive alerting. In Conference Proceedings on Uncertainty in Artificial Intelligence, 305-313.
27. HORVITZ, E., KOCH, P., AND APACIBLE, J. 2004. BusyBody: Creating and fielding personalized models of the cost of interruption. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (Chicago, IL). ACM, New York, 507-510.
28. HUDSON, S. E., FOGARTY, J., ATKESON, C. G., AVRAHAMI, D., FORLIZZI, J., KIESLER, S., LEE, J., AND YANG, J. 2003. Predicting human interruptibility with sensors: A wizard of Oz feasibility study. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 257-264.
29. HYTINTK, J., TOMMOLA, J., AND ALAJA, A. 1995. Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. Quart. J. Exper. Psych. 48A, 3, 598-612.
30. IQBAL, S. T., ADAMCZYK, P. D., ZHENG, S., AND BAILEY, B. P. 2005. Towards an index of opportunity: Understanding changes in mental workload during task execution. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 311-320.
31. IQBAL, S. T. AND BAILEY, B. P. 2005. Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 1489-1492.
32. IQBAL, S. T. AND BAILEY, B. P. 2006. Leveraging characteristics of task structure to predict costs of interruption. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 741-750.
33. IQBAL, S. T. AND BAILEY, B. P. 2007. Understanding and developing models for detecting and differentiating breakpoints during interactive tasks. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 697-706.
34. IQBAL, S. T., ZHENG, X. S., AND BAILEY, B. P. 2004. Task evoked pupillary response to mental workload in human-computer interaction. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 1477-1480.
35. JACKSON, T. W., DAWSON, R. J., AND WILSON, D. 2001. The cost of email interruption. J. Syst. Inf. Tech. 5, 1, 81-92.
36. JURIS, M. AND VELDEN, M. 1977. The pupillary response to mental overload. Phys. Psych. 5, 4, 421-424.
37. JUST, M. A., CARPENTER, P. A., AND MIYAKE, A. 2003. Neuroindices of cognitive workload: Neuroimaging, pupillometric, and event-related potential studies of brain work. Theoret. Issues Ergonom. 4, 56-88.
38. KAHNEMAN, D. 1967. Pupillary responses in a pitch-discrimination task. Percept. Psych. 2, 101-105.
39. KAHNEMAN, D. 1973. Attention and Effort. Prentice-Hall, Englewood Cliffs, N.J.
40. KOK, A. 1997. Event-related-potential (ERP) reflections of mental resources: A review and synthesis. Biological Psychology 45, 19-56.
41. KRAMER, A. F. 1991. Physiological metrics of mental workload: A review of recent progress. In Multiple-Task Performance, D. L. Damos, Ed. Taylor and Francis, London, England, pp. 279-328.
42. KRAMER, A. F., SCHNEIDER, W., FISK, A. D., AND DONCHIN, E. 1986. The effects of practice and task structure on components of event related brain potential. Psychophysiology 23, 33-47.
43. KREIFELDT, J. G. AND MCCARTHY, M. E. 1981. Interruption as a test of the user-computer interface. In Proceedings of the 17th Annual Conference on Manual Control, Jet Propulsion Laboratory, California Institute of Technology, JPL Publication 81-95, 655-667.
44. LATORELLA, K. A. 1996. Investigating interruptions: An example from the flight deck. In Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting. 249-253.
45. LATORELLA, K. A. 1998. Effects of modality on interrupted flight deck performance: Implications for data link. In 42nd Annual Meeting of the Human Factors and Ergonomics Society. 87-91.
46. LEE, J. D., HOFFMAN, J. D., AND HAYES, E. 2004. Collision warning design to mitigate driver distraction. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 65-72.
47. LIN, Y., ZHANG, W. J., AND WATSON, L. G. 2003. Using eye movement parameters for evaluating human-machine interface frameworks under normal control operation and fault detection situations. Int. J. Human-Comput. Stud. 59, 6, 837-873.
48. MAGLIO, P. AND CAMPBELL, C. S. 2003. Attentive agents. Commun. ACM 46, 3, 47-51.
49. MARSHALL, S. P. 2002. The index ofs cognitive activity: Measuring cognitive workload. In Proceedings of the 2002 IEEE 7th Conference on Human Factors and Power Plants. IEEE Computer Society Press, Los Alamitos, CA, 7.5-7.9.
50. MCCRICKARD, S., CHEWAR, C. M., SOMERVELL, J. P., AND NDIWALANA, A. 2003. A model for notification systems evaluation-assessing user goals for multitasking activity. ACM Trans. Computer-Hum. Interact. 10, 4, 312-338.
51. MCFARLANE, D. C. 1999. Coordinating the interruption of people in human-computer interaction. In Proceedings of the IFIP TC. 13 International Conference on Human-Computer Interaction. 295-303.
52. MCFARLANE, D. C. AND LATORELLA, K. A. 2002. The scope and importance of human interruption in HCI design. Human-Computer Interaction 17, 1, 1-61.
53. MIYATA, Y. AND NORMAN, D. A. 1986. Psychological issues in support of multiple activities. In User Centered System Design: New Perspectives on Human-Computer Interaction. D. A. Norman and S. W. Draper, Eds. Lawrence Erlbaum Associates, Hillsdale, N.J., pp. 265-284.
54. MONK, C. A., BOEHM-DAVIS, D. A., AND TRAFTON, J. G. 2002. The attentional costs of interrupting task performance at various stages. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting.
55. NAIR, R., VOIDA, S., AND MYNATT, E. 2005. Frequency-based detection of task switches. In Proceedings of the 19th British HCI Group Annual Conference (Edinburgh, Scotland). 94-99.
56. NAKAYAMA, M. AND TAKAHASHI, K. 2002. The act of task difficulty and eye-movement frequency for the oculo-motor indices. In Proceedings of Eye Tracking Research and Applications. 37-42.
57. NEWELL, A. AND ROSENBLOOM, P. S. 1981. Mechanisms of skill acquisition and the law of practice. In Cognitive Skills and their Acquisition, J. R. Anderson, Ed. Erlbaum, Hillsdale, NJ, pp. 1-55.
58. O'DONNELL, R. D. AND EGGEMEIER, F. T. 1986. Workload assessment methodology. In Handbook of Perception and Human Performance. Volume II, Cognitive Processes and Performance, K. R. Boff, L. Kaufman and J. P. Thomas, Eds., Wiley, New York, pp. 42/41-42/49.
59. PHELPS, M. P. AND MAZZIOTTA, J. 1985. Positron emission tomography: Human brain function and biochemistry. Science 228, 799-809.
60. POMPLUN, M. AND SUNKARA, S. 2003. Pupil dilation as an indicator of cognitive workload in human-computer interaction. In Proceedings of the 10th International Conference on Human-Computer Interaction, V. D. D. Harris, M. Smith and C. Stephanidis, Eds.
61. ROWE, D. W., SIBERT, J., AND IRWIN, D. 1998. Heart rate variability: indicator of user state as an aid to human-computer interaction. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 480-487.
62. RUBINSTEIN, J. S., MEYER, D. E., AND EVANS, J. E. 2001. Executive control of cognitive processes in task switching. J. Exper. Psych.: Human Percept. Perform. 27, 4, 763-797.
63. SHNEIDERMAN, B. 1997. Designing the user interface. Pearson Addison Wesley, Reading, MA, Third Edition.
64. SPEIER, C., VALACICH, J. S., AND VESSEY, I. 1999. The influence of task interruption on individual decision making: An information overload perspective. Decis. Sci. 30, 2, 337-360.
65. STANTON, N., ED. 1994. Human Factors in Alarm Design. Taylor and Francis, London, UK.
66. TAKAHASHI, K., NAKAYAMA, M., AND SHIMIZU, Y. 2000. The response of eye-movement and pupil size to audio instruction while viewing a moving target. In Proceedings of the ACM Conference on Eye Tracking Research and Applications. ACM, New York, 131-138.
67. TOBII-SYSTEMS. http://www.tobii.se/.
68. TRAFTON, J. G., ALTMANN, E. M., BROCK, D. P., AND MINTZ, F. E. 2003. Preparing to resume an interrupted task: effects of prospective goal encoding and retrospective rehearsal. Int. J. Human-Comput. Stud. 58, 583-603.
69. VERNEY, S. P., GRANHOLM, E., AND DIONISIO, D. P. 2001. Pupillary responses and processing resources on the visual backward masking task. Psychophysiology 38, 1, 76-83.
70. VERNEY, S. P., GRANOLM, E., AND MARSHALL, S. 2004. Pupillary responses during the visual backward masking task predict cognitive ability. Int. J. Psychophys. 52, 23-36.
71. WICKENS, C. D. 1980. The structure of attentional resources. In Attention and Performance VIII, R. Nickerson, Ed. Lawrence Erlbaum, Hillsdale, NJ, pp. 239-257.
72. WICKENS, C. D. 1991. Processing resources and attention. In Multiple-Task Performance, D. L. Damos, Ed. Taylor & Francis, London, UK, pp. 3-34.
73. WICKENS, C. D. 2002. Multiple resources and performance prediction. Theoret. Iss. Ergon. Sci. 3, 2, 159-177.
74. ZACKS, J., TVERSKY, B., AND IYER, G. 2001. Perceiving, remembering, and communicating structure in events. J. Exper. Psych. General 130, 1, 29-58.
75. ZIJLSTRA, F. R. H., ROE, R. A., LEONORA, A. B., AND KREDIET, I. 1999. Temporal factors in mental work: Effects of interrupted activities. J. Occupat. Org. Psych. 72, 163-185.