Neuroindices of cognitive workload: Neuroimaging, pupillometric and event-related potential studies of brain work

Theoretical Issues in Ergonomics Science

Volumn 4, Issue 1-2, 2003, Pages 59-88

  Just, Marcel Adam ^a   Carpenter, Patricia A ^a   Miyake, Akira ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF COLORADO   (United States)

Author keywords

Brain imaging; Cognitive modeling; ERP; FMRI; Mental rotation; Pupillary dilation

Indexed keywords

EID: 84987941700     PISSN: 1463922X     EISSN: 1464536X     Source Type: Journal    
DOI: 10.1080/14639220210159735     Document Type: Article

References (91)

1. Alivisatos, B. and Petrides, M. 1997, Functional activation of the human brain during mental rotation, Neuropsychologia, 35, 111-118.
2. Baker, S. C., Rogers, R. D., Owen A. M., Frith, C. D., Dolan, R. J., Frackowiak, R. S. J. and Robbins, T. W. 1996, Neural systems engaged by planning: a PET study of the Tower of London task, Neuropsychologia, 34, 515-526.
3. Beatty, J. 1982, Task-evoked pupillary responses, processing load, and the structure of processing resources, Psychological Bulletin, 91, 276-292.
4. Ben-Nun, Y. 1986, The use of pupillometry in the study of on-line verbal processing: evidence for depths of processing, Brain and Language, 28, 1-11.
5. Binder, J. R., Rao, S. M., Hammeke, T. A., Yetkin, F. Z., Jesmanowicz, A., Bandettini, P. A., Wong, E. C., Estkowski, L. D., Goldstein, M. D., Haughton, V. M. and Hyde, J. S. 1994, Functional magnetic resonance imaging of human auditory cortex, Annals of Neurology, 35, 662-672.
6. BOTTINI, G., CORCORAN, R., STERZI, R., PAULESU, E., SCHENONE, P., SCAPRA, P., FRACKOWIAK, R. S. J. and FRITH, C. D. 1994, The role of the right hemisphere in the interpretation of figurative aspects of language: a positron emission tomography activation study, Brain, 117, 1241-1253.
7. BRAVER, T. S., COHEN, J. D., NYSTROM, L. E., JONIDES, J., SMITH, E. E. and NOLL, D. C., 1997, A parametric study of prefrontal cortex involvement in human working memory, NeuroImage, 5, 49-62.
8. BYRNE, M. D. and BOVAIR, S. 1997, A working memory model of a common procedural error, Cognitive Science, 21, 31-61.
9. CARPENTER, P. A. and EISENBERG, P. 1978, Mental rotation and the frame of reference in blind and sighted individuals, Perception and Psychophysics, 23, 117-124.
10. CARPENTER, P. A. and JUST, M. A. 1999, Computational modeling of high-level cognition, in R. J. Sternberg (ed.), The concept of cognition (Cambridge, MA: MIT Press), 245-293.
11. CARPENTER, P. A., JUST, M. A., KELLER, T., EDDY, W. F. and THULBORN, K. R. 1999, Graded functional activation in the visuospatial system with the amount of task demand, Journal of Cognitive Neuroscience, 11, 9-24.
12. CARPENTER, P. A., JUST, M. A. and SHELL, P. 1990, What one intelligence test measures: a theoretical account of the processing in the Raven Progressive Matrices Test, Psychological Review, 97, 404-431.
13. CARROLL, J. B. 1993, Human cognitive abilities: A survey offactor-analysis studies (New York: Cambridge University Press).
14. CHARLOT, V., Tzourio, N., ZILBOVICIUS, M., MAZOYER, B. and DENIS, M. 1992, Different mental imagery abilities result in different regional cerebral blood flow activation patterns during cognitive tasks, Neuropsychologia, 30, 565-580.
15. COHEN, M. S., KOSSLYN, S. M., BREITER, H. C., DIGIROLAMO, G. J., THOMPSON, W. L., ANDERSON, A. K., BOOKHEIMER, S. Y., ROSEN, B. R. and BELLIVEAU, J. W. 1996, Changes in cortical activity during mental rotation: A mapping study using functional MRI, Brain, 119, 89-100.
16. DANEMAN, M. and CARPENTER, P. A. 1980, Individual differences in working memory and reading, Journal of Verbal Learning and Verbal Behavior, 19, 450-466.
17. D’ESPOSITO, M., DETRE, J. A., ALSOP, D. C., SHIN, R. K., ATLAS, S. and GROSSMAN, M. 1995, The neural basis of the central executive system of working memory, Nature, 378, 279-281.
18. DESROCHER, M. E., SMITH, M. L. and TAYLOR, M. J. 1995, Stimulus and sex differences in performance of mental rotation: evidence from event-related potentials, Brain and Cognition, 28, 14-38.
19. DUNCAN, J., EMSLIE, H., WILLIAMS, P., JOHNSON, R. and FREER, C. 1996, Intelligence and the frontal lobe: the organization of the goal-directed behavior, Cognitive Psychology, 30, 257-303.
20. DUNCAN, J., SEITZ, R. J., KOLODNY, J., Bor, D., HERZOG, H., AHMED, A., NEWELL, F. N. and EMSLIE, H. 2000, A neural basis for general intelligence, Nature, 289, 457-460.
21. EMERSON, M. J., MIYAKE, A. and RETTINGER, D. A. 1999, Individual differences in integrating and coordinating multiple sources of information, Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 1300-1321.
22. FRIEDMAN, H. R. and GOLDMAN-RAKIC, P. S. 1994, Coactivation of prefrontal cortex and inferior parietal cortex in working memory tasks revealed by 2DG functional mapping in the Rhesus monkey, Journal of Neuroscience, 14, 2775-2788.
23. FRIEDMAN, N. P. and MIYAKE, A. 2000, Differential roles for visuospatial and verbal working memory in situation model construction, Journal of Experimental Psychology: General, 129, 61-83.
24. FUSTER, J. M. 1997, The prefrontal cortex: anatomy, physiology, and neuropsychology of the frontal lobe, 3rd edn (Philadelphia: Lippincott-Raven).
25. GOLDMAN, S. R. and VARMA, S. 1995, CAPing the construction-integration model of discourse comprehension, in C. A. Weaver, S. Mannes and C. R. Fletcher (eds), Discourse com-prehension: Essays in honor of Walter Kintsch (Hillsdale, NJ: Erlbaum), 337-358.
26. GRASBY, P. M., FRITH, C. D., FRISTON, K. J., BENCH, C., FRACKOWIAK, R. S. J. and DOLAN, R. J. 1993, Functional mapping of brain areas implicated in auditory— verbal memory function, Brain, 116, 1-20.
27. Grasby, P. M., Frith, C. D., Friston, K. J., Simpson, J., Fletcher, P. C., Frackowiak, R. S. J. and Dolan, R. J. 1994, A graded task approach to the functional mapping of brain areas implicated in auditory-verbal memory, Brain, 117, 1271-1282.
28. Gunter, T. C., Jackson, J. L. and Mulder, G. 1995, Language, memory, and aging: an electrophysiological exploration of the N400 during reading of memory-demanding sentences, Psychophysiology, 32, 215-229.
29. Haarmann, H., Just, M. A. and Carpenter, P. A. 1997, Aphasic sentence comprehension as a resource deficit: a computational approach, Brain and Language, 59, 76-120.
30. Haier, R. J. 1993, Cerebral glucose metabolism and intelligence, in P. A. Vernon (ed.), Biological approaches to the study of human intelligence (Norwood, NJ: Ablex), 317-332.
31. Haier, R. J., Siegel, B. V. Jr., Nuechterlein, K. H., Hazlett, E., Wu, J. C., Paek, J., Browning, H. L. and Buchsbaum, M. S. 1988, Cortical glucose metabolic rate correlates of abstract reasoning and attention studied with positron emission tomography, Intelligence, 12, 199-217.
32. Haier, R. J., Siegel, B., Tang, C., Abel, L. and Buchsbaum, M. S. 1992, Intelligence and changes in regional cerebral glucose metabolic rate following learning, Intelligence, 16, 415-426.
33. Haxby, J. V., Grady, C. L., Ungerleider, L. G. and Horwitz, B. 1991, Mapping the functional neuroanatomy of the intact human brain with brain work imaging, Neuropsychologia, 29, 539-555.
34. Howard, D., Patterson, K., Wise, R., Brown, W. D., Friston, K., Weiller, C. and Frackowiak, R. 1992, The cortical localization of the lexicons: positron emission tomography evidence, Brain, 115, 1769-1782.
35. Huguenard, B. R., Lerch, F. J., Junker, B. W., Patz, R. J. and Kass, R. E. 1997, Working memory failure in phone-based interaction, ACM Transactions on Computer-Human Interaction, 4, 67-102.
36. Isreal, J. B., Wickens, C. D., Chesney, G. L. and Donchin, E. 1980, The event-related brain potential as an index of display-monitoring workload, Human Factors, 22, 211-224.
37. Janisse, M. P. 1977, Pupillometry: The psychology of the pupillary response (Washington, DC: Hemisphere).
38. Just, M. A. and Carpenter, P. A. 1985, Cognitive coordinate systems: accounts of mental rotation and individual differences in spatial ability, Psychological Review, 92, 137-172.
39. Just, M. A. and Carpenter, P. A. 1992, A capacity theory of comprehension: individual differences in working memory, Psychological Review, 99, 122-149.
40. Just, M. A. and Carpenter, P. A. 1993, The intensity of thought: pupillometric indices of sentence processing, Canadian Journal of Experimental Psychology, 47, 310-339.
41. JUST, M. A. and CARPENTER, P. A. 1995, Spatial working memory: cognitive capacity in mental rotation, Unpublished Manuscript. Department of Psychology, Carnegie Mellon University, Pittsburgh, PA.
42. Just, M. A. and Varma, S. 2002. A hybrid architecture for working memory: reply to MacDonald and Christiansen (2002), Psychological Review, 109, 55-65.
43. Just, M. A., Carpenter, P. A. and Hemphill, D. D. 1996a, Constraints on processing capacity: architectural or implementational?, in D. Steier and T. Mitchell (eds), Mind matters: A tribute to Allen Newell (Mahwah, NJ: Erlbaum), 141-178.
44. Just, M. A., Carpenter, P. A. and Keller, T. A. 1996b, The capacity theory of comprehension: new frontiers of evidence and arguments, Psychological Review, 103, 773-780.
45. Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F. and Thulborn, K. R. 1996c, Brain activation modulated by sentence comprehension, Science, 274, 114-116.
46. Just, M. A., Carpenter, P. A., Keller, T. A., Emery, L., Zajac, H. and Thulborn, K. R. 2001, Interdependence of non-overlapping cortical systems in dual cognitive tasks, Neuroimage, 14, 417-426.
47. Just, M. A., Carpenter, P. A. and Varma, S. 1999, Computational modeling of high-level cognition and brain function, Human Brain Mapping, 8, 128-136.
48. Kahneman, D. 1973, Attention and effort (Englewood Cliffs, NJ: Prentice Hall).
49. King, J. and Just, M. A. 1991, Individual differences in syntactic processing: the role of working memory, Journal of Memory and Language, 30, 580-602.
50. King, J. and Kutas, M. 1995, Who did what and when? Using work- and clause-level ERPs to monitor working memory usage in reading, Journal of Cognitive Neuroscience, 7, 376-395.
51. Kutas, M. and King, J. 1996, The potentials for basic sentence processing: differentiating integrative processes, in T. Inui and J. L. McClelland (eds), Attention and performance XVI: Information integration in perception and communication (Camgridge, MA: MIT Press), 501-546.
52. MacDonald, M. C., Just, M. and Carpenter, P. 1992, Working memory constraints on the processing of syntactic ambiguity, Cognitive Psychology, 24, 56-98.
53. Martin, J. H. 1991, Coding and processing of sensory information, in E. R. Kandel, J. H. Schwartz and T. M. Jessell (eds), Principles of neural science, 3rd edn (New York: Elsevier), 329-340.
54. Mecklinger, A., Schriefers, H., Steinhauer, K. and Friederici, A. D. 1995, Processing relative clauses varying on syntactic and semantic dimensions: an analysis with event- related potentials, Memory and Cognition, 23, 477-494.
55. Mesulam, M.-M. 1990, Large-scale neurocognitive networks and distributed processing for attention, language and memory, Annals of Neurology, 28, 597-613.
56. Metter, E. J. and Hanson, W. R. 1994, Use of positron emission tomography to study aphasia, in A. Kertesz (ed.), Localization and neuroimaging in neurospychology (San Diego, CA: Academic Press), 123-149.
57. Michel, C. M., Kaufman, L. and Williamson, S. J. 1994, Duration of EEG and MEG a suppression increases with angle in a mental rotation task, Journal of Cognitive Neuroscience, 6, 139-150.
58. Miyake, A. and Shah, P. 1999, Models of working memory: Mechanisms of active maintenance and executive control (New York: Cambridge University Press).
59. Miyake, A., Carpenter, P. A. and Just, M. A. 1994a, A capacity approach to syntactic comprehension disorders: making normal adults perform like aphasic patients, Cognitive Neuropsychology, 11, 671-717.
60. Miyake, A., Just, M. A. and Carpenter, P. A. 1994b, Working memory constraints on the resolution of lexical ambiguity: maintaining multiple interpretations in neutral contexts, Journal of Memory and Language, 33, 175-202.
61. Navon, D. 1984, Resources—a theoretical soup stone?, Psychological Review, 91, 216-234.
62. Navon, D. and Gopher, D. 1979, On the economy of the human processing system, Psychological Review, 86, 214-255.
63. Norman, D. A. and Bobrow, D. G. 1975, On data-limited and resource-limited processes, Cognitive Psychology, 7, 44-64.
64. Ojemann, G. A. 1991, Cortical organization of language, Journal of Neuroscience, 11, 2281-2287.
65. Parasuraman, R. 1985, Event-related brain potentials and intermodal divided attention, Proceedings of the Human Factors Society, 29, 971-975.
66. Parasuraman, R. 1990, Event-related brain potentials and human factors research, in J. W. Rohrbaugh, R. Parasuraman and R. Johnson (eds), Event-related brain potentials: Basic issues and applications (New York, NY: Oxford University Press), 279-300.
67. Parks, R. W., Crockett, D. J., Tuokko, H., Beattie, B. L., Ashford, J. W., Coburn, K. L., Zec, R. F., Becker, R. E., McGeer, P. L. and McGeer, E. G. 1989, Neuropsychological ‘systems efficiency’ and positron emission tomography, Journal of Neuropsychiatry, 1, 269-282.
68. Parks, R. W., Lowenstein, D. A., Dodrill, K. L., Barker, W. W., Yoshii, F., Chang, J. Y., Emran, A., Apicella, A., Sheramata, W. A. and Duara, R. 1988, Cerebral metabolic effects of a verbal fluency test: a PET scan study, Journal of Clinical and Experimental Neuropsychology, 10, 565-575.
69. Peronnet, F. and Farah, M. J. 1989, Mental rotation: an event-related potential study with a validated mental rotation task, Brain and Cognition, 9, 279-288.
70. Petrides, M. 1995, Impairments on nonspatial self-ordered and externally ordered working memory tasks after lesion of the mid-dorsal part of the lateral frontal cortex in the monkey, Journal of Neuroscience, 15, 359-375.
71. Raichle, M. E., Fiez, J. A., Videen, T. O., MacLeod, A. K., Pardo, J. V., Fox, P. T. and Petersen, S. E. 1994, Practice-related changes in human brain functional anatomy during nonmotor learning, Cerebral Cortex, 4, 8-26.
72. Raney, G. E. 1993, Monitoring changes in cognitive load during reading: an event-related brain potential and reaction time analysis, Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 51-69.
73. Reid, G. B. and Nygren, T. E. 1988, The subjective workload assessment technique: A scaling procedure for measuring mental workload, in P. A. Hancock and N. Meshkati (eds), Human mental workload (Amsterdam: Elsevier), 185-218.
74. Roberts, R. J., Hager, L. D. and Heron, C. 1994, Prefrontal cognitive processes: working memory and inhibition in the antisaccade task, Journal of Experimental Psychology: General, 123, 374-393.
75. Rogers, R. D. and Monsell, S. 1995, Costs of a predictable switch between simple cognitive tasks, Journal of Experimental Psychology: General, 124, 207-231.
76. Rosler, F., Heil, M., Bajric, J., Pauls, A. C. and Henninghausen, E. 1995, Patterns of cerebral activation while mental images are rotated and changed in size, Psychophysiology, 32, 135-149.
77. Schluroff, M. 1982, Pupil responses to grammatical complexity of sentences, Brain and Language, 17, 133-145.
78. Schluroff, M., Zimmermann, T. E., Freeman, R. B., Hofmeister, K., Lorscheid, T. and Weber, A. 1986, Pupillary responses to syntactic ambiguity of sentences, Brain and Language, 27, 322-344.
79. SERGENT, J. 1994, Brain imaging studies of cognitive functions, Trends in Neuroscience, 17, 221-227.
80. Shah, P. and Miyake, A. 1996, The separability of working memory resources for spatial thinking and language processing: an individual differences approach, Journal of Experimental Psychology: General, 125, 4-27.
81. Shallice, T. 1988, From neuropsychology to mental structure (New York: Cambridge University Press).
82. Siegler, R. S. and Shipley, C. 1995, Variation, selection, and cognitive change, in T. Simon and G. Halford (eds), Developing cognitive competence: New approaches to process modeling (Hillsdale, NJ:Erlbaum), 31-76.
83. Snow, R. E., Kyllonen, P. C. and Marshalek, B. 1984, The topography of ability and learning correlations, in R. J. Sternberg (ed.), Advances in the psychology of human intelligence, Vol. 2 (Hillsdale, NJ: Erlbaum), 47-103.
84. St. George, M., Mannes, S. and Hoffman, J. E. 1994, Global semantic expectancy and language comprehension, Journal of Cognitive Neuroscience, 6, 70-83.
85. Stanners, R. F., Headley, D. B. and Clark, W. R. 1972, The pupillary response to sentences: influences of listening set and deep structure, Journal of Verbal Learning and Verbal Behavior, 11, 257-263.
86. Stowe, L. A., Wijers, A. A., Willensen, A., Reuland, E., Paans, A. M. J. and Vaalburg, W. 1994, PET studies of language: an assessment of the reliability of the technique, Journal of Psycholinguistic Research, 23, 499-527.
87. Tompkins, C. A. and Mateer, C. A. 1985, Right hemisphere appreciation of prosodic and linguistic indications of implicit attitude, Brain and Language. 24, 185-203.
88. Ungerleider, L. G. and Haxby, J. V. 1994, ‘What’ and ‘where’ in the human brain, Current Opinion in Neurobiology, 4, 157-165.
89. Ungerleider, L. G. and Mishkin, M. 1982, Two cortical visual systems, in D. J. Ingle, M. A. Goodale and R. J. W. Mansfield (eds), Analysis of visual behavior (Cambridge, MA: MIT Press), 549-586.
90. Wickens, C. D. 1984, Processing resources in attention, in R. Parasuraman, and D. R. Davies (eds), Varieties of attention (New York: Academic Press), 63-102.
91. Wijers, A. A., Otten, L. J., Feenstra, S., Mulder, G. and Mulder, L. J. M. 1989, Brain potentials during selective attention, memory search, and mental rotation, Psychophysiology, 26, 452-467.