Process tracing of complex cognitive work tasks

Journal of Occupational and Organizational Psychology

Volumn 77, Issue 2, 2004, Pages 259-280

  Patrick, John ^a,c   James, Nic ^b  

^a CARDIFF UNIVERSITY   (United Kingdom)

^b SWANSEA UNIVERSITY   (United Kingdom)

^c CARDIFF UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 3042816308     PISSN: 09631798     EISSN: None     Source Type: Journal    
DOI: 10.1348/096317904774202171     Document Type: Article

References (85)

1. Anzai, Y. (1984). Cognitive control of real-time event-driven systems. Cognitive Science, 8, 221-254.
2. Baber, C., & Stanton, N. A. (1994). Task analysis for error identification: A methodology for designing error-tolerant consumer products. Ergonomics, 37, 1923-1941.
3. Bainbridge, L. (1974). The representation of working storage, and its use in the organisation of behaviour. In W. T. Singleton (Ed.), Measurement of human resources (pp. 165-183). London: Taylor & Francis.
4. Bainbridge, L. (1979). Verbal reports as evidence of the process operator's knowledge. International Journal of Man-Machine Studies, 11, 411-436.
5. Bambridge, L. (1987). Ironies of automation. In J. Rasmussen, K. D. Duncan, & J. Leplat (Eds.), New technology and human error (pp. 271-283). Chichester: Wiley.
6. Bainbridge, L. (1990). Verbal protocol analysis. In J. R. Wilson & E. N. Corlett (Eds.), Evaluation of human work: A practical ergonomics methodology (pp. 161-179). London: Taylor & Francis.
7. Bereiter, S. R., & Miller, S. M. (1989). A field based study of troubleshooting in computer-controlled manufacturing systems. IEEE Transactions on Systems, Man and Cybernetics, 19, 205-219.
8. Biehal, G. & Chakravarti, D. (1989). The effects of concurrent verbalisation on choice processing. Journal of Marketing Research, 26, 84-96.
9. Blackman, H. S., & Nelson, W. R. (1988). Techniques for incorporating operator expertise into intelligent decision aids and training. Reliability Engineering and System Safety, 22, 371-385.
10. Brinkman, J. A. (1993). Verbal protocol accuracy in fault diagnosis. Ergonomics, 36, 1381-1397.
11. Card, S. K. (1982). User perceptual mechanisms in the search of computer command menus. Proceedings of Human Factors in Computing Systems 190-196, Gaithersburg, MD: ACM.
12. Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6, 271-315.
13. Cooke, N. J. (1994). Varieties of knowledge elicitation techniques. International Journal of Human Computer Studies, 41, 801-849.
14. Covey, J. A., & Lovie, A. D. (1998). Information selection and utililization in hypothesis testing: A comparison of process-tracing and structural analysis techniques. Organisational Behavior and Human Decision Processes, 75, 56-74.
15. Decortis, F. (1993). Operator strategies in a dynamic environment in relation to an operator model. Ergonomics, 36, 1291-1304.
16. Doherty, M. E. (1993). A laboratory scientist's view of naturalistic decision making. In G. Klein, R. Calderwood & J. Orasanu (Eds.), Decision-making in action: Models and methods (pp. 362-388). New Jersey: Ablex.
17. Droivoldsmo, A., Skraaning, G., Sverrbo, M., Dalen, J., Grimstad, T., & Andresen, G. (1998). Continuous measures of situation awareness and workload. OECD Reactor Project, Halden, Norway. HWR-539.
18. Eekhout, J. M. Van, & Rouse, W. B. (1981). Human errors in detection, diagnosis and compensation for failures in the engine control room of a supertanker. IEEE Transactions on Systems, Man and Cybernetics, 11, 813-816.
19. Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215-251.
20. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (2nd ed.) Cambridge, MA: MIT.
21. Ford, J., Schmitt, N., Schectman, S., Hults, B., & Doherty, M. (1989). Process tracing methods: Contributions, problems and neglected research questions. Organisational Behaviour and Human Decision Processes, 43, 75-117.
22. Fowlkes, J. E., Salas, E., Baker, D. P., Cannon-Bowers, J. A., & Stout, R. J. (2000). The utility of event-based knowledge elicitation. Human Factors, 42, 24-35.
23. Gardner, P. H., Chmiel, N., & Wall, T. D. (1996). Implicit knowledge and fault diagnosis in the control of advanced manufacturing technology. Behaviour and Information Technology, 15, 205-212.
24. Goh, S. K., & Coury, B. G. (1994). Incorporating the effect of display formats in cognitive modelling. Festschrift in honour of Professor E. Nigel Corlett. Special Issue, Ergonomics, 37, 725-745.
25. Graf, W., Sigl, van der Heiden, & Kreuger, H. (1987). The applicability of eye movement analysis in the ergonomic evaluation of human-computer interaction. Proceedings of work with display units, (pp. 512-515) Stockholm, Sweden.
26. Hansen, J. P. (1991). The use of eye mark recordings to support verbal retrospection in software testing. Acta Psychologica, 76, 31-49.
27. Harvey, R. J. (1991). Job analysis. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., pp. 71-163) Palo Alto, CA: Consulting Psychologists Press.
28. Hasselbrock, F., & Prietula, M. J. (1992). A protocol-based coding scheme for the analysis of medical reasoning. International Journal of Man-Machine Studies, 37, 613-652.
29. Hella, F. (1987). Is the analysis of the eye movement recording a sufficient criterion for evaluating automobile instrument panel design? In J. K. O'Reagan & A. Levy-Schoen (Eds.), Eye movements: From physiology to cognition (pp. 555-561). Amsterdam: North Holland.
30. Hoc, J. M. (1989). Strategies in controlling a continuous process with long response latencies: Needs for computer support to diagnosis. International Journal of Man-Machine Studies, 30, 47-67.
31. Hoc, J. M., & Leplat, J. (1983). Evaluation of different modalities of verbalisation In a sorting task. International Journal of Man-Machine Studies, 18, 283-306.
32. Hoffman, R. R., Shadbolt, N. R., Burton, A. M., & Klein, G. (1995). Eliciting knowledge from experts: A methodological analysis. Organisational Behaviour and Human Decision Processes, 62, 129-158.
33. Hoffman, R. R., Crandall, B., & Shadbolt, N. (1998). Use of the Critical Decision Method to elicit expert knowledge: A case study in the methodology of cognitive task analysis. Human Factors, 40, 254-276.
34. Hollnagel, E., Pedersen, O. M., & Rasmussen, J. (1981). Notes on human performance analysis. Riso-M-2285, Riso National Laboratory, Roskilde, Denmark.
35. Holmström, C. B. O., Volden, F. S., & Endestad, T. (1993). Continued experimental evaluations of a diagnostic rule-based expert system for the nuclear industry. OECD Reactor Project, Halden, Norway.
36. Kaplan, C. A., & Schoenfeld, W. N. (1966). Oculomotor patterns during the solution of visually displayed anagrams. Journal of Experimental Psychology, 72, 447-451.
37. Kirschenbaum, S. S. (1992). Influence of experience on information gathering strategies. Journal of Applied Psychology, 77, 343-352.
38. Kirwan, B., & Ainsworth, L. K. (1992). A guide to ask analysis. London: Taylor & Francis.
39. Klein, G. A. (2000). Analysis of situation awareness from critical incident reports. In M. R. Endsley & D. J. Garland (Eds.), Situation awareness analysis and measurement (pp. 51-72). Mahwah, NJ: Erlbaum.
40. Klein, G. A., Calderwood, R., & MacGregor, D. (1989). Critical decision making for eliciting knowledge. IEEE Transactions of Systems, Man, and Cybernetics, 19, 462-472.
41. Kurland, L. C., & Tenney, Y. J. (1988). Issues in developing an intelligent tutor for a real-world domain: Training in radar mechanics. In J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned (pp. 119-180). Hillsdale, NJ: Erlbaum.
42. Leplat, J. (1987). Methodology of task analysis and design. Paper presented to European Methodologies in Work and Organisational Psychology Symposium, May, Hungary.
43. Leplat, J., & Hoc, J. M. (1981). Subsequent verbalisation in the study of cognitive processes. Ergonomics, 24, 745-755.
44. Long, J. B. (1986). People and computers: designing for usability. In M. D. Harrison & A. F. Monk (Eds.), People and computers: Designing for usability. Proceedings of HCI 1986. Cambridge: Cambridge University Press.
45. Mao, J-Y., & Benbasat, I. (1998). Contextualised access to knowledge: Theoretical perspectives and a process-tracing study. Information Systems Journal, 8, 217-239.
46. Marmaras, N., Lioukas, S., & Laios, L. (1992). Identifying competences for the design of systems supporting complex decision-making tasks: A managerial planning application. Ergonomics, 35, 1221-1241.
47. Militello, L. G., & Hutton, R. J. B. (1998). Applied cognitive task analysis (ACTA): A practitioner's toolkit for understanding cognitive demands. Ergonomics, 41, 1618-1641.
48. Moray, N. (1998). Identifying mental models of complex human-machine systems. International Journal of Industrial Ergonomics, 22, 293-297.
49. Moray, N., & Rotenberg, I. (1989). Fault management in process control: Eye movements and action. Ergonomics, 32, 1319-1342.
50. Morgeson, P. F., & Campion, M. A. (2000). Accuracy in job analysis: toward an inference-based model. Journal of Organizational Behavior, 21, 819-827.
51. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
52. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we know: Verbal reports on mental processes. Psychological Review, 84, 232-259.
53. O'Hare, D., Wiggins, M., Williams, A., & Wong, W. (1998). Cognitive task analyses for decision centred design and training. Ergonomics, 41, 11, 1698-1718.
54. Olson, J. R., & Biolsi, K. J. (1991). Techniques for representing expert knowledge. In K. A. Ericsson & J. Smith (Eds.), Towards a general theory of expertise (pp. 240-285). Cambridge: Cambridge University Press.
55. Papin, J. M., Metges, P., & Amalberti, R. (1984). Use of NAC eye work by radiologists. In A. Gale and F. Johnson (Eds.), Theoretical and applied aspects of eye movement research (pp. 323-330). Amsterdam: North Holland.
56. Patel, V. L., & Groen, G. J. (1986). Knowledge based solution strategies in medical reasoning. Cognitive Science, 10, 91-116.
57. Patrick, J., James, N., & Friend, C. (1996). A field study of training fault-finding. Le Travail Humain, 59, 23-44.
58. Patrick, J., Gregov, A., Halliday, P., Handley, J., & O'Reilly, S. (1999). Analysing operators' diagnostic reasoning during multiple events. Ergonomics, 42, 493-515.
59. Patrick, J., Grainger, L., Gregov, A., Halliday, P., Handley, J., & James, N. (1999). Training to break the barriers of habit in reasoning about unusual faults. Journal of Experimental Psychology: Applied, 5, 314-335.
60. Pew, R. W., Miller, D. C., & Feehrer, C. E. (1981). Evaluation of proposed control room improvements through analysis of critical operator decisions. EPRI NP-1982. Palo Alto, CA: Electrical Power Research Institute.
61. Pidgeon, N. F., Turner, B. A., & Blockley, D. I. (1991). The use of grounded theory for conceptual analysis in knowledge elicitation. International Journal of Man-Machine Studies, 35, 151-173.
62. Pinsky, L., & Theureu, J. (1987). Description of visual 'action' in 'natural' situations. In J. K. O'Reagan & A. Levy-Schoen (Eds.), Eye movements: From physiology to cognition (pp. 593-602). Amsterdam: North Holland.
63. Rasmussen, J. (1980). The human as a system component. In H. T. Smith & T. R. G. Green (Eds). Human interaction with computers (pp. 67-96). London: Academic Press.
64. Reed, N. E., & Johnson, P. E. (1993). Analysis of expert decision making in hardware diagnosis. International Journal of Man-machine Studies, 38, 251-280.
65. Reinartz, S. J. (1993). An empirical study of team behaviour in a complex and dynamic problem-solving context: A discussion of methodological and analytical aspects. Ergonomics, 36, 1281-1290.
66. Roth, B. (1966). The effect of overt verbalisation of problem solving. Dissertation Abstracts International, 27, (3-B) 957.
67. Roth, E., Bennett, K., & Woods, D. (1987). Human interaction with an 'intelligent' machine. International Journal of Man-Machine Studies, 27, 479-525, Special Issue.
68. Roth, E., Woods, D. D., & Pople, H. E. (1992). Cognitive simulation as a tool for cognitive task analysis. Ergonomics, 35, 1168-1198.
69. Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols. Memory and Cognition, 17, 759-769.
70. Sanchez, J. I., & Levine, E. L. (2000). Accuracy or consequential validity: Which is the better standard for job analysis data? Journal of Organizational Behavior, 21, 809-818.
71. Sanderson, P. M., Verhage, A. G., & Fuld, R. B. (1989). State-space and verbal protocol methods for studying the human operator in process control. Ergonomics, 32, 1343-1372.
72. Schaafstal, A. M. (1991). Diagnostic skill in process operation: A comparison between experts and novices. PhD thesis, Institute for Perception, TNO, Soesterberg.
73. Schraagen, J. M., Chipman, S. F., & Shalin, V. L. (Eds.) (2000). Cognitive task analysis. Mahwah, NJ: Erlbaum.
74. Schraagen, J. M. C., & Schaafstal, A. M. (1996). Training of systematic diagnosis: A case study in electronics troubleshooting. Le Travail Humain, 59, 5-21.
75. Schweiger, A. M. (1983). Is the simultaneous verbal protocol a viable method for studying managerial problem-solving and decision making? Academy of Management Journal, 26, 185-192.
76. Smolensky, M. W. (1993). Toward the physiological measurement of situation awareness. Proceedings of the Human Factors and Ergonomics Society, 37th Annual Meeting, Seattle, Washington pp. 41.
77. Stammers, R. B. (1995). Factors limiting the development of task analysis. Ergonomics, 38, 588-594.
78. Svenson, O. (1979). Process description of decision making. Organisational Behaviour and Human Performance, 23, 86-112.
79. Tenney, Y. J., & Kurland, L. C. (1988). The development of troubleshooting expertise in radar mechanics. In J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned (pp. 59-83). Hillsdale, NJ: Erlbaum.
80. Umbers, I. G. (1979). A study of the control skills of gas grid control engineers. Ergonomics, 22, 557-571.
81. Umbers, I. G. (1981). A study of control skills in an industrial task, and in a simulation, using the verbal protocol technique. Ergonomics, 24, 275-293.
82. Vicente, K. J. (1999). Cognitive work analysis: Toward safe, productive and healthy computer-based work. Mahwah, NJ: Erlbaum.
83. Winikoff, A. (1967). Eye movements as an aid to protocol analysis of problem solving behavior. Doctoral dissertation, Carnegie-Mellon University.
84. Woods, D. D. (1992). Process tracing methods for the study of cognition outside of the experimental psychology laboratory. In G. Klein, R. Calderwood, & J. Orasanu (Eds.), Decision-making in action: Models and methods (pp. 1-26). New Jersey: Ablex.
85. Wright, P. C., & Monk, A. F. (1989). Evaluation for design. In A. Sutcliffe & L. McCauley (Eds.), People and computers 5 (pp. 345-358). Cambridge: Cambridge University Press.