The automation-by-expertise-by-training interaction: Why automation-related accidents continue to occur in sociotechnical systems

Human Factors

Volumn 59, Issue 2, 2017, Pages 204-228

  Strauch, Barry ^a  

^a NATIONAL TRANSPORTATION SAFETY BOARD   (United States)

Author keywords

Accident analysis; Expert novice differences; Human error analysis; Sociotechnical systems

Indexed keywords

ACCIDENTS; ERRORS; HUMAN COMPUTER INTERACTION; PERSONNEL TRAINING;

ACCIDENT ANALYSIS; ACCIDENT INVESTIGATION; EXPERT-NOVICE DIFFERENCES; HUMAN ERROR ANALYSIS; OPERATOR PERFORMANCE; REPRESENTATIVE SAMPLE; SITUATION AWARENESS; SOCIOTECHNICAL SYSTEMS;

AUTOMATION;

ACCIDENT; AUTOMATION; AWARENESS; ERROR; ETHNOGRAPHIC RESEARCH; HUMAN; HUMAN COMPUTER INTERACTION; IN SERVICE TRAINING; MAN MACHINE INTERACTION;

AUTOMATION; HUMANS; INSERVICE TRAINING; MAN-MACHINE SYSTEMS;

EID: 85018179846     PISSN: 00187208     EISSN: 15478181     Source Type: Journal    
DOI: 10.1177/0018720816665459     Document Type: Article

References (94)

1. Accident Investigation Commission. (1996). Informe finale, accidente de la aeronave Boeing 757-200, operada por La Empresa de Transporte Aereo Del Peru S. A., Aeroperu, ocurrido el dia 02 de Octubre de 1996 [Final report, accident of the Boeing 757-200, operated by the transport company AeroPeru, on October 2, 1996]. Lima, Peru: Author.
2. Aeronautica Civil of the Government of Colombia. (1996). Aircraft accident report, controlled flight into terrain, American Airlines Flight 965, Boeing 757-223, N651AA, near Cali, Colombia, December 20, 1995. Bogotá, Colombia: Author.
3. Amalberti, R., & Wilbaux, F. (1994). Advanced automated glass cockpit certification: Being wary of human factors. In J. A. Wise, V. D. Hopkin, & D. J. Garland (Eds.), Human factors certification of advanced aviation technologies (pp. 308-319). Daytona Beach, FL: Embry-Riddle Aeronautical University Press.
4. Bainbridge, L. (1983). Ironies of automation. Automatica, 19, 775-779.
5. Bauer, J. S., Newman, M. W., & Kientz, J. A. (2014). What designers talk about when they talk about context. Human-Computer Interaction, 29, 420-450.
6. Birney, D. P., Beckmann, J. F., & Wood, R. E. (2012). Precursors to the development of flexible expertise: Metacognitive selfevaluations as antecedences and consequences in adult learning. Learning and Individual Differences, 22, 563-574.
7. Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
8. Bureau d'Enquêtes et d'Analyses pour la Sécurité de l'Aviation Civile. (2012). Final report on the accident on 1st June 2009, to the Airbus A330-203, registered F-GZCP, operated by Air France, Flight AF 447 Rio de Janeiro-Paris. Paris, France: Author.
9. Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Converse, S. A. (1999). Toward an integration of training theory and technique. Human Factors, 33, 281-192.
10. Casner, S. M., Geven, R. W., Recker, M. P., & Schooler, J. W. (2014). The retention of manual flying skills in the automated cockpit. Human Factors, 56, 1506-1516.
11. Casner, S. M., Geven, R. W., & Williams, K. T. (2013). The effectiveness of airline pilot training for abnormal events. Human Factors, 55, 477-485.
12. Cellier, J. M., Eyrolle, H., & Marine, C. (1997). Expertise in dynamic environments. Ergonomics, 40, 28-50.
13. Charness, N., & Tuffiash, M. (2008). The role of expertise research and human factors in capturing, explaining, and producing superior performance. Human Factors, 50, 427-432.
14. Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4, 55-61.
15. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1982). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152.
16. Christoffersen, K., & Woods, D. D. (2002). How to make automated systems team players. In E. Salas (Ed.), Advances in human performance and cognitive engineering research (pp. 1-13). Bingley, UK: Emerald.
17. Coury, B. G., Ellingstad, V. S., & Kolly, J. M. (2010). Transportation accident investigation: The development of human factors research and practice. Reviews of Human Factors and Ergonomics, 6, 1-33.
18. Day, S. B., & Goldstone, R. L. (2012). The import of knowledge export: Connecting findings and theories of transfer of learning. Educational Psychologist, 47, 153-176.
19. Degani, A., Barshi, I., & Shafto, M. G. (2013). Information organization in the airline cockpit: Lessons from Flight 236. Journal of Cognitive Engineering and Decision Making, 7, 330-352.
20. Degani, A., & Heymann, M. (2002). Formal verification of humanautomation interaction. Human Factors, 44, 28-43.
21. Durso, F. T., & Dattel, A. R. (2006). Expertise and transportation. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman. (Eds.), The Cambridge handbook of expertise and expert performance (pp. 355-372). Cambridge, UK: Cambridge University Press.
22. Dutch Safety Board. (2010). Crashed during approach, Boeing 737-800, near Amsterdam Schiphol Airport, 25 February 2009. The Hague, Netherlands: Author.
23. Dutch Safety Board. (2016). Hard landing during automation approach at Amsterdam Airport Schiphol. The Hague, Netherlands: Author.
24. Ehrenstein, A., Walker, B. N., Czerwinski, M., & Feldman, E. M. (1994). Some fundamentals of training and transfer: Practice benefits are not automatic. In M. A Quinones & A. Ehrenstein (Eds.), Training for a rapidly changing workplace: Applications of psychological research (pp. 119-147). Washington, DC: American Psychological Association.
25. Endsley, M. R. (1995). Toward a theory of situation awareness. Human Factors, 37, 32-64.
26. Endsley, M. R. (2006). Expertise and situation awareness. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 633-652). Cambridge, UK: Cambridge University Press.
27. Endsley, M. R., & Kaber, D. B. (1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics, 42, 462-492.
28. Endsley, M. R., & Kiris, E. O. (1995). The out-of-The-loop performance problem and level of control in automation. Human Factors, 37, 381-394.
29. Ericsson, K. (2006). The influence of experience and deliberate practice on the development of superior expert performance. In K. A. Ericsson, N. Charness, P. J. Feltovich, & Robert R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 685-705). Cambridge, UK: Cambridge University Press.
30. Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49, 725-747.
31. Ericsson, K. A., Prietula, M. J., & Cokely, E. T. (2007). The making of an expert. Harvard Business Review, 85, 114-121.
32. Fadde, P. J. (2009). Instructional design for advanced learners: Training recognition skills to hasten expertise. Educational Technology Research and Development, 57, 359-376.
33. Federal Aviation Administration. (1996). Human factors team report on the interfaces between flightcrews and modern flight deck systems. Washington, DC: Author.
34. Federal Aviation Administration. (2004). Line operational simulations: Line oriented flight training, special purpose operational training, line operational evaluation (AC No. 120-35C). Washington, DC: Author.
35. Federal Aviation Administration. (2013). Final report of the performance-based operations aviation rulemaking committee/commercial aviation safety team flight deck automation working group. Washington, DC: Author.
36. Flin, R., Slaven, G., & Stewart, K. (1996). Emergency decision making in the offshore oil and gas industry. Human Factors, 38, 262-277.
37. Furukawa, H., & Parasuraman, R. (2003). Supporting systemcentered view of operators through ecological interface design: Two experiments on human-centered automation. In Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting (pp. 567-571). Santa Monica, CA: Human Factors and Ergonomics Society.
38. Gist, M. E. (1997). Training design and pedagogy: Implications for skill acquisition, maintenance, and generalization. In M. A Quinones & A. Ehrenstein (Eds.), Training for a rapidly changing workplace: Applications of psychological research (pp. 201-222). Washington, DC: American Psychological Association.
39. Glaser, R. (1990). The reemergence of learning theory within instructional research. American Psychologist, 45, 29-39.
40. Goodman, E., Stolterman, E., & Wakkary, R. (2011). Understanding interaction design practices. In Proceedings of the CHI 2011 Conference on Human Factors in Computer Systems (pp. 1061-1070). New York, NY: ACM.
41. Gorman, J. C., Cooke, N. J., & Salas, E. (2010). Preface to the special issue on collaboration, coordination, and adaptation in complex sociotechnical settings. Human Factors, 52, 143-146.
42. His Majesty's Government of Nepal. (1993). Report on the accident of Thai Airways International A310, Flight TG 311 (HSTID), on 31 July 1991. Katmandu, Nepal: Author.
43. Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16, 235-266.
44. Hmelo-Silver, C. E., & Pfeffer, M. G. (2004). Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cognitive Science, 28, 127-138.
45. International Civil Aviation Organization. (2002). Line operations safety audit (LOSA). Doc 9803, AN/761. Montreal, Canada: Author.
46. Jamieson, G. A., & Vicente, K. J. (2005). Designing effective human-automation-plant interfaces: A control-theoretic perspective. Human Factors, 47, 12-34.
47. Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional design. Human Factors, 40, 1-17.
48. Kirwan, B. (2001). Coping with accelerating socio-technical systems. Safety Science, 37, 77-107.
49. Klein, G. (1993) A recognition-primed decision (RPD) model of rapid decision making. In G. A. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 138-147). Norwood, NJ: Ablex.
50. Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012) The knowledgelearning-instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36, 757-798.
51. Lewis, C., & Rieman, J. (1994). Task-centered user interface design: A practical introduction. Retrieved from http://scholar.google.com/scholar
52. McLaughlin, K., Rikers, R. M., & Schmidt, H. G. (2008). Is analytic information processing a feature of expertise in medicine? Advances in Health Science Education, 13, 123-128.
53. McKinney, Jr. E. H., & Davis, K. J. (2005). Effects of deliberate practice on crisis decision performance. Human Factors, 45, 436-444.
54. Menges, R. J. (1975). Assessing readiness for professional practice. Review of Educational Research, 45, 173-208.
55. Molloy, R., & Parasuraman, R. (1996). Monitoring an automated system for a single failure: Vigilance and task complexity effects. Human Factors, 38, 311-322.
56. Mosier, K. L., Fischer, U., Morrow, D., Feigh, K. M., Durso, F. T., Sullivan, K., & Pop, V. (2013) Automation, task, and context features: Impacts on pilots' judgments of human-automation interaction. Journal of Cognitive Engineering and Decision Making, 7, 377-399.
57. Mosier, K. L., & Skitka, L. J. (1996). Human decision makers and automated decision aids: Made for each other? In R. Parasuraman & M. Mouloua (Eds.), Automation and human performance: Theory and application (pp. 201-220). Mahwah, NJ: Lawrence Erlbaum.
58. Mosier, K. L., Skitka, L. J., Heers, S., & Burdick, M. (1998). Automation bias: Decision making and performance in high-tech cockpits. International Journal of Aviation Psychology, 8, 47-63.
59. Mumaw, R. J., Roth, E. M., Vicente, K. J., & Burns, C. M. (2000). There is more to monitoring a nuclear power plant than meets the eye. Human Factors, 42, 36-55.
60. National Transportation Safety Board. (1984). Aircraft accident report, Scandinavian Airlines System, Flight 901, McDonnell Douglas DC-10-30, John F. Kennedy International Airport, Jamaica, New York, February 28, 1984 (Report No. AAR-84/15). Washington, DC: Author.
61. National Transportation Safety Board. (2008). Marine accident report, heeling accident on M/V Crown Princess, Atlantic Ocean off Port Canaveral, Florida, July 18, 2006 (Report No. MAR-08/01). Washington, DC: Author.
62. National Transportation Safety Board. (2012). Pipeline accident report, Enbridge Incorporated, hazardous liquid pipeline rupture and release, Marshall, Michigan, July 25, 2010 (Report No. PAR 12/01). Washington, DC: Author.
63. National Transportation Safety Board. (2014). Aircraft accident report, descent below visual glidepath and impact with seawall, Asiana Airlines Flight 214, Boeing 777-200ER, HL7742, San Francisco, California, July 6, 2013 (Report No. AAR-14/01). Washington, DC: Author.
64. Nikolec, M. I., & Sarter, N. B. (2007). Flight deck disturbance management: A simulator study of diagnosis and recovery from breakdowns in pilot-automation coordination. Human Factors, 49, 553-563.
65. Norman, D. A. (1990). The problem of automation: Inappropriate feedback and interaction, not over-automation. Philosophical Transactions of the Royal Society of London, B, 327, 585-593.
66. O'Connor, P., Campbell, J., Newon, J., Melton, J., Salas, E., & Wilson, K. A. (2008). Crew resource management training effectiveness: A meta-analysis and some critical needs. International Journal of Aviation Psychology, 18, 353-368.
67. Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38, 1-4.
68. Parasuraman, R., Bahri, T., Deaton, J., Morrison, J., & Barnes, M. (1992). Theory and design of adaptive automation in aviation systems (Progress Rep. No. NAWCADWAR-92033-60). Warminster, PA: Naval Air Warfare Center.
69. Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52, 381-410.
70. Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse and abuse. Human Factors, 39, 230-253.
71. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction, with automation. IEEE Transactions on Systems, Man, and Cybernetics- Part A: Systems and Humans, 30, 286-297.
72. Rasmussen, J. (1983). Skills, rules, and knowledge: Signals, signs, and symbols, and other distinctions in human performance models. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, 257-266.
73. Rasmussen, J., & Vicente, K. J. (1989). Coping with human errors through system design: Implications for ecological interface design. International Journal of Man-Machine Studies, 31, 517-534.
74. Salas, E., Cooke, N. J., & Rosen, M. A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50, 540-547.
75. Salas, E., Fowlkes, J. E., Stout, R. J., Milanovich, D. M., & Prince, C. (1999). Does CRM training improve teamwork skills in the cockpit? Two evaluation studies. Human Factors, 41, 326-343.
76. Salas, E., Rosen, M. A., Burke, C. S., Goodwin, G. F., & Fiore, S. M. (2006). The making of a dream team: When expert teams do best. In K. Ericsson et al. (Eds.), The Cambridge handbook of expertise and expert performance (pp. 439-456). Cambridge, UK: Cambridge University Press.
77. Salas, E., Wilson, K. A., Burke, C. S., & Wightman, D. C. (2006). Does crew resource management training work? An update, an extension, and some critical needs. Human Factors, 48, 392-412.
78. Sarter, N., & Woods, D. D. (1995). How in the world did we ever get into that mode? Mode error and awareness in supervisory control. Human Factors, 37, 5-19.
79. Sarter, N., & Woods, D. D. (1997). Team play with a powerful and independent agent: Operational experiences and automation surprises on the Airbus A-320. Human Factors, 39, 553-569.
80. Schaafstal, A., Schraagen, J. M., & van Berlo, M. (2000). Cognitive task analysis and innovation of training: The case of structured troubleshooting. Human Factors, 42, 75-86.
81. Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26, 113-125.
82. Sheridan, T. B., & Parasuraman, R. (2000). Human versus automation in responding to failures: An expected-value analysis. Human Factors, 42, 403-407.
83. Sheridan, T. B., & Parasuraman, R. (2005). Human-automation interaction. Reviews of Human Factors and Ergonomics, 1, 89-129.
84. Skitka, L. J., & Mosier, K. L. (2000). Automation bias and errors: Are crews better than individuals? International Journal of Aviation Psychology, 10, 85-97.
85. Smith, E. M., Ford, J. K., & Kozlowski, S. W. J. (1997). Building adaptive expertise: Implications for training design strategies, In M. A Quinones & A. Ehrenstein (Eds.), Training for a rapidly changing workplace: Applications of psychological research (pp. 89-118). Washington, DC: American Psychological Association.
86. Smith, P. J. (2003). Workplace learning and flexible delivery. Review of Educational Research, 73, 53-88.
87. Swezey, R. W., Perez, R. S., & Allen, J. A. (1991). Effects of instructional strategy and motion presentation conditions on the acquisition and transfer of electromechanical troubleshooting skill. Human Factors, 33, 309-323.
88. Taylor, G. S., Reinerman-Jones, L. E., Szalma, J. L., Mouloua, M., & Hancock, P. A. (2013). What to automate: Addressing the multidimensionality of cognitive resources through system design. Journal of Cognitive Engineering and Decision Making, 7, 311-329.
89. Vicente, K. J. (2002). Ecological interface design: Progress and challenges. Human Factors, 44, 62-78.
90. Wickens, C. D., Clegg, B. A., Vieane, A. Z., & Sebok, A. L. (2015). Complacency and automation bias in the use of imperfect automation. Human Factors, 57, 728-739.
91. Wiener, E. L. (1989). Human factors of advanced technology ("glass cockpit") transport aircraft (NASA Contractor Report 177528). Moffett Field, CA: NASA Ames Research Center.
92. Wiener, E. L., & Curry, R. E. (1980). Flight-deck automation: Promises and problems. Ergonomics, 23, 995-1011.
93. Wood, S. (2003). Flight crew reliance on automation (CAA Paper 2004/10). London, UK: Civil Aviation Authority.
94. Woods, D. D. (1996). Decomposing automation: Apparent simplicity, real complexity. In R. Parasuraman & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp. 3-17), Mahwah, NJ: Lawrence Erlbaum.