Framework to support scenario development for human-centered alerting system evaluation

IEEE Transactions on Human-Machine Systems

Volumn 43, Issue 6, 2013, Pages 595-607

  Bolton, Matthew L ^a   Göknur, Sinan ^b   Bass, Ellen J ^c  

^a University of Illinois at Chicago   (United States)

^b DUKE UNIVERSITY   (United States)

^c Drexel University   (United States)

Author keywords

Air traffic control; Alarm systems; Alerting systems; Decision support systems; Man machine systems

Indexed keywords

ALERTING SYSTEMS; EXECUTION SEQUENCES; INDEPENDENT VARIABLES; SCENARIO DEVELOPMENT; SELF SEPARATIONS; SYSTEM-LEVEL PERFORMANCE;

AIR TRAFFIC CONTROL; ALARM SYSTEMS; DECISION SUPPORT SYSTEMS; MAN MACHINE SYSTEMS;

ARTIFICIAL INTELLIGENCE;

EID: 84893116456     PISSN: 21682291     EISSN: None     Source Type: Journal    
DOI: 10.1109/THMS.2013.2283399     Document Type: Article

References (43)

1. R. Barhydt, T. M. Eischeid, M. T. Palmer, and D. J. Wing, "Regaining lost separation in a piloted simulation of autonomous aircraft operations," presented at the 5th USA/Eur. Air Traffic Management R&D Semin., Budapest, Hungary, Jun. 2003.
2. B. Barmore, E. Johnson, D. J. Wing, and R. Barhydt, "Airborne conflict management within confined airspace in a piloted simulation of DAG-TM autonomous aircraft operations," presented at the 5th USA/Eur. Air Traffic Manag. R&D Semin., Budapest, Hungary, Jun. 2003.
3. E. J. Bass and A. R. Pritchett, "Human-Automated Judge Learning: A methodology for examining human interaction with information analysis automation," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no. 4, pp. 759-776, Jul. 2008.
4. E. J. Bass, S. T. Ernst-Fortin, R. L. Small, and J. T. Hogans Jr., "Architecture and development environment of a knowledge-based monitor that facilitate incremental knowledge-base development," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 34, no. 4, pp. 441-449, Jul. 2004.
5. E. J. Bass, L. A. Baumgart, and K. K. Shepley, "The effect of information analysis automation display content on human judgment performance in noisy environments," J. Cognit. Eng. Decision Making, vol. 7, no. 1, pp. 49-65, 2013.
6. M. L. Bolton, R. I. Siminiceanu, and E. J. Bass, "A systematic approach to model checking human-automation interaction using task-analytic models," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 41, no. 5, pp. 961-976, Sep. 2011.
7. G. Booch, The Unified Modeling Language User Guide, 2/E. Delhi, India: Pearson Education, 2005.
8. S. S.Choi, J. K. Park, J. H. Hong, H. G. Kim, S. H. Chang, andK. S.Kang, "Development strategies of an intelligent human-machine interface for next generation nuclear power plants," IEEE Trans. Nucl. Sci., vol. 43, no. 3, pp. 2096-2114, Jun. 1996. (Pubitemid 126770880)
9. S. D. Davis and A. R. Pritchett, "Alerting system assertiveness, knowledge, and over-reliance," J. Inf. Technol. Impact, vol. 1, no. 3, pp. 119-143, 1999.
10. W. den Braven, "Analysis of aircraft/air traffic control performance," in Proc AIAA Guid., Navigat., Control Conf., Baltimore, MD, USA, Aug. 7-10, 1995, pp. 1721-1738.
11. J. Ellerbroek, K. C. R. Brantegem, M. M. van Paassen, N. de Gelder, and M. Mulder, "Experimental evaluation of a coplanar airborne separation display," IEEE Trans. Human-Mach. Syst., vol. 43, no. 3, pp. 290-301, May 2013.
12. J. Ellerbroek, K. C. R. Brantegem, M. M. van Paassen, and M. Mulder, "Design of a Coplanar Airborne SeparationDisplay," IEEE Trans.Human- Mach. Syst., vol. 43, no. 3, pp. 277-289, May 2013.
13. M. R. Endsley, "Situation awareness global assessment technique (SAGAT)," in Proc. Nat. Aerosp. Electron. Conf., 1998, pp. 789-795.
14. FAA. (2011). FAA AC 120-55C - Air Carrier Operational Approval and use of TCAS II. [Online]. Available: http://www.faa.gov/document Library/media/ Advisory-Circular/AC%20120-55C.pdf
15. S. Göknur,M. L. Bolton, and E. J. Bass, "Adding a motor control component to the operator function model expert system to investigate air traffic management concepts using simulation," presented at the IEEE Int. Conf. Syst., Man, Cybernet., Hague, The Netherlands, Oct. 2004.
16. T.Weilkiens, Systems Engineering with SysML/UML: Modeling, Analysis, Design. Morgan Kaufmann, 2008.
17. S. G. Hart and L. E. Staveland, "Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research," in Human Mental Workload, P. A. Hancock and N. Meshkati, Eds. New York, NY, USA: Elsevier Science Publishers, 1988, pp. 77-106.
18. D. Harel, "On visual formalisms," ACMCommun., vol. 31, no. 5, pp. 514- 530, 1988.
19. J. G. Hollands and C. D. Wickens, Engineering Psychology and Human Performance. Englewood Cliffs, NJ, USA: Prentice Hall, 1999.
20. J. K. Kuchar, "Methodology for alerting-system performance evaluation," J. Guid., Control, Dyn., vol. 19, no. 2, pp. 438-444, 1996. (Pubitemid 126536020)
21. K. Latorella and J. Chamberlain, "Tactical vs. strategic behavior: General aviation piloting in convective weather scenarios," presented at the Human Factors Ergon. Annu. Meet., Baltimore, MD, USA, 2002.
22. D. Meister and G. F. Rabideau, Human Factors Evaluation in System Development. New York, NY, USA: John Wiley, 1965.
23. J. Meyer and Y. Bitan, "Why better operators receive worse warnings?" Human Factors, vol. 44, no. 3, pp. 343-353, 2002. (Pubitemid 35448637)
24. S. Mondoloni,M. T. Palmer, and D. J.Wing, "Development of a prototype airborne conflict detection and resolution simulation capability," presented at the AIAA Guidance, Navigation Control Conf., Monterey, CA, USA, 2002, paper AIAA-2002-4446.
25. R. Parasuraman and V. Riley, "Humans and automation: Use, misuse, disuse, abuse," Human Factors, vol. 39, no. 2, pp. 230-253, 1997.
26. A. R. Pritchett, "Reviewing the safety contribution of cockpit alerting systems," Human Factors Aerosp. Safety, vol. 1, no. 1, 2001.
27. A.R. Pritchett and J. R. Hansman, "Pilot non-conformance to alerting system commands," presented at the 9th Int. Symp. Aviation Psychol., Columbus, OH, USA, 1997.
28. A. R. Pritchett and B. Vándor, "Designing situation displays to promote conformance to automatic alerts," in Proc. 45th Annu. Meet. Human Factors, Ergonomics Soc., 2001, pp. 311-355.
29. A. R. Pritchett, E. S. Fleming, J. J. Zoetrum, W. P. Cleveland, V. M. Popescu, and D. A. Thakkar, "Examining pilot compliance to collision avoidance advisories," in Advances in Human Aspects of Aviation, S. J. Landry, Ed., Boca Raton, FL, USA: CRC Press, 2012, pp. 214- 223.
30. N. B. Sarter and D. D. Woods, "Pilot interaction with cockpit automation: Operational experiences with the flight management system," Int. J. Aviation Psychol., vol. 2, no. 4, pp. 303-321, 1992.
31. N. B. Sarter and D. D. Woods, "Pilot interaction with cockpit automation II: An experimental study of pilots' model and awareness of the flight management system," Int. J. Aviation Psychol., vol. 4, no. 1, pp. 1-28, 1994.
32. P. C. Shutte, "Definitions of tactical and strategic: An informal study," NASA Tech. Rep., NASA/TM-2004-213024, Nov. 2004.
33. A. B. M. Skjerve andG. Skraaning, Jr., "The quality of human-automation co-operation in human-system interface for nuclear power plants," Int. J. Human-Comput. Stud., vol. 61, pp. 649-677, 2004.
34. R. L. Small and E. J. Bass, "Certify for success: A methodology for human-centered certification of advanced aviation systems," in Human Factors in Certification, J. A. Wise and V. D. Hopkin, Eds. Mahwah, NJ, USA: Lawrence Earlbaum Associates, 2000, pp. 139-150.
35. R. D. Sorkin and D. D. Woods, "Systems with human monitors: A signal detection analysis," Human-Comput. Interaction, vol. 1, no. 1, pp. 49-75, 1985.
36. S. V. M. V. Clari, R. C. J Ruigrok, B. W. M. Heesben, and J. Groeneweg, "Advanced aviation concepts via simulation: Research flight simulation of future autonomous aircraft operations," in Proc. Winter Simul. Conf., 2002, pp. 1226-12234.
37. E. L. Wiener and R. E. Curry, "Flight deck automation: Promises and Problems," Ergonomics, vol. 23, no. 10, pp. 995-1011, 1980. (Pubitemid 11195680)
38. D. J. Wing, K. Krishnamurthy, R. Barhydt, and B. Barmore, "Pilot interactions in an over-constrained conflict scenario as studied in a piloted simulation of autonomous aircraft operations," presented at the 5th USA/Eur. Air Traffic Management R&D Semin., Budapest, Hungary, 2003.
39. L. C. Yang and J. K. Kuchar, "Performance metric alerting: A new design approach for complex alerting problems," IEEE Trans. Syst.,Man, Cybern. A, Syst., Humans, vol. 32, no. 1, pp. 123-134, Jan. 2002. (Pubitemid 34554800)
40. Yourdon, Inc., Yourdon Systems Methods. Englewood Cliffs, NJ, USA: Yourdon Press, 1993.
41. A. P. Sage, and W. B. Rouse, Handbook of Systems Engineering and Management. New York, NY, USA: John Wiley, 2011.
42. D. M. Buede, The Engineering Design of Systems: Methods and Models. 2nd ed. (Wiley Series in Systems Engineering and Management), New York, NY, USA: John Wiley, 2009.
43. A. R. Pritchett, and L. J. Yankosky, "Pilot-performed in-trail spacing and merging: An experimental study," J. Guidance, Control, Dynamics, vol. 26, no. 1, pp. 143-150, 2003.