Development and evaluation of the NOTSS behaviour rating system for intraoperative surgery (2003-2008)

Safer Surgery: Analysing Behaviour in the Operating Theatre

Volumn , Issue , 2009, Pages 7-25

  Yule, Steven ^a   Flin, Rhona ^a   Maran, Nikki ^b,c   Rowley, David ^d,e   Youngson, George ^f,g   Duncan, John ^g,h   Paterson Brown, Simon ^d  

^a UNIVERSITY OF ABERDEEN   (United Kingdom)

^b ROYAL INFIRMARY OF EDINBURGH   (United Kingdom)

^c Scottish Simulator Centre   (United Kingdom)

^d UNIVERSITY OF EDINBURGH   (United Kingdom)

^e UNIVERSITY OF DUNDEE   (United Kingdom)

^f ROYAL ABERDEEN CHILDREN S HOSPITAL   (United Kingdom)

^g Royal College of Surgeons   (United Kingdom)

^h RAIGMORE HOSPITAL   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 79751471905     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Chapter

References (46)

1. ACGME (2007) Common Program Requirements: General Competencies Accreditation Counsel for Graduate Medical Education. Available from: [accessed October 2008].
2. Baldwin, P.J., Paisley, A.M. and Paterson-Brown, S. (1999) Consultant surgeons' opinions of the skills required of basic surgical trainees. British Journal of Surgery 86, 1078-82.
3. Baker, D., Mulqueen, C. and Dismukes, R. (2001) Training raters to assess resource management skills. In E. Salas, C. Bowers and E. Edens (eds), Improving Teamwork in Organizations. New Jersey: LEA, 131-45.
4. Catchpole, K.R., Giddings, A.E.B., Wilkinson, M., Hirst, G., Dale, T. and de Leval, M. (2007) Improving patient safety by identifying latent failures in successful operations. Surgery 142, 102-10.
5. Christian, C., Gustafson, M., Roth, E., Sheridan T., Gandhi, T., Dwyer, K., Zinner, M. and Dierks, M. (2006) A prospective study of patient safety in the operating room. Surgery 139, 159-73.
6. Crandall, B., Klein, G. and Hoffman, R. (2006) Working Minds: A Practitioner's Guide to Cognitive Task Analysis. Boston: MIT Press.
7. Davidson, P. (2002) The surgeon of the future and implications for training. ANZ Journal of Surgery 72, 822-8.
8. Edmondson, A.C. (2003) Speaking up in the operating room: How team leaders promote learning in interdisciplinary action teams. Journal of Management Studies 40(6), 1419-52.
9. Flanagan, J. (1954) The critical incident technique. Psychological Bulletin 51, 327-58.
10. Fletcher, G., Flin, R., McGeorge, P., Glavin, R., Maran, N. and Patey, R. (2004) Rating non-technical skills: Developing a behavioural marker system for use in anaesthesia. Cognition Technology and Work 6, 165-71.
11. Flin, R., Goeters, K., Amalberti, R., et al. (2003) The development of the NOTECHS system for evaluating pilots' CRM skills. Human Factors and Aerospace Safety 3, 95-117.
12. Flin, R., Yule, S., McKenzie, L., Paterson-Brown, S. and Maran, N. (2006a) Attitudes to teamwork and safety in the operating theatre. The Surgeon 4, 145-51.
13. Flin, R., Yule, S., Paterson-Brown, S., Maran, N. and Rowley, D. (2006b) The Non-Technical Skills for Surgeons (NOTSS) System Handbook (v1.2). Available at:
14. Flin, R., Yule, S., Paterson-Brown, S., Maran, N., Rowley, D.R. and Youngson, G.G. (2007) Teaching surgeons about non-technical skills. The Surgeon 5(2), 86-9.
15. Frank, J.R. (ed.) (2005) The CanMEDS 2005 Physician Competency Framework. Better Standards. Better Physicians. Better Care. Ottawa: The Royal College of Physicians and Surgeons of Canada.
16. Gawande, A.A., Zinner, M.J., Studdert, D.M. and Brennan, T.A. (2003) Analysis of errors reported by surgeons at three teaching hospitals. Surgery 133, 614-21.
17. Giddings, A.E.B. and Williamson, C. (2007) The Leadership and Management of Surgical Teams. London: The Royal College of Surgeons of England.
18. Glaser, B.G. and Strauss, A.L. (1967) The Discovery of Grounded Theory. Chicago: Aldine.
19. GMC (2001) Good Medical Practice. London: General Medical Council.
20. Gordon, S.E. (1993) Systematic Training Programme Design: Maximizing Effectiveness and Minimizing Liability. Englewood Cliffs, NJ: Prentice Hall.
21. Hall, J.C., Ellis, C. and Hamdorf, J. (2002) Surgeons and cognitive processes. British Journal of Surgery 90, 10-16.
22. Helmreich, R. and Davies, J. (1996) Human factors in the operating room: Interpersonal determinants of safety, efficiency and morale. In A. Aikenhead (ed.) Balliere's Clinical Anaesthesiology 10(2), 277-95.
23. Helmreich, R. and Schaefer, H. (1994) Team performance in the operating room. In M. Bogner (ed.) Human Error in Medicine. Hillsdale, NJ: LEA, 225-53.
24. Helmreich, R., Sexton, B. and Merritt, A. (1997) The Operating Room Management Attitudes Questionnaire (ORMAQ). University of Texas Aerospace Crew Research Project Technical Report 97-6. Austin, TX: The University of Texas.
25. Hoffmann, R., Crandall, B., and Shadbolt, N. (1998) A case study in cognitive task analysis methodology: The Critical Decision Method for the elicitation of expert knowledge. Human Factors 40, 254-76.
26. Jacklin, R., Sevdalis, N., Darzi, A. and Vincent, C. (2008) Mapping surgical practice decision making: An interview study to evaluate decisions in surgical care. The American Journal of Surgery, 195, 689-96.
27. Kneebone, R. and Darzi, A. (2005) New professional roles in surgery. British Medical Journal 330, 803-804.
28. Klampfer, B., Flin, R., Helmreich, R.L., Hausler, R., Sexton, B., Fletcher, G., et al. (2001) Group Interaction in High Risk Environments: Enhancing Performance in High Risk Environments, Recommendations for the use of Behavioural Markers. Berlin: GIHRE. Available on [last accessed November 2008].
29. Mishra, K., Catchpole, K., Dale, T. and McCulloch, P. (2008) The influence of nontechnical performance on technical outcome in laparoscopic cholecystectomy. Surgical Endoscopy 22, 68-73.
30. Perrow, C. (1999) Normal Accidents: Living with High-risk Technologies. Princeton, NJ: Princeton University Press.
31. Reason, J.T. (1997) Managing the Risks of Organizational Accidents. Aldershot: Ashgate.
32. SASM (2003) Scottish Audit of Surgical Mortality Annual Report - 2001 data. Glasgow: SASM.
33. Seamster, T., Redding, R. and Kaempf, G. (1997) Applied Cognitive Task Analysis in Aviation. Aldershot: Avebury.
34. Sevdalis, N., Healey, A.N. and Vincent, C.A. (2007) Distracting communications in the operating theatre. Journal of Evaluation in Clinical Practice 13, 390-4
35. Sevdalis, N., Davis, R., Koutantji, M., Undre, S., Darzi, A. and Vincent, C.A. (2008) Reliability of a revised NOTECHS scale for use in surgical teams. The American Journal of Surgery 196, 184-90.
36. Sexton, B., Thomas, E. and Helmreich, R. (2000) Error, stress, and teamwork in medicine and aviation: Cross sectional surveys. British Medical Journal 320, 745-9.
37. Studdert, D.M., Mello, M.M., Gawande, A.A., et al. (2006) Claims, errors, and compensation payments in medical malpractice litigation. The New England Journal of Medicine 354, 2024-33.
38. Taylor-Adams, S. and Vincent, C. (2004) Systems analysis of clinical incidents: The London protocol. Clinical Risk 10, 211-20.
39. Undre, S., Sevdalis, N., Healey, A.N., Darzi, A. and Vincent, C. (2007) Observational Teamwork Assessment for Surgery (OTAS): Refinement and application in urological surgery. World Journal of Surgery 31, 1373-81.
40. Vincent, C.A., Neale, G. and Woloshynowych, M. (2001) Adverse events in British hospitals: Preliminary restrospective record review. British Journal of Medicine 322, 517-19.
41. Youngson, G.G. (2000) Surgical Competence: Acquisition, Measurement, and Retention. Edinburgh: The Royal College of Surgeons of Edinburgh.
42. Yule, S., Flin, R., Paterson-Brown, S. and Maran, N. (2006a) Non-technical skills for surgeons: A review of the literature. Surgery 139, 140-9.
43. Yule, S., Flin, R., Paterson-Brown, S., Maran, N. and Rowley, D. (2006b) Development of a rating system for surgeons' non-technical skills. Medical Education 40, 1098-104.
44. Yule, S., Flin, R., Maran, N., Rowley, D. R., Youngson, G.G. and Paterson-Brown, S. (2008a) Surgeons' non-technical skills in the operating room: Reliability testing of the NOTSS behaviour rating system. World Journal of Surgery 32, 548-56.
45. Yule, S., Flin, R., Rowley, D., Mitchell, A., Youngson, G.G., Maran, N. and Paterson-Brown, S. (2008b) Debriefing surgical trainees on non-technical skills (NOTSS). Cognition, Technology & Work 10, 265-74.
46. Yule, S., Rowley, D., Flin, R., Maran, N., Youngson, G.G., Duncan, J. and Paterson- Brown, S. (2009) Experience matters: Comparing novice and expert ratings of non-technical skills using the NOTSS system. Australian Journal of Surgery 79, 154-160.