Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery

Journal of Surgical Research

Volumn 171, Issue 1, 2011, Pages

  Oropesa, Ignacio ^a,b   Sánchez González, Patricia ^a,b   Lamata, Pablo ^a   Chmarra, Magdalena K ^c   Pagador, José B ^d   Sánchez Margallo, Juan A ^d   Sánchez Margallo, Francisco M ^d   Gómez, Enrique J ^a,b  

^a UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

^b BIOMATERIALES Y NANOMEDICINA CIBER BBN   (Spain)

^c DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^d JESÚS USÓN MINIMALLY INVASIVE SURGERY CENTRE   (Spain)

Author keywords

accreditation; basic skills; classification; human motion tracking; laparoscopic surgery; metrics; objective evaluation; virtual reality

Indexed keywords

ACCREDITATION; ARTICLE; HUMAN; LAPAROSCOPIC SURGERY; MACHINE LEARNING; PATIENT SAFETY; PRIORITY JOURNAL; PROFESSIONAL COMPETENCE; PSYCHOMOTOR PERFORMANCE; SURGICAL TRAINING;

EID: 80054715352     PISSN: 00224804     EISSN: 10958673     Source Type: Journal    
DOI: 10.1016/j.jss.2011.06.034     Document Type: Review

References (126)

1. A. Cuschieri Laparoscopic surgery: Current status, issues and future developments Surgeon 3 2005 125
2. J. Usón, F.M. Sánchez-Margallo, S. Pascual, and S. Climent Formación en Cirugía Laparoscópica Paso a Paso 4th ed. 2010 Minimally Invasive Surgery Centre Jesús Usón eds Cáceres, Spain
3. P. Breedveld, H.G. Stassen, and D.W. Meijer Observation in laparoscopic surgery: Overview of impending effects and supporting aids J Laparoendosc Adv 10 2000 231
4. R. Aggarwal, K. Moorthy, and A. Darzi Laparoscopic skills training and assessment Br J Surg 91 2004 1549
5. A.E. Park, and D. Witzke The surgical competence conundrum Surg Endosc 16 2002 555
6. W.P. Ritchie Basic certification in surgery by the American Board of Surgery (ABS). What does it mean? Does it have value? Is it relevant? A personal opinion Ann Surg 239 2004 133
7. W.S. Halsted The training of surgeons Johns Hopkins Hosp Bull 15 1904 267
8. R.M. Satava, A. Cuschieri, and J. Hamdorf Metrics for objective assessment Surg Endosc 17 2003 220
9. R. Sidhu, E. Grober, and L. Musselman Assessing competency in surgery: Where to begin? Surgery 135 2004 6
10. K.R. Wanzel, M. Ward, and R.K. Reznick Teaching the surgical craft: From selection to certification Curr Prob Surg 39 2002 573
11. M.C. Vassiliou, L.S. Feldman, and C.G. Andrew A global assessment tool for evaluation of intraoperative laparoscopic skills Am J Surg 190 2005 107
12. G.M. Fried, and L.S. Feldman Objective assessment of technical performance World J Surg 32 2008 156
13. R.M. Harden, M. Stevenson, and W.W. Downie Assessment of clinical competence using objective structured examination Br Med J 1 1975 447
14. J. Martin, G. Regehr, and R. Reznick Objective structured assessment of technical skill (OSATS) for surgical residents Br J Surg 84 1997
15. K. Moorthy, Y. Munz, and S.K. Sarker Objective assessment of technical skills in surgery Br Med J 327 2003 1032
16. M.K. Chmarra, C.A. Grimbergen, and F.W. Jansen How to objectively classify residents based on their psychomotor laparoscopic skills? Minim Invasive Ther Allied Technol 19 2010 2
17. A. Dosis, R. Aggarwal, and F. Bello Synchronized video and motion analysis for the assessment of procedures in the operating theater Arch Surg 140 2005 293
18. P.D. van Hove, G.J.M. Tuijthof, and E.G.G. Verdaasdonk Objective assessment of technical surgical skills Br J Surg 97 2010 972
19. R.M. Satava Historical review of surgical simulation - A personal perspective World J Surg 32 2008 141
20. M.K. Chmarra, C.A. Grimbergen, and J. Dankelman Systems for tracking minimally invasive surgical instruments Minim Invasive Ther Allied Technol 16 2007 328
21. I. Oropesa, P. Sánchez-González, and A.M. Cano Objective evaluation methodology for surgical motor skills assessment Minim Invasive Ther Allied Technol 10 2010 55
22. S. Cotin, N. Stylopoulos, and M. Ottensmeyer Metrics for laparoscopic skills trainers: The weakest link! T. Dohi, R. Kikinis, Medical image computing and computer-assisted intervention - MICCAI 2002 2002 Springer Berlin/Heidelberg 35 43
23. P. Lamata Methodologies for the analysis, design and evaluation of laparoscopic surgical simulators. PhD Thesis Louvain: Presses univ de Louvain 2006
24. L.F. Sánchez-Peralta, F.M. Sánchez-Margallo, and J.L. Moyano-Cuevas Construct and face validity of SINERGIA laparoscopic virtual reality simulator Int J Comput Assist Radiol Surg 5 2010 307
25. K.R. van Sickle, D.A. McClusky 3rd, and A.G. Gallagher Construct validation of the ProMIS simulator using a novel laparoscopic suturing task Surg Endosc 19 2005 1227
26. M.G. Pellen, L.F. Horgan, and J.R. Barton Construct validity of the ProMIS laparoscopic simulator Surg Endosc 23 2009 130
27. D. Broe, P.F. Ridgway, and S. Johnson Construct validation of a novel hybrid surgical simulator Surg Endosc 20 2006 900
28. M. Pellen, L. Horgan, and J.R. Barton Laparoscopic surgical skills assessment: Can simulators replace experts? World J Surg 33 2009 440
29. S. Yamaguchi, K. Konishi, and T. Yasunaga Construct validity for eye-hand coordination skill on a virtual reality laparoscopic surgical simulator Surg Endosc 21 2007 2253
30. A. Zhang, M. Hünerbein, and Y. Dai Construct validity testing of a laparoscopic surgery simulator (Lap Mentor®): Evaluation of surgical skill with a virtual laparoscopic training simulator Surg Endosc 22 2008 1440
31. R. Aggarwal, P. Crochet, and A. Dias Development of a virtual reality training curriculum for laparoscopic cholecystectomy Br J Surg 96 2009 1086
32. E.M. McDougall, F.A. Corica, and J.R. Boker Construct validity testing of a laparoscopic surgical simulator J Am Coll Surg 202 2006 779
33. E.G.G. Verdaasdonk, L.P.S. Stassen, and M.P. Schijven Construct validity and assessment of the learning curve for the SIMENDO endoscopic simulator Surg Endosc 21 2007 1406
34. J.R. Eriksen, and T. Grantcharov Objective assessment of laparoscopic skills using a virtual reality stimulator Surg Endosc 19 2005 1216
35. V. Sherman, L. Feldman, and D. Stanbridge Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator Surg Endosc 19 2005 678
36. A. Duffy, N. Hogle, and H. McCarthy Construct validity for the LAPSIM laparoscopic surgical simulator Surg Endosc 19 2005 401
37. C.R. Larsen, T. Grantcharov, and R. Aggarwal Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator Surg Endosc 20 2006 1460
38. D.T. Woodrum, P.B. Andreatta, and R.K. Yellamanchilli Construct validity of the LapSim laparoscopic surgical simulator Am J Surg 191 2006 28
39. H.W.R. Schreuder, K.W. van Dongen, and S.J. Roeleveld Face and construct validity of virtual reality simulation of laparoscopic gynecologic surgery Obstet Gynecol 200 2009 540
40. K.W. van Dongen, E. Tournoij, and D.C. van der Zee Construct validity of the LapSim: Can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc 21 2007 1413
41. L. Panait, R.L. Bell, and K.E. Roberts Designing and validating a customized vitual reality-based laparoscopic skills curriculum J Sur Educ 65 2008 413
42. A. Gallagher, A. Lederman, and K. McGlade Discriminative validity of the minimally invasive surgical trainer in virtual reality (MIST-VR) using criteria levels based on expert performance Surg Endosc 18 2004 660
43. S.S. McNatt, and C.D. Smith A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons Surg Endosc 15 2001 1085
44. S. Maithel, R. Sierra, and J. Korndorffer Construct and face validity of MIST-VR, Endotower, and CELTS Surg Endosc 20 2006 104
45. N. Stylopoulos, S. Cotin, and S. Maithel Computer-enhanced laparoscopic training system (CELTS): Bridging the gap Surg Endosc 18 2004 782
46. M.K. Chmarra, S. Klein, and J.C.F. de Winter Objective classification of residents based on their psychomotor laparoscopic skills Surg Endosc 24 2010 1031
47. H. Egi, M. Okajima, and M. Yoshimitsu Objective assessment of endoscopic surgical skills by analyzing direction-dependent dexterity using the Hiroshima University Endoscopic Surgical Assessment Device (HUESAD) Surg Today 38 2008 705
48. C. Sokollik, J. Gross, and G. Buess New model for skills assessment and training progress in minimally invasive surgery Surg Endosc 18 2004 495
49. J. Rosen, M. Solazzo, and B. Hannaford Task decomposition of laparoscopic surgery for objective evaluation of surgical residents' learning curve using hidden Markov model Comput Aided Surg 7 2002 49
50. J. Rosen, J.D. Brown, and L. Chang Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model IEEE Trans Biomed Eng 53 2006 399
51. S.D. Bann, M.S. Khan, and A.W. Darzi Measurement of surgical dexterity using motion analysis of simple bench tasks World J Surg 27 2003 390
52. V. Datta, S. Mackay, and M. Mandalia The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model J Am Coll Surg 193 2001 479
53. K. Moorthy, Y. Munz, and A. Dosis Bimodal assessment of laparoscopic suturing skills Surg Endosc 18 2004 1608
54. R. Brydges, R. Classen, and J. Larmer Computer-assisted assessment of one-handed knot tying skills performed within various contexts: A construct validity study Am J Surg 192 2006 109
55. N.K. Francis, G.B. Hanna, and A. Cuschieri The performance of master surgeons on the advanced dundee endoscopic psychomotor tester contrast validity study Arch Surg 37 2002 841
56. T. Horeman, S.P. Rodrigues, and F.W. Jansen Force measurement platform for training and assessment of laparoscopic skills Surg Endosc 24 2010 3102
57. A.S. Thijssen, and M.P. Schijven Contemporary virtual reality laparoscopy simulators: Quicksand or solid grounds for assessing surgical trainees? Am J Surg 199 2010 529
58. M.K. Chmarra, F.W. Jansen, and C.A. Grimbergen Retracting and seeking movements during laparoscopic goal-oriented movements. Is the shortest path length optimal? Surg Endosc 22 2008 943
59. J. Rosen, B. Hannaford, and C. Richards Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills IEEE Trans Biomed Eng 48 2001 579
60. A. Cuschieri, S.N. Francis, and J. Crosby What do master surgeons think of surgical competence and revalidation? Am J Surg 182 2001 110
61. J. Dankelman, M.K. Chmarra, and E.G.G. Verdaasdonk Fundamental aspects of learning minimally invasive surgical skills Minim Invasive Ther Allied Technol 14 2005 247
62. K.S. Gurusamy, R. Aggarwal, and L. Palanivelu Virtual reality training for surgical trainees in laparoscopic surgery Cochrane Database Syst Rev 1 2009 CD006575
63. B. Dunkin, G.L. Adrales, and K. Apelgren Surgical simulation: A current review Surg Endosc 21 2007 357
64. P.C. Neary, E. Boyle, and C.P. Delaney Construct validation of a novel hybrid virtual-reality simulator for training and assessing laparoscopic colectomy; Results from the first course for experienced senior laparoscopic surgeons Surg Endosc 22 2008 2301
65. G. Ahlberg, O. Kruuna, and C.E. Leijonmarck Is the learning curve for laparoscopic fundoplication determined by the teacher or the pupil? Am J Surg 189 2005 184
66. A.K. Madan, C.T. Frantzides, and L.M. Sasso Laparoscopic baseline ability assessment by virtual reality Laparoendosc J Surg Tech A 15 2005 13
67. D. Stefanidis, J.R. Korndorffer Jr., and F.W. Black Psychomotor testing predicts rate of skill acquisition for proficiency-based laparoscopic skills training Surgery 140 2006 252
68. T.P. Grantcharov, J. Rosenberg, and E. Pahle Virtual reality computer simulation Surg Endosc 15 2001 242
69. Okrainec A, Greco E, Aarts MA, et al. Virtual reality training on basic laparoscopic tasks versus virtual reality training of an entire surgical procedure: A randomized controlled trial using real world operations as an outcome. Proceedings of the 2008 Annual Meeting of the Association for Surgical Education, Toronto, Canada, 2008.
70. E. Greco, J. Escallon, and T. Grantcharov Do performance measures on the LapMentor VR simulator predict FLS performance? Can J Surg 51 2008 15
71. P.L. Youngblood, S. Srivastava, and M. Curet Comparison of training on two laparoscopic simulators and assessment of skills transfer to surgical performance J Am Coll Surg 200 2005 546
72. J. Newmark, V. Dandolu, and R. Milner Correlating virtual reality and box trainer tasks in the assessment of laparoscopic surgical skills Obstet Gynecol 197 2007 546
73. I. Hassan, T. Osei-Agymang, and D. Radu Simulation of laparoscopic surgery-Four years' experience at the Department of Surgery of the University Hospital Marburg Wien Klin Wochenschr 120 2008 70
74. E.M. Ritter, T.W. Kindelan, and C. Michael Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS) Surg Endosc 21 2007 1441
75. S.M. Botden, S.N. Buzink, and M.P. Schijven Augmented versus virtual reality laparoscopic simulation: What is the difference? A comparison of the ProMIS augmented reality simulator versus LapSim virtual reality laparoscopic simulator World J Surg 31 2007 764
76. A. McCluney, L. Feldman, and G. Fried Validation of the ProMIS hybrid simulator using a standard set of laparoscopic tasks Proc Sages 2006
77. E.G.G. Verdaasdonk, L.P.S. Stassen, and L.J. Monteny Validation of a new basic virtual reality simulator for training of basic endoscopic skill: The SIMENDO Surg Endosc 20 2006 511
78. P. Lamata, E.J. Gomez, and F. Bello Conceptual framework for laparoscopic VR simulators IEEE Comput Graph Appl 26 2006 69
79. M.K. Chmarra, J. Dankelman, and J.J. van den Dobbelsteen Force feedback and basic laparoscopic skills Surg Endosc 22 2008 2140
80. L. Panait, E. Akkary, and R.L. Bell The role of haptic feedback in laparoscopic simulation training J Surg Res 156 2009 312
81. S.B. Issenberg, W.C. Mcgaghie, and E.R. Petrusa Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review Med Teach 27 2005 10
82. J. Lynch, P. Aughwane, and T.M. Hammond Video games and surgical ability: A literature review J Surg Educ 67 2010 184
83. P. Lamata, E.J. Gómez, and F.M. Sánchez-Margallo SINERGIA laparoscopic virtual reality simulator: Didactic design and technical development Comput Methods Programs Biomed 85 2007 273
84. P. Lamata, E.J. Gomez, and F.L. Hernandez Understanding perceptual boundaries in laparoscopic surgery IEEE Trans Bio Eng 55 2008 866
85. P. Sánchez-González, I. Oropesa, and F.M. Sánchez-Margallo A virtual reality simulator-based system for MIS objective evaluation Minim Invasive Ther Allied Technol 10 2010 15
86. G. Hanna, T. Frank, and A. Cuschieri Objective assessment of endoscopic knot quality Am J Surg 174 1997 410
87. J. Solis, N. Oshima, and H. Ishii Towards understanding the suture/ligature skills during the training process using WKS-2RII Int J Comput Assist Radiol Surg 3 2008 231
88. F.M. Sánchez-Margallo, J.B. Pagador, and L.F. Sánchez- Peralta Estudios de validez de un dispositivo háptico de realidad aumentada (ARH) para la evaluación objetiva de la sutura laparoscópica intracorpórea Cir Esp 88 2010 218
89. V. Datta, M. Mandalia, and S. Mackay Relationship between skill and outcome in the laboratory-based model Surgery 131 2002 318
90. V. Datta, S. Bann, and M. Mandalia The surgical efficiency score: A feasible, reliable, and valid method of skills assessment Am J Surg 192 2006 372
91. A.I.M. Macmillan, and A. Cuschieri Assessment of innate ability and skills for endoscopic manipulations by the Advanced Dundee Endoscopic Psychomotor Tester: Predictive and concurrent validity Am J Surg 177 1999 274
92. G. Hanna, T. Drew, and P. Clinch Computer-controlled endoscopic performance assessment system Surg Endosc 12 1998 997
93. M.K. Chmarra, N.H. Bakker, and C.A. Grimbergen TrEndo, a device for tracking minimally invasive surgical instruments in training setups Sensor Actuat: A Physical 126 2006 328
94. J.B. Pagador, L.F. Sánchez, and J.A. Sánchez Augmented reality haptic (ARH): An approach of electromagnetic tracking in minimally invasive surgery Int J Comput Assist Radiol Surg 6 2011 257
95. J. Rosen, J.D. Brown, and M. Barreca The Blue DRAGON-A system for monitoring the kinematics and the dynamics of endoscopic tools in minimally invasive surgery for objective laparoscopic skill assessment Stud Health Technol Inform 85 2002 412
96. M.A. Hayter, Z. Friedman, and M.D. Bould Validation of the Imperial College Surgical Assessment Device (ICSAD) for labour epidural placement Can J Anesth 56 2009 49
97. R. Aggarwal, T. Grantcharov, and K. Moorthy An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room Ann Surg 245 2007 992
98. Sánchez-González P, Cano AM, Oropesa I, et al. Laparoscopic video analysis for training and image guided surgery. Minim Invasive Ther Allied Technol 2010 (in press).
99. S. Voros, J. Long, and P. Cinquin Automatic localization of laparoscopic instruments for the visual serving of an endoscopic camera holder Med Image Comput Comput Assist Interv 9 2006 535
100. C. Doignon, F. Nageotte, and B. Maurin Pose estimation and feature tracking for robot assisted surgery with medical imaging D. Kragic, V. Kyrik, Unifying perspectives in computational and robot vision vol. 8 2008 Springer US New York 79
101. J.A. Sánchez-Margallo, F.M. Sánchez-Margallo, and J.B. Pagador Video-based assistance system for training in minimally invasive surgery Minim Invasive Ther Allied Technol 2010 (In press).
102. J.B. Pagador, F.M. Sánchez-Margallo, and L.F. Sánchez- Peralta Objective assessment of basic laparoscopic skills using automatic video-based technique Minim Invasive Ther Allied Technol 19 2010 55
103. A. Cano, F. Gayá, and P. Lamata Laparoscopic tool tracking method for augmented reality surgical applications F. Bello, P. Edwards, Biomedical Simulation 2008 Springer Berlin/Heidelberg 191 196
104. G.M. Fried, L.S. Feldman, and M.C. Vassiliou Proving the value of simulation in laparoscopic surgery Ann Surg 3 2004 518
105. A.K. Jain, R.P.W. Duin, and J. Mao Statistical pattern recognition: A review IEEE Trans Pattern Anal Machine Intell 22 2000 4
106. C.E. Reiley, and G.D. Hager Task versus subtask surgical skill evaluation of robotic minimally invasive surgery Med Image Comput Comput Assist Interv 12 2009 435
107. G. Megali, S. Sinigaglia, and O. Tonet Modelling and evaluation of surgical performance using hidden Markov models IEEE Trans Biomed Eng 53 2006 1911
108. G. Megali, S. Sinigaglia, and O. Tonet Understanding expertise in surgical gesture by means of hidden Markov models Proc IEEE RAS EMBS Int Conf Biomed Robot Biomechatron 2006 625
109. J.J.H. Leong, M. Nicolaou, and L. Atallah HMM assessment of quality of movement trajectory in laparoscopic surgery Comput Aided Surg 12 2007 335
110. T.E. Murphy, C.M. Vignes, and D.D. Yuh Automatic motion recognition and skill evaluation for dynamic tasks Proc Euro Haptics 2003 363
111. T.E. Murphy Towards objective surgical skill evaluation with hidden Markov model-based motion recognition 2004 Johns Hopkins University Baltimore
112. L. Rabiner A tutorial on hidden Markov models and selected applications in speech recognition Proc IEEE 77 1989 257
113. H.C. Lin, I. Shafran, and D. Yuh Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions Comput Aided Surg 11 2006 220
114. J. Huang, S. Payandeh, and P. Doris Fuzzy classification: Towards evaluating performance on a surgical simulator Stud Health Technol Inform 111 2005 194
115. I. Hajshirmohammadi, and S. Payandeh Fuzzy set theory for performance evaluation in a surgical simulator Presence 16 2007 603
116. Estebanez B, Jimenez G, Muñoz VF, et al. Minimally invasive surgery maneuver recognition based on surgeon model. Intelligent Robots and Systems, 2009 IROS 2009 IEEE/RSJ International Conference on, 2009;5522.
117. T. Blum, N. Padoy, and H. Feußner Modeling and online recognition of surgical phases using hidden Markov models D. Metaxas, L. Axel, G. Fichtinger, G. Székely, Medical image computing and computer-assisted intervention-MICCAI 2008 2008 Springer Berlin/Heidelberg 627
118. R. Kneebone Simulation in surgical training: Educational issues and practical implications Med Educ 37 2003 267
119. E.M. Ritter, and D. Scott Design of a proficiency-based skills training curriculum for the fundamentals of laparoscopic surgery Surg Innov 14 2007 107
120. E. Hiemstra, W. Kolkman, and F.W. Jansen Skills training in minimally invasive surgery in Dutch obstetrics and gynecology residency curriculum Gynecol Surg 5 2008 321
121. D.J. Scott, E.M. Ritter, and S.T. Tesfay Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training Surg Endosc 8 2008 1887
122. L.S. Feldman, V. Sherman, and G.M. Fried Using simulators to assess laparoscopic competence: Ready for widespread use? Surgery 135 2004 28
123. I. Oropesa, P. Lamata, and P. Sánchez-González Virtual reality simulators for objective evaluation on laparoscopic surgery: Current trends and benefits J. Kim, Virtual reality 2011 Intech Open Access Publisher Rijeka
124. C. Langelotz, M. Kilian, and C. Paul LapSim virtual reality laparoscopic simulator reflects clinical experience in German surgeons Langenbecks Arch Surg 390 2005 534
125. C.Y. Ro, I.K. Toumpoulis, and R.C. Ashton Jr. The LapSim: A learning environment for both experts and novices Stud Health Technol Inform 111 2005 414
126. K.R. van Sickle, E.M. Ritter, and D.A. McClusky III Attempted establishment of proficiency levels for laparoscopic performance on a national scale using simulation: The results from the 2004 SAGES minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) learning center study Surg Endosc 21 2007 5