Review of methods for objective surgical skill evaluation

Surgical Endoscopy

Volumn 25, Issue 2, 2011, Pages 356-366

  Reiley, Carol E ^a   Lin, Henry C ^a   Yuh, David D ^b,c   Hager, Gregory D ^a  

^a JOHNS HOPKINS UNIVERSITY   (United States)

^b JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c DEPARTMENT OF PATHOLOGY   (United States)

Author keywords

Human robotic training; Surgical training courses

Indexed keywords

ARM MOVEMENT; CLINICAL PRACTICE; COMPUTER ASSISTED SURGERY; COMPUTER SIMULATION; EDUCATION PROGRAM; HIDDEN MARKOV MODEL; HUMAN; LAPAROSCOPIC SURGERY; MAN MACHINE INTERACTION; MINIMALLY INVASIVE SURGERY; MOTION ANALYSIS SYSTEM; PERFORMANCE MEASUREMENT SYSTEM; PERSONNEL MANAGEMENT; PRIORITY JOURNAL; PROFESSIONAL COMPETENCE; RESIDENCY EDUCATION; REVIEW; ROBOTICS; STATISTICAL MODEL; SURGICAL TECHNOLOGY; SURGICAL TRAINING; TASK PERFORMANCE; VIRTUAL REALITY;

EID: 79954589813     PISSN: 09302794     EISSN: 14322218     Source Type: Journal    
DOI: 10.1007/s00464-010-1190-z     Document Type: Review

References (78)

1. Reznick RK, MacRae H (2006) Teaching surgical skills-changes in the wind. N Engl J Med 355(25):2664-2669
2. DeFrances CJ, Hall MJ (2005) National hospital discharge survey. Adv Data Vital Health Stat 385:1-19
3. Anderson P, GelijnsA, Moskowitz A, Arons R, GuptaL, Weinberg A, Faries P, Nowygrod R, Kent K (2004) Understanding trends in inpatient surgical volume: vascular interventions, 1980-2000. J Vasc Surg 39(6):1200-1208 (Pubitemid 38952586) (Pubitemid 38952586)
4. Zhan C, Miller M (2003) Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization. JAMA 290(14):1868-1874 (Pubitemid 37430603) (Pubitemid 37430603)
5. Peracchia A (2001) Surgical education in the third millennium. Ann Surg 234(6):709-712 (Pubitemid 33117190) (Pubitemid 33117190)
6. Carter BN (1952) The fruition of Halsted's concept of surgical training. Surgery 32:518-527
7. Darzi A, Smith S, Taffinder N (1999) Assessing operative skill needs to become more objective. Br Med J 318:887-888
8. Barden CB, Specht MC, McCarter MD, Daly JM, Fahey TJ (2003) Effects of limited work hours on surgical training. Obstet Gynecol Surv 58(4):244-245
9. Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domain. Acad Med 79:70-81
10. Bann SD, Datta VK, Khan MS, Ridgway PF, Darzi AW (2005) Attitudes towards skills examinations for basic surgical trainees. Int J Clin Pract 59(1):107-113 (Pubitemid 40260123) (Pubitemid 40260123)
11. Kopta JA (1971) An approach to the evaluation of operative skills. Surgery 70:297-303
12. Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165:358-361
13. Lin HC, Shafran I, Yuh DD, Hager GD (2006) Towards automatic skill evaluation: detection and segmentation of robotassisted surgical motions. Comp Aid Surg 11(5):220-223
14. Bridgewater B, Grayson AD, Jackson M, Brooks N, Grotte GJ, Keenan DJM, Millner R, Fabri BM, Jones M (2003) Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. BMJ 327:13-17 (Pubitemid 36801883) (Pubitemid 36801883)
15. Moorthy K, Munz Y, Sarker SK, Darzi A (2003) Objective assessment of technical skills in surgery. Br Med Assoc 327: 1032-1037 (Pubitemid 37357470) (Pubitemid 37357470)
16. Reznick R, Regehr G, MacRae H, Martin J, McCulloch W (1997) Testing technical skill via an innovative bench station examination. Am J Surg 173(3):226-230 (Pubitemid 27163850)
17. Doyle J, Webber E, Sidhu R (2007) A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 193(5):551-555 (Pubitemid 46559902) (Pubitemid 46559902)
18. Neumann M, Hahn C, Horbach T, Schneide I, Meining A, Heldwein W, Rsch T (2003) Score card endoscopy: a multicenter study to evaluate learning curves in 1-week courses using the Erlangen endotrainer. Endoscopy 35:515-520 (Pubitemid 36705644) (Pubitemid 36705644)
19. Moorthy K, Munz Y, Dosis A, Hernandez J, Martin S, Bello F, Rockall T, Darzi A (2004) Dexterity enhancement with robotic surgery. Surg Endosc 18:790-795 (Pubitemid 38650061) (Pubitemid 38650061)
20. Shah J, Darzi A (2001) Surgical skills assessment: an ongoing debate. BJU Int 88(7):655-660
21. Rosenbaum DA (1992) Reaching and grasping. Human Motor Control 197-225
22. Chmarra MK, Grimbergen CA, Dankelman J (2007) Systems for tracking minimally invasive surgical instruments. Min Inv Ther All Tech 16(6):328-340 (Pubitemid 350176884) (Pubitemid 350176884)
23. Riedl S (2002) Modern operating room management in the workflow of surgery, spectrum of tasks and challenges of the future. Anaesthesist 73:105-110
24. Thompson CA (2004) Jcaho gears up to survey sterile compounding practices. Am J Health, Syst Pharm
25. Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135:21-27 (Pubitemid 38041450) (Pubitemid 38041450)
26. Figert PL, Park AE, Witzke DB, Schwartz RW (2001) Transfer of training in acquiring laparoscopic skills. J Am Coll Surg 193(5): 533-537 (Pubitemid 33043457) (Pubitemid 33043457)
27. Moltenbrey K (2006) Tools of the trade. Comp Graph World 29(11)
28. 5DT Data Glove Ulta
29. Francis N (2002) The performance of master surgeons on the advanced Dundee endoscopic psychomotor tester. Arch Surg 137: 841-844 (Pubitemid 34743177) (Pubitemid 34743177)
30. Datta V, Mackay S, Mandalia M, Darzi A (2001) The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in laboratory-based model. J Am Coll Surg 193:479-485 (Pubitemid 33043449) (Pubitemid 33043449)
31. Taffinder N, Smith S, Mair J, Russel RC, Darzi A (1999) Can a computer measure surgical precision? Reliability, validity and feasibility of the ICSAD. Surg Endosc 13(1):81
32. Xin H, Zelek JS, Carnahan H (2006) Laparoscopic surgery, perceptual limitations and force: a review. In: First joint eurohaptics conference and symposium on haptic interfaces for virtual environment
33. Guthart GS, Salisbury JK (2000) The intuitive telesurgery system: overview and application. In: IEEE International conference on robotics automation, pp 618-621
34. Intuitive Surgical Inc (2007) Intuitive surgical annual report
35. Nawrat Z, Kostka P (2007) The robin heart vision, telemanipulator for camera holding - preliminary test results. J Automat Mobile Robotics Intelligent Syst 1(1):48-53
36. Mayer H, Gomex F, Wierstra D, Nagy I, Knoll A, Schmidhuber J (2006) A system for robotic heart surgery that learns to tie knots using recurrent neural networks. ICRA, pp 543-548
37. Global Strategic Business (2007) Computer assisted surgical systems. A global strategic business report
38. Richards C, Rosen J, Hannaford B, Pellegrini C, Sinanan M (2000) Skills evaluation in minimally invasive surgery using force/torque signatures. Surg Endosc 14:791-798
39. Rosen J, Brown JD, Chang L, Sinanan MN, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Trans Biomed Eng 53(3):399-413 (Pubitemid 43288817) (Pubitemid 43288817)
40. Pugh CM, Youngblood P (2002) Development and validation of assessment measures for a newly developed physical examination simulator. J Am Med Inform Assoc 9(5):448-460 (Pubitemid 35024450) (Pubitemid 35024450)
41. Ko S, Kwon D (2004) A surgical knowledge based interaction method for a laparoscopic assistant robot. In: RO-MAN - IEEE, International workshop on robot and human interactive communication, pp 313-318 (Pubitemid 40822687) (Pubitemid 40822687)
42. Ibbotson JA, MacKenzie CL, Cao CG, Lomax AJ (1999) Gaze patterns in laparoscopic surgery. In: MMVR, pp 154-160
43. Rosen J, Hannaford B, Richards CG, Sinanan MN (2001) Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills. IEEE Trans Biomed Eng 48(5):579-591 (Pubitemid 32378059) (Pubitemid 32378059)
44. Yamauchi Y, Yamashita J, Morikawa O, Hashimoto R (2002) Surgical skill evaluation by force data for endoscopic sinus surgery training system. In: MICCAI, pp 44-51
45. Judkins TN, Oleynikov D, Stergiou N (2008) Objective evaluation of expert performance during human robotic surgical procedures. J Robotics Surg 1(4):307-312
46. Van Sickle KR, McClusky DA, Gallagher AG, Smith CD (2005) Construct validation of the promise simulator using a novel laparoscopic suturing task. Surg Endosc 19(9):1227-1231 (Pubitemid 41558418) (Pubitemid 41558418)
47. Acosta E, Temkin B (2005) Haptic laparoscopic skills trainer with practical user evaluation metrics. MMVR 111:8-11
48. Hernandez JD, Bann SD, Munz Y, Moorthy K, Datta V (2004) Qualitative and quantitative analysis of the learning curve of a simulated surgical task on the da Vinci system. Surg Endosc 372-378 (Pubitemid 38524501) (Pubitemid 38524501)
49. Rosen J, Solazzo M, Hannaford B, Sinanan MN (2002) Task decomposition of laparoscopic surgery for objective evaluation of surgical residents' learning curve using hidden Markov model. Comp Aid Surg 7(1):49-61 (Pubitemid 34601715) (Pubitemid 34601715)
50. Cao CG, MacKenzie RD, Payandeh S (1996) Task and motion analyses in endoscopic surgery. In: ASME IMECE, pp 583-590
51. Padoy N, Blum T, Essa IA, Feussne H, Berger MO, Navab N (2007) A boosted segmentation method for surgical workflow analysis. In: MICCAI, pp 102-109
52. Datta V, Bann S, Hernandez J, Darzi A (2007) Objective assessment comparing hand-assisted and conventional laparoscopic surgery. Surg Endosc 21:414-417
53. Datta V, Mandalia M, Mackay S, Chang A, Cheshire N, Darzi A (2002) Relationship between skill and outcome in the laboratory based model. Surgery 131:318-323 (Pubitemid 34229785) (Pubitemid 34229785)
54. Fried GM, Feldman LS, Vassiliou MC, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518-528 (Pubitemid 39121346) (Pubitemid 39121346)
55. Acosta E, Temkin B (2005) Dynamic generation of surgery specific simulators: a feasibility study. StudHealth Technol Inform 111:1-7
56. Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2002) Assessment of technical surgical skills. Eur J Surg 168:139-144 (Pubitemid 34712956) (Pubitemid 34712956)
57. Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A (2004) Laparoscopic virtual reality and box trainers: is one superior to the other? Surg Endosc 18:485-494 (Pubitemid 38524523) (Pubitemid 38524523)
58. Cotin S, Stylopoulos N, Ottensmeyer M, Neumann P, Rattner D, Dawson S (2002) Metrics for laparoscopic skills trainers: the weakest link. MICCAI 2488:35-43
59. O'Toole RV, Playter RR, Krummel TM, Blank WC, Cornelius NH, Roberts WR, Bell WJ, Raibert M (1999) Measuring and developing suturing technique with a VR surgical simulator. Am Coll Surg 189(1):114-127
60. Liu A, Tendick F, Cleary K, Kaufmann C (2003) A survey of surgical simulation: applications, technology, and education. Presence Teleop Virt Environ 12(6):599-614 (Pubitemid 38415107) (Pubitemid 38415107)
61. Issenberg SB, McGaghie WC (2002) Clinical skills training practice makes perfect. Med Educat 36(3):210-211 (Pubitemid 34227236) (Pubitemid 34227236)
62. Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241(2):364-372
63. Seymour NE, Anthony G, Sanziana A, O'Brien MK, Bansal VK, Andersen DK (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236(4):458-464
64. Chaudhry A, Sutton C, Wood J, Stone R, McCloy R (1999) Learning rate for laparoscopic surgical skills on mist VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl 81:281286
65. Nagy I, Mayer H, Knoll A (2003) The Endopar system for minimally invasive robotic surgery. Technical report, TUM
66. MacKenzie CL, Ibbotson JA, Cao CLGL, Lomax AJ (2001) Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment. Minim Invas Ther Allied Technol 10(3):121-127 (Pubitemid 32586261) (Pubitemid 32586261)
67. Ahmadi SA, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2006) Recovery of surgical workflow without explicit models. Med Imag Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv 9(1):420-428 (Pubitemid 44609133) (Pubitemid 44609133)
68. Mackel T, Rosen J, Pugh C (2007) Application of hidden Markov modeling to objective medical skill evaluation. Stud Health Technol Inform 125:316-318
69. Leong JH, Nicolaou M, Atallah L, Mylonas GP, Darzi AW, Yang G (2006) Hmm assessment of quality of movement trajectory in laparoscopic surgery. MICCAI 4190:752-759 (Pubitemid 44609173) (Pubitemid 44609173)
70. Speidel S, Delles ML, Gutt C, Dillmann R (2006) Tracking of instruments in minimally invasive surgery for surgical skill analysis. Med Imag Augment Real 4091:148-155 (Pubitemid 44502811) (Pubitemid 44502811)
71. Hundtofte CS, Hager GD, Okamura AM (2002) Building a task language for segmentation and recognition of user input to cooperative manipulation systems. Haptic Interf Virt Environ Teleop Syst, pp 225-230
72. Murphy TE, Vignes CM, Yuh DD, Okamura AM (2003) Automatic motion recognition and skill evaluation for dynamic tasks. Eurohaptics 363-373
73. Reiley CE, Lin HC, Varadarajan B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Med Virt Real 132:396-401
74. Castellani A, Botturi D, Bicego M, Fiorini P (2004) Hybrid hmm/svm model for the analysis and segmentation of teleoperation tasks. IEEE Int Conf Robotics Autom 3:2918-2923
75. Li M, Okamura AM (2003) Recognition of operator motions for realtime assistance using virtual fixtures. Int Sympos Haptic Interf Virt Environ Teleop Syst 1954-1959
76. Kragic D, Marayong P, Li M, Okamura AM, Hager GD (2005) Human-machine collaborative systems for microsurgical applications. Int J Robotics Res 24(9):731-741 (Pubitemid 41363638) (Pubitemid 41363638)
77. Intrafocus. http://www.intra-focus.com
78. Okamura AM (2004 Methods for haptic feedback in teleoperated robot-assisted surgery. Industr Robot Int J 31:499-508