Crowd-sourced assessment of technical skills: A novel method to evaluate surgical performance

Journal of Surgical Research

Volumn 187, Issue 1, 2014, Pages 65-71

  Chen, Carolyn ^a   White, Lee ^b   Kowalewski, Timothy ^c   Aggarwal, Rajesh ^d   Lintott, Chris ^e   Comstock, Bryan ^f   Kuksenok, Katie ^f   Aragon, Cecilia ^f   Holst, Daniel ^a   Lendvay, Thomas ^f  

^a UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE   (United States)

^b University of Washington   (United States)

^c UNIVERSITY OF MINNESOTA   (United States)

^d UNIVERSITY OF PENNSYLVANIA   (United States)

^e UNIVERSITY OF OXFORD   (United Kingdom)

^f UNIVERSITY OF WASHINGTON   (United States)

Author keywords

Crowdsourcing; Education; GEARS; OSATS; Robotic surgery; Training

Indexed keywords

CROWDSOURCING; EDUCATION; GEARS; OSATS; ROBOTIC SURGERY; TRAINING;

ADULT; COMPETENCY-BASED EDUCATION; CROWDSOURCING; DATA COLLECTION; DEPTH PERCEPTION; EDUCATIONAL MEASUREMENT; GENERAL SURGERY; HUMANS; INTERNET; INTERNSHIP AND RESIDENCY; REFERENCE STANDARDS; ROBOTICS; SUTURE TECHNIQUES; YOUNG ADULT;

EID: 84894042583     PISSN: 00224804     EISSN: 10958673     Source Type: Journal    
DOI: 10.1016/j.jss.2013.09.024     Document Type: Article

References (24)

1. L.T. Kohn, J.M. Corrigan, and M.S. Donaldson To err is human: building a safer health system 2000 National Academies Press Washington, DC
2. C. Zhan, and M.R. Miller Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization JAMA 290 2003 1868
3. R.J. Scalese, V.T. Obeso, and S.B. Issenberg Simulation technology for skills training and competency assessment in medical education J Gen Intern Med 23 Suppl 1 2008 46
4. P.D. van Hove, G.J.M. Tuijthof, E.G.G. Verdaasdonk, L.P. Stassen, and J. Dankelman Objective assessment of technical surgical skills Br J Surg 97 2010 972
5. V. Datta, S. Bann, M. Mandalia, and A. Darzi The surgical efficiency score: a feasible, reliable, and valid method of skills assessment Am J Surg 192 2006 372
6. Quinn AJ, Bederson BB. Human computation: a survey and taxonomy of a growing field. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM, pp. 1403-1412, 2011.
7. J. Surowiecki The wisdom of crowds 1st Anchor books ed. 2005 Anchor Books New York, NY
8. Bernstein MS, Little G, Miller RC, et al. Soylent: a word processor with a crowd inside. Presented at the Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology, 2010.
9. Zaidan OF, Callison-Burch C. Crowdsourcing translation: professional quality from non-professionals. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - vol 1. Stroudsburg, PA: Association for Computational Linguistics, pp. 1220-1229, 2011.
10. Bigham J, Jayant C, Ji H, et al. VizWiz: nearly real-time answers to visual questions. Presented at the Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology 2012.
11. Foldit Wiki. http://foldit.wikia.com/wiki/Foldit-Wiki [accessed 18.04.13].
12. CrowdMed. CrowdMed Beta, https://http://www.crowdmed.com/; 2012 [accessed 18.04.13].
13. Amazon.com I. Amazon Mechanical Turk: artificial artificial intelligence. 2005-2013. https://http://www.mturk.com/mturk/ [accessed 18.04.13].
14. A.C. Goh, D.W. Goldfarb, J.C. Sander, B.J. Miles, and B.J. Dunkin Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills J Urol 187 2012 247
15. G.M. Fried FLS assessment of competency using simulated laparoscopic tasks J Gastrointest Surg 12 2008 210
16. T.S. Lendvay, T.C. Brand, and L. White et al. Virtual reality robotic surgery warm-up improves task performance in a dry laboratory environment: a prospective randomized controlled study J Am Coll Surg 216 2013 1181
17. T.J. Tausch, T.M. Kowalewski, L.W. White, P.S. McDonough, T.C. Brand, and T.S. Lendvay Content and construct validation of a robotic surgery curriculum using an electromagnetic instrument tracker J Urol 188 2012 919
18. T.S. Lendvay, B. Hannaford, and R.M. Satava Future of robotic surgery Cancer J (Sudbury, Mass.) 19 2013 109
19. C.K. Chung, and J.W. Pennebaker The psychological function of function words K. Fiedler, Social Communication 2007 Psychology Press New York 343 359
20. S. Jamieson Likert scales: how to (ab) use them Med Educ 38 2004 1217
21. G. Piaggio, D.R. Elbourne, D.G. Altman, S.J. Pocock, and S.J. Evans Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement JAMA 295 2006 1152
22. R.C. Team R: a language and environment for statistical computing 2008 R Foundation for Statistical Computing Vienna, Austria ISBN 3-900051-07-0. Available from: http://www.R-project.org 2011
23. D. Bertsekas, and J. Tsitsiklis Introduction to probability 2nd ed. 2008 Athena Scientific Belmont, MA
24. Rzeszotarski J, Kittur A. CrowdScape: interactively visualizing user behavior and output. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology. New York, NY: ACM, pp. 55-62, 2012.