Ground truth generation in medical imaging: A crowdsourcing-based iterative approach

CrowdMM 2012 - Proceedings of the 2012 ACM Workshop on Crowdsourcing for Multimedia, Co-located with ACM Multimedia 2012

Volumn , Issue , 2012, Pages 9-14

  Foncubierta Rodríguez, Antonio ^a   Müller, Henning ^a  

^a UNIVERSITY OF APPLIED SCIENCES WESTERN SWITZERLAND   (Switzerland)

Author keywords

Crowdsourcing; Ground truth; Information classification and retrieval

Indexed keywords

AUTOMATIC CLASSIFICATION; AUTOMATIC SYSTEMS; COMPUTER BASED TOOLS; CROWDSOURCING; DATA SETS; DOMAIN EXPERTS; FEED-BACK LOOP; GROUND TRUTH; HIGH QUALITY; HUMAN INTELLIGENCE; IMAGE MODALITY; INFORMATION CLASSIFICATION; ITERATIVE APPROACH; MANUAL ANNOTATION; MEDICAL DOCTORS; MEDICAL IMAGES; SEMI-AUTOMATIC TOOLS; SPECIFIC TASKS;

MEDICAL IMAGING; TEXT PROCESSING;

QUALITY CONTROL;

EID: 84870551497     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2390803.2390808     Document Type: Conference Paper

References (17)

1. C. Akgül, D. Rubin, S. Napel, C. Beaulieu, H. Greenspan, and B. Acar. Content-based image retrieval in radiology: Current status and future directions. Journal of Digital Imaging, 24(2):208-222, 2011.
2. O. Alonso and S. Mizzaro. Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment. In SIGIR 2009 Workshop on The Future of IR Evaluation, 2009.
3. O. Alonso, D. E. Rose, and B. Stewart. Crowdsourcing for relevance evaluation. SIGIR Forum, 42(2):9-15, December 2008.
4. C. Arning and U. Grzyska. Color doppler imaging of cervicocephalic fibromuscular dysplasiâaǍć. Color Doppler imaging of cervicocephalic fibromuscular dysplasia, 2, 2004.
5. B. Caputo, H. Müller, T. S. Mahmood, J. Kalpathy-Cramer, F. Wang, and J. Duncan. Editorial of miccai workshop proceedings on medical content-based retrieval for clinical decision support. In Proceedings on MICCAI Workshop on Medical Content-based Retrieval for Clinical Decision Support, volume 5853 of Lecture Notes in Computer Science (LNCS). Springer, 2009.
6. A. Depeursinge, H. Greenspan, T. Syeda Mahmood, and H. Müller. Overview of the second workshop on medical content-based retrieval for clinical decision support. In H. Greenspan, H. Müller, and T. Syeda Mahmood, editors, Medical Content-based Retrieval for Clinical Decision Support, MCBR-CDS 2011. Lecture Notes in Computer Sciences (LNCS), Sept. 2011.
7. S. Gupta, S. D. Shenoi, and V. Mehta. Acquired Lymphangioma Of Vulva Secondary To Radiotherapy For Carcinoma Cervix. Indian Journal of Dermatology, 53(4):221-222, 2008.
8. J. Howe. Crowdsourcing. http://www.crowdsourcing.com/cs, 2012. [Online; accessed 29-Jun-2012].
9. Y. T. Hwang, A. W. Mccartney, S. K. Gidda, and R. T. Mullen. Localization of the Carnation Italian ringspot virus replication protein p36 to the mitochondrial outer membrane is mediated by an internal targeting signal and the TOM complex. BMC Cell Biology, 9:54+, 2008.
10. J. Kalpathy-Cramer, H. Müller, S. Bedrick, I. Eggel, A. G. Seco de Herrera, and T. Tsikrika. The CLEF 2011 medical image retrieval and classification tasks. In Working Notes of CLEF 2011 (Cross Language Evaluation Forum), September 2011.
11. H. Müller, J. Kalpathy-Cramer, D. Demner-Fushman, and S. Antani. Creating a classification of image types in the medical literature for visual categorization. In SPIE medical imaging, 2012.
12. H. Müller, N. Michoux, D. Bandon, and A. Geissbuhler. A review of content-based image retrieval systems in medicine-clinical benefits and future directions. International Journal of Medical Informatics, 73(1):1-23, 2004. (Pubitemid 38229184)
13. T. B. Nguyen, S. Wang, V. Anugu, N. Rose, M. McKenna, N. Petrick, J. E. Burns, and R. M. Summers. Distributed human intelligence for colonic polyp classification in computer-aided detection for CT colonography. Radiology, 262(3):824-828, March 2012.
14. S. Nowak and S. Rüger. How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation. In Proceedings of the international conference on Multimedia information retrieval, MIR '10, pages 557-566, New York, NY, USA, 2010. ACM.
15. J. Staab, T. O'Connell, and S. Gomez. Enhancing metabolomic data analysis with Progressive Consensus Alignment of NMR Spectra (PCANS). BMC Bioinformatics, 11(1):123+, 2010.
16. H. Tang and J. H. K. Ng. Googling for a diagnosis - use of Google as a diagnostic aid: Internet based study. British Medical Journal, 333:1283-1284, 2006.
17. J. Thiele, R. Nimmo, W. Rowell, S. Quinn, and G. Jones. A randomized single blind crossover trial comparing leather and commercial wrist splints for treating chronic wrist pain in adults. BMC Musculoskeletal Disorders, 10:129+, 2009.