Detection accuracy of collective intelligence assessments for skin cancer diagnosis

JAMA Dermatology

Volumn 151, Issue 12, 2015, Pages 1346-1353

  Kurvers, Ralf H J M ^a,b   Krause, Jens ^a,c   Argenziano, Giuseppe ^d   Zalaudek, Iris ^e   Wolf, Max ^a  

^a LEIBNIZ INSTITUTE OF FRESHWATER ECOLOGY AND INLAND FISHERIES   (Germany)

^b MAX PLANCK INSTITUTE FOR HUMAN DEVELOPMENT   (Germany)

^c HUMBOLDT UNIVERSITY   (Germany)

^d SECOND UNIVERSITY OF NAPLES   (Italy)

^e MEDICAL UNIVERSITY OF GRAZ   (Austria)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; CANCER DIAGNOSIS; COLLECTIVE INTELLIGENCE; CONTROLLED STUDY; DIAGNOSTIC ACCURACY; DIAGNOSTIC TEST ACCURACY STUDY; FALSE POSITIVE RESULT; HISTOPATHOLOGY; HUMAN; IMAGE ANALYSIS; IMAGE QUALITY; INTELLIGENCE; INTERNET; PRIORITY JOURNAL; SENSITIVITY AND SPECIFICITY; SKIN CANCER;

EID: 84949655573     PISSN: 21686068     EISSN: None     Source Type: Journal    
DOI: 10.1001/jamadermatol.2015.3149     Document Type: Article

References (42)

1. Garbe C, Leiter U. Melanoma epidemiology and trends. Clin Dermatol. 2009;27(1):3-9.
2. Mayer JE, Swetter SM, Fu T, Geller AC. Screening, early detection, education, and trends for melanoma: current status (2007-2013) and future directions, part I: epidemiology, high-risk groups, clinical strategies, and diagnostic technology. J Am Acad Dermatol. 2014;71(4):599.e1-599.e12. doi:10 .1016/j.jaad.2014.05.046.
3. Mayer JE, Swetter SM, Fu T, Geller AC. Screening, early detection, education, and trends for melanoma: current status (2007-2013) and future directions, part II: screening, education, and future directions. J AmAcad Dermatol. 2014;71(4):611.e1-611.e10. doi:10.1016/j.jaad.2014.05.045.
4. Kittler H, Pehamberger H,Wolff K, Binder M. Diagnostic accuracy of dermoscopy. Lancet Oncol. 2002;3(3):159-165.
5. BafountaML, Beauchet A, Aegerter P, Saiag P. Is dermoscopy (epiluminescence microscopy) useful for the diagnosis of melanoma? results of a meta-analysis using techniques adapted to the evaluation of diagnostic tests. Arch Dermatol. 2001; 137(10):1343-1350.
6. Ascierto PA, Satriano RA, Palmieri G, Parasole R, Bosco L, Castello G. Epiluminescence microscopy as a useful approach in the early diagnosis of cutaneous malignant melanoma. Melanoma Res. 1998;8(6):529-537.
7. VestergaardME, Macaskill P, Holt PE, Menzies SW. Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: ameta-analysis of studies performed in a clinical setting. Br J Dermatol. 2008;159(3):669-676.
8. Rajpara SM, Botello AP, Townend J, Ormerod AD. Systematic review of dermoscopy and digital dermoscopy/ artificial intelligence for the diagnosis of melanoma. Br J Dermatol. 2009;161(3):591-604.
9. Rubegni P, Burroni M, Cevenini G, et al. Digital dermoscopy analysis and artificial neural network for the differentiation of clinically atypical pigmented skin lesions: a retrospective study. J Invest Dermatol. 2002;119(2):471-474.
10. Ganster H, Pinz A, Röhrer R, Wildling E, Binder M, Kittler H. Automated melanoma recognition. IEEE Trans Med Imaging. 2001;20(3):233-239.
11. Garbe C, Eigentler TK. Diagnosis and treatment of cutaneous melanoma: state of the art 2006. Melanoma Res. 2007;17(2):117-127.
12. Krause J, Ruxton GD, Krause S. Swarm intelligence in animals and humans. Trends Ecol Evol. 2010;25(1):28-34.
13. Bonabeau E, DorigoM, Theraulaz G. Swarm Intelligence: From Natural to Artificial Systems. Oxford, England: Oxford University Press; 1999.
14. Couzin ID. Collective cognition in animal groups. Trends Cogn Sci. 2009;13(1):36-43.
15. Woolley AW, Chabris CF, Pentland A, Hashmi N, Malone TW. Evidence for a collective intelligence factor in the performance of human groups. Science. 2010;330(6004):686-688.
16. Surowiecki J. TheWisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. New York, NY: Knopf Doubleday Publishing Group; 2004.
17. Arrow KJ, Forsythe R, Gorham M, et al. Economics: the promise of prediction markets. Science. 2008;320(5878):877-878.
18. Climent RJ, Krause S, von Engelhardt N, Faria JJ, Krause J, Kurvers RH. Collective cognition in humans: groups outperform their best members in a sentence reconstruction task. PLoS One. 2013;8 (10):e77943. doi:10.1371/journal.pone.0077943.
19. King AJ, Gehl RW, Grossman D, Jensen JD. Skin self-examinations and visual identification of atypical nevi: comparing individual and crowdsourcing approaches. Cancer Epidemiol. 2013;37(6):979-984.
20. Argenziano G, Soyer HP, Chimenti S, et al. Dermoscopy of pigmented skin lesions: results of a consensus meeting via the Internet. J AmAcad Dermatol. 2003;48(5):679-693.
21. Zalaudek I, Argenziano G, Soyer HP, et al; DermoscopyWorking Group. Three-point checklist of dermoscopy: an open Internet study. Br J Dermatol. 2006;154(3):431-437.
22. Sumpter DJ, Pratt SC. Quorum responses and consensus decision making. Philos Trans R Soc B Biol Sci. 2009;364(1518):743-753.
23. Hastie R, Kameda T. The robust beauty of majority rules in group decisions. Psychol Rev. 2005;112(2):494-508.
24. Sorkin RD,West R, Robinson DE. Group performance depends on the majority rule. Psychol Sci. 1998;9(6):456-463.
25. WolfM, Kurvers RH,Ward AJ, Krause S, Krause J. Accurate decisions in an uncertain world: collective cognition increases true positives while decreasing false positives. Proc R Soc B. 2013;280 (1756):20122777. doi:10.1098/rspb.2012.2777.
26. Kurvers RJM,Wolf M, Krause J. Humans use social information to adjust their quorum thresholds adaptively in a simulated predator detection experiment. Behav Ecol Sociobiol. 2014; 68(3):449-456.
27. Swets JA. The science of choosing the right decision threshold in high-stakes diagnostics. Am Psychol. 1992;47(4):522-532.
28. Swets JA, Dawes RM, Monahan J. Psychological science can improve diagnostic decisions. Psychol Sci Public Interest. 2000;1(1):1-26.
29. Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240(4857):1285-1293.
30. Argenziano G, Fabbrocini G, Carli P, De Giorgi V, Sammarco E, Delfino M. Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions: comparison of the ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis. Arch Dermatol. 1998;134 (12):1563-1570.
31. Conradt L, List C, Roper TJ. Swarm intelligence: when uncertainty meets conflict. Am Nat. 2013;182 (5):592-610.
32. Kao AB, Couzin ID. Decision accuracy in complex environments is often maximized by small group sizes [published online April 23, 2014]. Proc R Soc B. doi:10.1098/rspb.2013.3305.
33. Krause S, James R, Faria JJ, Ruxton GD, Krause J. Swarm intelligence in humans: diversity can trump ability. Anim Behav. 2011;81(5):941-948.
34. Luan S, Katsikopoulos KV, Reimer T. When does diversity trump ability (and vice versa) in group decision making? a simulation study. PLoS One. 2012;7(2):e31043.
35. Hukkinen K, Kivisaari L, Vehmas T. Impact of the number of readers on mammography interpretation. Acta Radiol. 2006;47(7):655-659.
36. Duijm LEM, LouwmanMWJ, Groenewoud JH, van de Poll-Franse LV, Fracheboud J, Coebergh JW. Inter-observer variability in mammography screening and effect of type and number of readers on screening outcome. Br J Cancer. 2009;100(6): 901-907.
37. Farnetani F, Scope A, Braun RP, et al. Skin cancer diagnosis with reflectance confocal microscopy: reproducibility of feature recognition and accuracy of diagnosis [published onlineMay 20, 2015]. JAMA Dermatol. doi:10.1001 /jamadermatol.2015.0810.
38. WolfM, Krause J, Carney PA, Bogart A, Kurvers RHJM. Collective intelligence meets medical decision making: the collective outperforms the best radiologist [published online August 12, 2015]. PLoS One. doi:10.1371/journal.pone.0134269.
39. Massone C, Brunasso AMG, Hofmann-Wellenhof R, Gulia A, Soyer HP. Teledermoscopy: education, discussion forums, teleconsulting and mobile teledermoscopy. G Ital Dermatol Venereol. 2010;145(1):127-132.
40. Massone C, Hofmann-Wellenhof R, Ahlgrimm-Siess V, Gabler G, Ebner C, Soyer HP. Melanoma screening with cellular phones. PLoS One. 2007;2(5):e483. doi:10.1371/journal.pone.0000483.
41. Kroemer S, Frihauf J, Campbell TM, et al. Mobile teledermatology for skin tumour screening: diagnostic accuracy of clinical and dermoscopic image tele-evaluation using cellular phones. Br J Dermatol. 2011;164(5):973-979.
42. Ebner C,Wurm EMT, Binder B, et al. Mobile teledermatology: a feasibility study of 58 subjects using mobile phones. J Telemed Telecare. 2008;14 (1):2-7.