Collective intelligence meets medical decision-making: The collective outperforms the best radiologist

PLoS ONE

Volumn 10, Issue 8, 2015, Pages

  Wolf, Max ^a   Krause, Jens ^a,d   Carney, Patricia A ^b   Bogart, Andy ^c   Kurvers, Ralf H J M ^a  

^a LEIBNIZ INSTITUTE OF FRESHWATER ECOLOGY AND INLAND FISHERIES   (Germany)

^b OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^c RAND CORPORATION   (United States)

^d HUMBOLDT UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; AGED; ARTICLE; BREAST CANCER; CANCER REGISTRY; COLLECTIVE INTELLIGENCE; DIAGNOSTIC ACCURACY; FALSE NEGATIVE RESULT; FALSE POSITIVE RESULT; FEMALE; HUMAN; INTELLIGENCE; MAMMOGRAPHY; MEDICAL DECISION MAKING; PROFESSIONAL COMPETENCE; RADIOLOGIST; RESPONSIBILITY; VALIDATION STUDY; BREAST TUMOR; CLINICAL DECISION MAKING; DIAGNOSTIC IMAGING; MALE; MANPOWER; RADIOLOGY; STANDARDS;

BREAST NEOPLASMS; CLINICAL DECISION-MAKING; FEMALE; HUMANS; MALE; MAMMOGRAPHY; RADIOLOGY;

EID: 84942849529     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0134269     Document Type: Article

References (42)

1. Bonabeau E, Dorigo M, Theraulaz G (1999) Swarm Intelligence: From Natural to Artificial Systems. Oxford: Oxford University Press.
2. Surowiecki J (2005) The wisdom of crowds: Random House LLC.
3. Couzin ID (2009) Collective cognition in animal groups. Trends in Cognitive Sciences 13: 36-43. doi:10.1016/j.tics.2008.10.002 PMID: 19058992
4. Krause J, Ruxton GD, Krause S (2010) Swarm intelligence in animals and humans. Trends in Ecology and Evolution 25: 28-34. doi:10.1016/j.tree.2009.06.016 PMID: 19735961
5. Galton F (1907) Vox populi. Nature 75: 450-451.
6. Franks NR, Pratt SC, Mallon EB, Britton NF, Sumpter DJ (2002) Information flow, opinion polling and collective intelligence in house-hunting social insects. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences 357: 1567-1583. PMID: 12495514
7. Arrow KJ, Forsythe R, Gorham M, Hahn R, Hanson R, Ledyard JO, et al. (2008) Economics - The promise of prediction markets. Science 320: 877-878. doi:10.1126/science.1157679 PMID: 18487176
8. Woolley AW, Chabris CF, Pentland A, Hashmi N, Malone TW (2010) Evidence for a Collective Intelligence Factor in the Performance of Human Groups. Science 330: 686-688. doi:10.1126/science.1193147 PMID: 20929725
9. Ward AJW, Herbert-Read JE, Sumpter DJT, Krause J (2011) Fast and accurate decisions through collective vigilance in fish shoals. Proceedings of the National Academy of Sciences of the United States of America 108: 2312-2315. doi:10.1073/pnas.1007102108 PMID: 21262802
10. Eckstein MP, Das K, Pham BT, Peterson MF, Abbey CK, Sy JL, et al. (2012) Neural decoding of collective wisdom with multi-brain computing. Neuroimage 59: 94-108. doi:10.1016/j.neuroimage.2011.07.009 PMID: 21782959
11. Berdahl A, Torney CJ, Ioannou CC, Faria JJ, Couzin ID (2013) Emergent Sensing of Complex Environments by Mobile Animal Groups. Science 339: 574-576. doi:10.1126/science.1225883 PMID: 23372013
12. Ross-Gillespie A, Kümmerli R (2014) Collective decision-making in microbes. Frontiers in microbiology 5.
13. Sorkin RD, Hays CJ, West R (2001) Signal-detection analysis of group decision making. Psychological Review 108: 183-203. PMID: 11212627
14. Bahrami B, Olsen K, Latham PE, Roepstorff A, Rees G, Frith CD (2010) Optimally Interacting Minds. Science 329: 1081-1085. doi:10.1126/science.1185718 PMID: 20798320
15. Barr S, Gold JM (2014) Redundant Visual Information Enhances Group Decisions. Journal of Experimental Psychology-Human Perception and Performance 40: 2124-2130. doi:10.1037/a0038224 PMID: 25365569
16. Ferrell WR, Hillmann BJ, Brewer ML, A mendola MA, Thornbury JR (1989) Interactive, mathematical, and sequential consultative methods in diagnosing renal masses on excretory urograms. Investigative Radiology 24: 456-462. PMID: 2521127
17. Metz CE, Shen J-H (1992) Gains in accuracy from replicated readings of diagnostic images prediction and assessment in terms of ROC analysis. Medical Decision Making 12: 60-75. PMID: 1538634
18. Landman BA, Asman AJ, Scoggins AG, Bogovic JA, Stein JA, Prince JL (2012) Foibles, follies, and fusion: Web-based collaboration for medical image labeling. NeuroImage 59: 530-539. doi:10.1016/j.neuroimage.2011.07.085 PMID: 21839181
19. Siegel R, Ma J, Zou Z, Jemal A (2014) Cancer statistics, 2014. CA: A Cancer Journal for Clinicians 64: 9-29.
20. Metz CE (1978) Basic principles of ROC analysis. Seminars in Nuclear Medicine 8: 283-298. PMID: 112681
21. Swets JA (1988) Measuring the accuracy of diagnostic systems. Science 240: 1285-1293. PMID: 3287615
22. Giordano L, von Karsa L, Tomatis M, Majek O, de Wolf C, Lancucki L, et al. (2012) Mammographic screening programmes in Europe: organization, coverage and participation. Journal of Medical Screening 19: 72-82.
23. http://www.cancer.gov/types/breast/mammograms-fact-sheet.
24. Sorkin RD, West R, Robinson DE (1998) Group performance depends on the majority rule. Psychological Science 9: 456-463.
25. Hastie R, Kameda T (2005) The robust beauty of majority rules in group decisions. Psychological Review 112: 494. PMID: 15783295
26. Wolf M, Kurvers RHJM, Ward AJW, Krause S, Krause J (2013) Accurate decisions in an uncertain world: collective cognition increases true positives while decreasing false positives. Proceedings of the Royal Society B: Biological Sciences 280: 20122777. doi:10.1098/rspb.2012.2777 PMID: 23407830
27. Carney PA, Bogart TA, Geller BM, Haneuse S, Kerlikowske K, Buist DSM, et al. (2012) Association between time spent interpreting, level of confidence, and accuracy of screening mammography. American Journal of Roentgenology 198: 970-978. doi:10.2214/AJR.11.6988 PMID: 22451568
28. Carney PA, Bogart A, Sickles EA, Smith R, Buist DSM, Kerlikowske K, et al. (2013) Feasibility and acceptability of conducting a randomized clinical trial designed to improve interpretation of screening mammography. Academic Radiology 20: 1389-1398. doi:10.1016/j.acra.2013.08.017 PMID: 24119351
29. Geller BM, Bogart A, Carney PA, Elmore JG, Monsees BS, Miglioretti DL (2012) Is Confidence of Mammographic Assessment a Good Predictor of Accuracy? American Journal of Roentgenology 199: W134-W141. doi:10.2214/AJR.11.7701 PMID: 22733922
30. Onega T, Anderson ML, Miglioretti DL, Buist DSM, Geller B, Bogart A, et al. (2013) Establishing a Gold Standard for Test Sets: Variation in Interpretive Agreement of Expert Mammographers. Academic Radiology 20: 731-739. doi:10.1016/j.acra.2013.01.012 PMID: 23664400
31. Geller BM, Bogart A, Carney PA, Sickles EA, Smith R, Monsees B, et al. (2014) Educational Interventions to Improve Screening Mammography Interpretation: A Randomized Controlled Trial. American Journal of Roentgenology 202: W586-W596. doi:10.2214/AJR.13.11147 PMID: 24848854
32. Grofman B, Owen G, Feld SL (1983) Thirteen theorems in search of the truth. Theory and Decision 15: 261-278.
33. Katsikopoulos KV, Martignon L (2006) Naive heuristics for paired comparisons: Some results on their relative accuracy. Journal of Mathematical Psychology 50: 488-494.
34. Kurvers R, Wolf M, Krause J (2014) Humans use social information to adjust their quorum thresholds adaptively in a simulated predator detection experiment. Behavioral Ecology and Sociobiology 68: 449-456.
35. Gromet M (2008) Comparison of computer-aided detection to double reading of screening mammograms: Review of 231,221 mammograms. American Journal of Roentgenology 190: 854-859. doi:10.2214/AJR.07.2812 PMID: 18356428
36. Dromain C, Boyer B, Ferre R, Canale S, Delaloge S, Balleyguier C (2013) Computed-aided diagnosis (CAD) in the detection of breast cancer. European Journal of Radiology 82: 417-423. doi:10.1016/j.ejrad.2012.03.005 PMID: 22939365
37. Dinnes J, Moss S, Melia J, Blanks R, Song F, Kleijnen J (2001) Effectiveness and cost-effectiveness of double reading of mammograms in breast cancer screening: findings of a systematic review. Breast 10: 455-463. PMID: 14965624
38. Harvey SC, Geller B, Oppenheimer RG, Pinet M, Riddell L, Garra B (2003) Increase in cancer detection and recall rates with independent double interpretation of screening mammography. American Journal of Roentgenology 180: 1461-1467. PMID: 12704069
39. Helvie M (2007) Improving mammographic interpretation: double reading and computer-aided diagnosis. Radiologic Clinics of North America 45: 801-811. PMID: 17888770
40. Shaw CM, Flanagan FL, Fenlon HM, McNicholas MM (2009) Consensus review of discordant findings maximizes cancer detection rate in double-reader screening mammography: Irish National Breast Screening Program experience. Radiology 250: 354-362. doi:10.1148/radiol.2502080224 PMID: 19188311
41. Kerr NL, Tindale RS (2004) Group performance and decision making. Annual Review of Psychology 55: 623-655. PMID: 14744229
42. Bankier AA, Levine D, Halpern EF, Kressel HY (2010) Consensus interpretation in imaging research: is there a better way? Radiology 257: 14-17. doi:10.1148/radiol.10100252 PMID: 20851935