Assessing the effect of visualizations on bayesian reasoning through crowdsourcing

IEEE Transactions on Visualization and Computer Graphics

Volumn 18, Issue 12, 2012, Pages 2536-2545

  Micallef, Luana ^a   Dragicevic, Pierre ^b   Fekete, Jean Daniel ^b  

^a UNIVERSITY OF KENT   (United Kingdom)

^b INRIA   (France)

Author keywords

base rate fallacy; Bayesian reasoning; crowdsourcing; Euler diagrams; glyphs; probabilistic judgment

Indexed keywords

BASE RATE; BAYESIAN REASONING; CROWDSOURCING; EULER DIAGRAMS; GLYPHS; PROBABILISTIC JUDGMENT;

EXPERIMENTS;

VISUALIZATION;

BAYES THEOREM; COMPREHENSION; COMPUTER GRAPHICS; CROWDSOURCING; FEMALE; HUMAN; MALE; MAMMOGRAPHY; VISION;

BAYES THEOREM; COMPREHENSION; COMPUTER GRAPHICS; CROWDSOURCING; FEMALE; HUMANS; MALE; MAMMOGRAPHY; VISUAL PERCEPTION;

EID: 84867765634     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2012.199     Document Type: Article

References (58)

1. I. Ajzen. Intuitive theories of events and the effects of base-rate information on prediction. Personality and Social Psychology, 35(5):303-314, 1977.
2. J. S. Ancker, E. U.Weber, and R. Kukafka. Effect of arrangement of stick figures on estimates of proportion in risk graphics. Medical Decision Making, 31(1):143-150, 2011.
3. M. Bar-Hillel. The base-rate fallacy in probability judgments. Acta Psychologica, 44(3):211-233, 1980.
4. A. K. Barbey and S. A. Sloman. Base-rate respect: From ecological rationality to dual processes. Behavioral and Brain Sciences, 30(3):241- 254; discussion 255-297, 2007.
5. A. K. Barbey and S. A. Sloman. Base-rate respect: From statistical formats to cognitive structures. Behavioral and Brain Sciences, 30(3):287- 297, 2007.
6. M. I. Bauer and P. N. Johnson-Laird. How Diagrams Can Improve Reasoning. Psychological Science, 4(6):372-378, 1993.
7. F. Benoy and P. Rodgers. Evaluating the Comprehension of Euler Diagrams. In Conf Information Visualization (IV), pages 771-778, 2007. (Pubitemid 47591821)
8. G. L. Brase. Pictorial representations in statistical reasoning. Applied Cognitive Psychology, 23(3):369-381, 2009.
9. S. Brown, J. Culver, K. Osann, D. MacDonald, S. Sand, A. Thornton, M. Grant, D. Bowen, K. Metcalfe, H. Burke, M. Robson, S. Friedman, and J. Weitzel. Health literacy, numeracy, and interpretation of graphical breast cancer risk estimates. Patient Education and Counseling, 83(1):92-98, 2011.
10. K. Burns. Painting pictures to augment advice. In Conf Advanced Visual Interfaces, pages 344-349, 2004.
11. G. B. Chapman and J. Liu. Numeracy, frequency, and Bayesian reasoning. Judgment and Decision Making, 4(1):34-40, 2009.
12. J. J. Chen, N. J. Menezes, A. D. Bradley, and T. A. North. Opportunities for Crowdsourcing Research on Amazon Mechanical Turk. Human Factors, 5:3, 2011.
13. Chicago Tribune. A false hiv test caused 18 months of hell, (3/5/1993).
14. J. Cohen. The Earth is Round (p < .05). American Psychologist, 49(12):997-1003, 1994.
15. W. G. Cole. Understanding Bayesian reasoning via graphical displays. In ACM Conf Human Factors in Computing Systems, pages 381-386, 1989.
16. W. G. Cole and J. E. Davidson. Graphic Representation Can Lead To Fast and Accurate Bayesian Reasoning. In Symp Computer Application in Medical Care, pages 227-231, 1989.
17. C. Collins, G. Penn, and S. Carpendale. Bubble sets: revealing set relations with isocontours over existing visualizations. IEEE Transactions on Visualization and Computer Graphics, 15(6):1009-1016, 2009.
18. L. Cosmides and J. Tooby. Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty. Cognition, 58(1), 1996.
19. T. C. Davis, N. C. Dolan, M. R. Ferreira, C. Tomori, K. W. Green, A. M. Sipler, and C. L. Bennett. The role of inadequate health literacy skills in colorectal cancer screening. Cancer Investigation, 19(2):193-200, 2001. (Pubitemid 32233791)
20. J. G. Dolan and S. Iadarola. Risk communication formats for low probability events: an exploratory study of patient preferences. BMC Medical Informatics and Decision Making, 8(14):14, 2008.
21. D. M. Eddy. Probabilistic reasoning in clinical medicine: problems and opportunities, pages 249-267. Cambridge University Press, 1982.
22. R. B. Ekstrom, J. W. French, H. H. Harman, and D. Dermen. Manual for Kit of Factor-Referenced Cognitive Tests. Educational Testing Service, 1976.
23. A. Fagerlin, B. J. Zikmund-Fisher, P. A. Ubel, A. Jankovic, H. A. Derry, and D. M. Smith. Measuring numeracy without a math test: development of the Subjective Numeracy Scale. Medical Decision Making, 27(5):672- 80, 2007. (Pubitemid 47547594)
24. M. Galesic, R. Garcia-Retamero, and G. Gigerenzer. Using Icon Arrays to Communicate Medical Risks: Overcoming Low Numeracy. Health Psychology, 28(2):210-216, 2009.
25. G. Gigerenzer. Ecological intelligence: An adaptation for frequencies, volume 39, pages 9-29. Oxford University Press, 1998.
26. G. Gigerenzer, W. Gaissmaier, E. Kurz-Milcke, L. M. Schwartz, and S. Woloshin. Helping Doctors and Patients Make Sense of Health Statistics. Psychological Science in the Public Interest, 8(2):53-96, 2007.
27. G. Gigerenzer and U. Hoffrage. How to Improve Bayesian Reasoning Without Instruction: Frequency Formats. Psychological Review, 102(4):684-704, 1995.
28. V. Girotto and M. Gonzalez. Solving probabilistic and statistical problems: a matter of information structure and question form. Cognition, 78(3):247-276, 2001.
29. I. J. Good. When batterer turns murderer. Nature, 375(6532):541, 1995.
30. J. Heer and M. Bostock. Crowdsourcing Graphical Perception: Using Mechanical Turk to Assess Visualization Design. In ACM Conf Human Factors in Computing Systems, pages 203-212, 2010.
31. U. Hoffrage and G. Gigerenzer. How to Improve the Diagnostic Inferences of Medical Experts. Experts in Science and Society, pages 249- 268, 2004.
32. U. Hoffrage, S. Lindsey, R. Hertwig, and G. Gigerenzer. Medicine: Communicating statistical information. Science, 290(5500):2261-2262, 2000.
33. P. N. Johnson-Laird, P. Legrenzi, V. Girotto, M. S. Legrenzi, and J. P. Caverni. Naive probability: a mental model theory of extensional reasoning. Psychological Review, 106(1):62-88, 1999.
34. D. Kahneman, P. Slovic, and A. Tversky. Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press, 1982.
35. V. Kellen, S. Chan, and X. Fang. Facilitating conditional probability problems with visuals. In Conf Human-Computer Interaction, pages 63-71, 2007.
36. J. J. Koehler. One in Millions, Billions and Trillions: Lessons from People V. Collins (1968) for People V. Simpson (1995). Legal Education, 47:214-223, 1997.
37. K. Koffka. Principles of Gestalt Psychology. Harcourt, Brace, 1935.
38. L. Micallef and P. Rodgers. Drawing Area-Proportional Venn-3 Diagrams Using Ellipses. Technical Report TR-3-11, School of Computing, University of Kent, UK, 2011. http://www.eulerdiagrams.org/eulerAPE.
39. A. L. Nichols and J. K. Maner. The good-subject effect: investigating participant demand characteristics. General Psychology, 135(2):151-165, 2008.
40. G. Paolacci, J. Chandler, and P. G. Ipeirotis. Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5):411- 419, 2010.
41. E. C. Poulton and P. R. Freeman. Unwanted asymmetrical transfer effects with balanced experimental designs. Psychological Bulletin, 66(1):1-8, 1966.
42. M. Price, R. Cameron, and P. Butow. Communicating risk information: the influence of graphical display format on quantitative information perception-accuracy, comprehension and preferences. Patient Education and Counseling, 69(1-3):121-128, 2007. (Pubitemid 350051682)
43. N. H. Riche and T. Dwyer. Untangling Euler Diagrams. IEEE Transactions on Visualization and Computer Graphics, 16(6):1090-1099, 2010.
44. P. Rodgers, L. Zhang, and A. Fish. General Euler Diagram Generation. In Conf Diagrams, LNCS (LNAI) 5223, pages 13-27, 2008.
45. P. Rodgers, L. Zhang, and H. Purchase. Wellformedness Properties in Euler Diagrams: Which Should Be Used? IEEE Transactions on Visualization and Computer Graphics, 18(7):1089-1100, 2012.
46. J. Ross, L. Irani, M. S. Silberman, A. Zaldivar, and B. Tomlinson. Who are the Crowdworkers? Shifting Demographics in Mechanical Turk. In ACM Conf Human Factors in Computing Systems, pages 2863-2872, 2010.
47. T. A. Salthouse. The processing-speed theory of adult age differences in cognition. Psychological Review, 103(3):403-428, 1996. (Pubitemid 126432206)
48. P. Sedlmeier and G. Gigerenzer. Teaching Bayesian reasoning in less than two hours. Experimental Psychology General, 130(3):380-400, 2001. (Pubitemid 33597700)
49. S. A. Sloman, D. Over, L. Slovak, and J. M. Stibel. Frequency illusions and other fallacies. Organizational Behavior and Human Decision Processes, 91(2):296-309, 2003. (Pubitemid 36952878)
50. G. Stine. Acquired immune deficiency syndrome: biological, medical, social, and legal issues. Prentice Hall, 1996.
51. E. R. Stone, J. F. Yates, and A. M. Parker. Effects of numerical and graphical displays on professed risk-taking behavior. Experimental Psychology Applied, 3(4), 1997.
52. J. Surowiecki. The Wisdom of Crowds. Anchor, 2005.
53. E. R. Tufte. Beautiful Evidence. Graphics Press, 2006.
54. B. Victor. Explorable Explanations. http://worrydream.com, 2011.
55. H. G. Welch and W. C. Black. Overdiagnosis in Cancer. J National Cancer Institute, 102(9):605-613, 2010.
56. L. Wilkinson. Exact and Approximate Area-proportional Circular Venn and Euler Diagrams. IEEE Transactions on Visualization and Computer Graphics, 18(2):321-331, 2011.
57. K. Yamagishi. Facilitating normative judgments of conditional probability: frequency or nested sets? Experimental Psychology, 50(2):97-106, 2003. (Pubitemid 38355429)
58. P.-H. Zahl and J. Maehlen. Overdiagnosis in mammography screening. Tidsskr Norske Laege, 124(17):2238-2239, 2004. (Pubitemid 40320443)