Ecology, statistics, and the art of misdiagnosis: The need for a paradigm shift

Environmental Reviews

Volumn 7, Issue 4, 1999, Pages 167-190

  Germano, Joseph D ^a  

^a EVS Environment Consultants   (United States)

Author keywords

Bayesian statistics; Risk assessment; Statistical significance

Indexed keywords

EID: 0001189324     PISSN: 11818700     EISSN: None     Source Type: Journal    
DOI: 10.1139/er-7-4-167     Document Type: Article

References (143)

1. Arkes, H.E. 1981. Impediments to accurate clinical judgment and possible ways to minimize their impact. J. Consult. Clin. Psychol. 49: 323-330.
2. Arkes, H.E., and Harkness, A.R. 1980. The effect of making a diagnosis on subsequent recognition of symptoms. J. Exp. Psychol. Hum. Learn. Mem. 6: 568-575.
3. Arkes, H.E., Wortmann, R.L., Saville, R.D., and Harkness, A.R. 1981. The hindsight bias among physicians weighing the likelihood of a diagnosis. J. Appl. Psychol. 66: 252-254.
4. Arkes, H.E., Dawes, R.M., and Christensen, C. 1986. Factors influencing the use of a decision rule in a probabilistic task. Behav. Hum. Decis. Processes, 37: 93-110.
5. Bakan, D. 1966. The test of significance in psychological research. Psychol. Bull. 66: 423-437.
6. Berger, J.O. 1985. Statistical decision theory and Bayesian analysis. 2nd ed. Springer-Verlag, New York.
7. Berger, J.O., and Berry, D.A. 1988. Statistical analysis and the illusion of objectivity. Am. Sci. 76: 159-165.
8. Berkson, J. 1938. Some difficulties of interpretation encountered in the application of the chi-square test. J. Am. Stat. Assoc. 33: 526-542.
9. Bernstein, B. 1990. A framework for trend detection: coupling ecological and managerial perspectives. Presented at the International Symposium on Ecological Indicators, Fort Lauderdale, Fla., October 16-19, 1990.
10. Bernstein, B.B., and Zalinski, J. 1986. A philosophy for effective monitoring. In Proceedings of IEEE OCEANS '86 Conference, Washington, D.C., September 23-25, 1986. pp. 1024-1029.
11. Bower, B. 1999. Simple minds, smart choices. Sci. News, 155: 348-349.
12. Box, G.E.P. 1980. Sampling and Bayes inference in scientific modeling and robustness (with discussion). J. R. Stat. Soc. Ser. A, 143: 383-430.
13. Carver, R.P. 1978. The case against statistical significance testing. Harv. Educ. Rev. 48: 378-399.
14. Cassie, R.M. 1972. Fauna and sediments of an intertidal mud-flat: an alternative multivariate analysis. J. Mar. Biol. Ecol. 9: 55-64.
15. Chapman, L.J., and Chapman, J.P. 1967. Genesis of popular but erroneous psychodiagnostic observations. J. Abnorm. Psychol. 72: 193-204.
16. Chapman, L.J., and Chapman, J.P. 1969. Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. J. Abnorm. Psychol. 74: 271-280.
17. Chapman, P.M., Dexter, R.M., and Long, E.R. 1987. Synoptic measures of sediment contamination, toxicity and infaunal community composition (the sediment quality triad) in San Francisco Bay. Mar. Ecol. Prog. Ser. 37: 75-96.
18. Chardy, P., Glémarec, M., and Laurec, A. 1976. Application of inertia methods to benthic marine ecology: practical implications of the basic options. Estuarine Coastal Mar. Sci. 4: 179-205.
19. Clark, C.A. 1963. Hypothesis testing in relation to statistical methodology. Rev. Educ. Res. 33: 455-473.
20. Cohen, J. 1962. The statistical power of abnormal-social psychological research: a review. J. Abnorm. Soc. Psychol. 65: 145-153.
21. Cohen, J. 1994. The earth is round (p < .05). Am. Psychol. 49: 997-1003.
22. Cowles, M., and Davis, C. 1982. On the origins of the .05 level of statistical significance. Am. Psychol. 37: 553-558.
23. Crane, M., and Newman, M.C. 1996. Scientific method in environmental toxicology. Environ. Rev. 4: 112-122.
24. Cronbach, L.J., and Snow, R.E. 1977. Aptitudes and instructional methods: a handbook for research on interactions. Irvington, New York.
25. Culp, J.M., and Davies, R.W. 1980. Reciprocal averaging and polar ordination as techniques for analyzing lotie macroinvertebrate communities. Can. J. Fish. Aquat. Sci. 37: 1358-1364.
26. Dawes, R.M. 1971. A case study of graduate admissions: Application of three principles of human decision making. Am. Psychol. 26: 180-188.
27. Dawes, R.M. 1979. The robust beauty of improper linear models in decision making. Am. Psychol. 34: 571-582.
28. Dawes, R.M., and Corrigan, B. 1974. Linear models in decision making. Psychol. Bull. 81: 95-106.
29. Dawes, R.M., Faust, D., and Meehl, P.E. 1989. Clinical versus actuarial judgment. Science (Washington, D.C.), 243: 1668-1674.
30. de Finetti, B. 1959. La probabilità e la statistica nei rapporti con l'induzione, secondo i diversi punti da vista. In Induzione e statistica. Istituto Matematico dell'Universita, Rome.
31. Eberhardt, L.L., and Thomas, J.M. 1991. Designing environmental field studies. Ecol. Monogr. 61: 53-73
32. Edwards, W., Lindman, H., and Savage, L.J. 1963. Bayesian statistical inference for psychological research. Psychol. Rev. 70: 193-242.
33. Efron, B. 1986. Why isn't everyone a Bayesian? Am. Stat. 40: 1-11.
34. Efron, B., and Gong, G. 1983. A leisurely look at the bootstrap, the jackknife, and cross-validation. Am. Stat. 37: 36-48.
35. Einhorn, H.J. 1972. Expert measurements and mechanical combination. Organ. Behav. Hum. Perform. 7: 86-106.
36. Einhorn, H.J., and Hogarth, R.M. 1981. Behavioral decision theory: processes of judgment and choice. Annu. Rev. Psychol. 32: 53-88.
37. Elstein, A.S., Shulman, A.S., and Sprafka, S.A. 1978. Medical problem solving: an analysis of clinical reasoning. Harvard University Press, Cambridge, Mass.
38. Erman, D.C., and Helm, W.T. 1971. Comparison of some species importance values and ordination techniques used to analyse benthic invertebrate communities. Oikos, 22: 240-247.
39. Faust, D. 1984. The limits of scientific judgment. University of Minnesota Press, Minneapolis.
40. Faust, D. 1986a. Learning and maintaining rules for decreasing judgment accuracy. J. Pers. Assess. 50: 585-600.
41. Faust, D. 1986b. Research on human judgment and its application to clinical practice. Prof. Psych. Res. Pract. 17: 420-430.
42. Faust, D. 1989. Data integration in legal evaluations: Can clinicians deliver on their premises? Behav. Sci. Law, 7: 469-483.
43. Field, J.G. 1971. A numerical analysis of changes in the soft-bottom fauna along a transect across False Bay, South Africa. J. Mar. Biol. Ecol. 7: 215-253.
44. Fischhoff, B. 1975. Hindsight ≠ foresight: the effect of outcome knowledge on judgment under uncertainty. J. Exp. Psychol. 1: 288-299.
45. Fischhoff, B. 1982. Debiasing. In Judgment under uncertainty. Edited by D. Kahneman, P. Slovic, and A. Tversky. Cambridge University Press, New York. pp. 422-444.
46. Fisher, R.A. 1947. The design of experiments. 4th ed. Oliver & Boyd, Edinburgh.
47. Forbes, V.E., and Forbes, T.L. 1994. Ecotoxicology in theory and practice. Chapman and Hall, London, U.K.
48. Galen, R.S., and Gambino, S.R. 1975. Beyond normality: the predictive value and efficiency of medical diagnoses. Wiley, New York.
49. Germano, J.D. 1985. An evaluation of Gray's log-normal method. White paper prepared for U.S. Army Corps of Engineers, Waterways Experiment Station. EVS Environment Consultants, 200 West Mercer Street, Suite 403, Seattle, WA 98119, U.S.A.
50. Germano, J.D. 1991. To grab or not two grabs: infaunal benthic sampling strategies and the need for replication. A discussion of statistical power analysis. White paper submitted to EPA Region IX under Contract 68-C8-0061. EVS Environment Consultants, 200 West Mercer Street, Suite 403, Seattle, WA 98119, U.S.A.
51. Germano, J.D. Reflections on statistics, ecology, and risk assessment. In Organism-sediment interactions. Edited by J.Y. Aller, S.A. Woodin, and R.C. Aller. Belle Baruch Library in Marine Science. University of South Carolina Press, Columbia. In press.
52. Germano, J.D., Rhoads, D.C., and Lunz, J.D. 1994. An integrated, tiered approach to monitoring and management of dredged material disposal sites in the New England regions. DAMOS Contribution 87. U.S. Army Corps of Engineers, New England Branch, 696 Virginia Road, Concord, MA 01742, U.S.A.
53. Goldberg, L.R. 1959. The effectiveness of clinicians' judgements: the diagnosis of organic brain damage from the Bender-Gestalt Test. J. Consult. Psychol. 23: 25-33.
54. Goldberg, L.R. 1968. Simple models or simple processes? Some research on clinical judgments. Am. Psychol. 23: 483-496.
55. Good, I.J. 1973. The probabilistic explication of evidence, surprise, causality, explanation, and utility. In Foundations of statistical inference. Edited by V.P. Godambe and D.A. Sprott. Holt, Rinehart, and Winston, Toronto.
56. Gould, S.J. 1981. The mismeasure of man. W.W. Norton & Co., New York.
57. Green, R.H. 1979. Sampling design and statistical methods for environmental biologists. John Wiley & Sons, New York.
58. Green, R.H. 1984. Some guidelines for the design of biological monitoring programs in the marine environment. In Concepts in marine pollution measurements. Edited by H.H. White. Maryland Sea Grant College, College Park. pp. 647-655.
59. Hays, W.L. 1963. Statistics. Holt, Rinehart, & Winston, New York.
60. Hill, R.A. 1996. From science to decision-making: the applicability of Bayesian methods to risk assessment. Hum. Ecol. Risk Assess. 2: 636-642.
61. Holling, C.S. 1978. Adaptive environmental assessment and management. John Wiley & Sons, Toronto.
62. Howson, C., and Urbach, P. 1991. Bayesian reasoning in science. Nature (Lond.), 350: 371-374.
63. Hughes, R.N., and Thomas, M.L.H. 1971. The classification and ordination of shallow-water benthic samples from Prince Edward Island, Canada. J. Exp. Mar. Biol. Ecol. 7: 1-39.
64. Iversen, G.R. 1984. Bayesian statistical inference. Sage University Paper series on Quantitative Applications in the Social Sciences, 07-043. Sage Publications, Beverly Hills and London.
65. Jefferys, W.H., and Berger, J.O. 1992. Ockham's razor and Bayesian analysis. Am. Sci. 80: 64-72.
66. Kahneman, D., and Tversky, A. 1973. On the psychology of prediction. Psychol. Rev. 80: 237-251.
67. Kaiser, J. 1996. Scientists angle for answers. Science (Washington D.C.), 274: 1837-1838.
68. Keller, E.F. 1985. Reflections on gender and science. Yale University Press, New Haven, Conn.
69. Kohn, A.J., and Riggs, A.C. 1982. Sample size dependence in measures of proportional similarity. Mar. Ecol. Prog. Ser. 9: 147-151.
70. Koriat, A., Lichtenstein, S., and Fischhoff, B. 1980. Reasons for confidence. J. Exp. Psychol. 6: 107-118.
71. Kuhn, T.S. 1962. The structure of scientific revolutions. University of Chicago Press, Chicago.
72. Lee, P.M. 1989. Bayesian statistics: an introduction. Oxford University Press, New York.
73. Lipsey, M.W. 1990. Design sensitivity: statistical power for experimental research. Sage Publications, Newbury Park, Calif.
74. Loehle, C. 1987. Hypothesis testing in ecology: psychological aspects and the importance of theory maturation. Q. Rev. Biol. 62: 397-409.
75. Loehle, C. 1988. Philosophical tools: potential contributions to ecology. Oikos, 51: 97-104.
76. Long, E.R., and Morgan, L.G. 1990. The potential for biological effects of sediment-sorbed contaminants tested in the national Status and Trends Program. NOAA Tech. Mem. NOS-OMA-52.
77. Long, E.R., Field, L.J., and MacDonald, D.D. 1998. Predicting toxicity in marine sediments with numerical sediment quality guidelines. Environ. Toxicol. Chem. 17: 714-727.
78. Lord, C.G., Ross, L., and Lepper, M.R. 1979. Biased assimilation and attitude polarization: the effects of prior theories on subsequently considered evidence. J. Pers. Soc. Psychol. 37: 2098-2109.
79. Lusted, L.B. 1968. Introduction to medical decision making. Charles Thomas, Springfield, Ill.
80. Lykken, D.T. 1968. Statistical significance in psychological research. Psychol. Bull. 70: 151-159.
81. Meehl, P.E. 1954. Clinical versus statistical prediction: a theoretical analysis and a review of the evidence. University of Minnesota Press, Minneapolis.
82. Meehl, P.E. 1965. Seer over sign: the first good example. J. Exp. Res. Pers. 1: 27-32.
83. Meehl, P.E. 1967. Theory testing in psychology and physics: a methodological paradox. Philos. Sci. 34: 103-115.
84. Meehl, P.E. 1973. Psychodiagnosis: selected papers. University of Minnesota Press, Minneapolis.
85. Meehl, P.E. 1986. Causes and effects of my disturbing little book. J. Pers. Assess. 50: 370-375.
86. M'Gonigle, R.M., Jamieson, T.L., McAllister, M.K., and Peterman, R.M. 1994. Taking uncertainty seriously: from permissive regulation to preventative design in environmental decision making. Osgoode Hall Law J. 32: 99-169.
87. Mohr, L.B. 1990. Understanding significance testing. Sage University Paper Series on Quantitative Applications in the Social Sciences, 07-073. Sage Publications, Newbury Park, Calif.
88. Morrison, D.E., and Henkel, R.E. 1970. The significance test controversy. Aldine, Chicago.
89. Neyman, J., and Pearson, E. 1933. On the problem of the most efficient tests of statistical hypotheses. Philos. Tran. R. Soc. Lond. 231: 289-337.
90. Nisbett, R.E., and Ross, L. 1980. Human inferences: strategies and shortcomings of social judgment. Prentice-Hall, Englewood Cliffs, N.J.
91. Nisbett, R.E., and Wilson, T.D. 1977. Telling more than we can know: verbal reports on mental processes. Psychol. Rev. 84: 231-259.
92. Nisbett, R.E., Borgida, E., Crandall, R., and Reed, H. 1976. Popular induction: information is not necessarily informative. In Cognition and social behavior. Edited by J.S. Carroll and J.W. Payne. Erlbaum, Hillsdale, N.J. pp. 227-236.
93. Nunnally, J. 1960. The place of statistics in psychology. Educ. Psychol. Meas. 20: 641-650.
94. Oskamp, S. 1965. Overconfidence in case-study judgments. J. Consult. Psychol. 29: 261-265.
95. Oskamp, S. 1967. Clinical judgment from the MMPI: simple or complex. J. Clin. Psychol. 23: 411-415.
96. Pielou, E.C. 1984. The interpretation of ecological data. John Wiley & Sons, New York.
97. Pitz, G.F., and Sachs, N.J. 1984. Judgment and decision: theory and application. Annu. Rev. Psychol. 35: 139-163.
98. Platt, J.R. 1964. Strong inference. Science (Washington D.C.), 146: 347-353.
99. Pollard, P., and Richardson, J.T.E. 1987. On the probability of making Type I errors. Psychol. Bull. 102: 159-163.
100. Poore, G.C.B., and Mobley, M.C. 1980. Canonical correlation analysis of marine macrobenthos survey data. J. Mar. Biol. Ecol. 45: 37-50.
101. Press, S.J. 1989. Bayesian statistics: principles, models, and applications. John Wiley & Sons, New York.
102. Preston, T. 1981. The clay pedestal. Madrona Publishers, Seattle, Wash.
103. PTI. 1988. The apparent effect threshold. Briefing report to the EPA Science Advisory Board. PTI Environmental Services, 15375 SE 30th Place, Suite 250, Bellevue, WA 98007, U.S.A.
104. Punt, A.E., and Hilborn, R. 1997. Fisheries stock assessment and decision analysis: the Bayesian approach. Rev. Fish Biol. Fish. 7: 35-63.
105. Raiffa, H. 1968. Decision analysis. Random House, New York.
106. Rand, R.R. 1996. Statistics for the social sciences. Academic Press, San Diego, Calif.
107. Rand, R.R. 1997. Introduction to robust estimation and hypothesis testing. Academic Press, San Diego, Calif.
108. Rand, R.R. 1998. How many discoveries have been lost by ignoring modern statistical methods? Am. Psychol. 53: 300-314.
109. Reckhow, K.H. 1990. Bayesian inference in non-replicated ecological studies. Ecology, 71: 2053-2059.
110. Reckhow, K.H. 1994. Importance of scientific uncertainty in decision-making. Environ. Manage. 18: 161-166.
111. Reckhow, K.H., and Chapra, S.C. 1983. Confirmation of water quality models. Ecol. Modell. 20: 113-133.
112. Rhoads, D.C., and Germano, J.D. 1982. Characterization of organism-sediment relations using sediment profile imaging: an efficient method of remote ecological monitoring of the seafloor [REMOTS™ System]. Mar. Ecol. Prog. Ser. 8: 115-128.
113. Rhoads, D.C., and Germano, J.D. 1986. Interpreting long-term changes in benthic community structure: a new protocol. Hydrobiologia, 142: 291-308.
114. Rohlf, F.J., and Sokal, R.R. 1969. Statistical tables. W.H. Freeman & Co., San Francisco.
115. Rose, K.A., and Smith, E.P. 1992. Experimental design: the neglected aspect of environmental monitoring. Environ. Manage. 16: 691-700.
116. Ross, L., Lepper, M.R., Strack, F., and Steinmetz, J. 1977. Social explanation and social expectation: effects of real and hypothetical explanations on subjective likelihood. J. Pers. Soc. Psychol. 35: 817-829.
117. Rousseau, D.L. 1992. Case studies in pathological science. Am. Sci. 80: 54-63.
118. Rozeboom, W.W. 1960. The fallacy of the null hypothesis significance test. Psychol. Bull. 57: 416-428.
119. Sawyer, J. 1966. Measure and prediction, clinical and statistical. Psychol. Bull. 66: 178-200.
120. Schlaifer, R. 1961. Introduction to statistics for business decisions. McGraw-Hill, New York.
121. Schmitt, S.A. 1969. Measuring uncertainty: an elementary introduction to Bayesian statistics. Addison-Wesley, Reading, Mass.
122. Schwartz, W.B., Gorry, G.A., Kassirer, J.P., and Essig, A. 1973. Decision analysis and clinical judgment. Am. J. Med. 55: 459-472.
123. Shweder, R.A. 1977. Likeness and likelihood in everyday thought: magical thinking in judgments about personality. Curr. Anthropol. 18: 637-648.
124. Slovic, P., and Fischhoff, B. 1977. On the psychology of experimental surprises. J. Exp. Psychol. 3: 544-551.
125. Slovic, P., and MacPhillamy, D. 1974. Dimensional commensurability and cue utilization in comparative judgment. Organ. Behav. Hum. Perf. 11: 172-194.
126. Smedslund, J. 1963. The concept of correlation in adults. Scand. J. Psychol. 4: 165-173.
127. Spies, R.B. 1989. Sediment bioassays, chemical contaminants and benthic ecology: New insights or just muddy water? Mar. Environ. Res. 27: 73-75.
128. Stearns, S.C. 1976. Life-history tactics: a review of the ideas. Q. Rev. Biol. 51: 3-47.
129. Stevens, S.S. 1971. Issues in psychophysical measurement. Psychol. Rev. 78: 426-450.
130. Summers, D.A., Taliaferro, D.J., and Fletcher, D.J. 1969. Subjective vs. objective description of judgment policy. Psychon. Sci. 18: 249-250.
131. Toft, C.A., and Shea, P.J. 1983. Detecting community-wide patterns: estimating power strengthens statistical inference. Am. Nat. 122: 618-625.
132. Tversky, A., and Kahneman, D. 1971. Belief in the law of small numbers. Psychol. Bull. 76: 105-110.
133. Tversky, A., and Kahneman, D. 1974. Judgment under uncertainty: heuristics and biases. Science (Washington D.C.), 183: 1124-1131.
134. Tversky, A., and Kahneman, D. 1978. Causal schemata in judgments under uncertainty. In Progress in social psychology. Edited by M. Fishbein. Erlbaum, Hillsdale, N.J. pp. 49-72.
135. Tyler, R.W. 1931. What is statistical significance? Educ. Res. Bull. 10: 115-118.
136. USEPA/USACE. 1991. Evaluation of dredged material proposed for ocean disposal (testing manual). U.S. Environmental Protection Agency, Office of Water (WH-556F) and Department of the Army, U.S. Army Corps of Engineers. EPA-503/8-91/001.
137. USEPA/USACE. 1998. Evaluation of dredged material proposed for discharge in waters of the U.S. Testing manual - inland testing manual. U.S. Environmental Protection Agency/U.S. Army Corps of Engineers. EPA-823-B-94-002. U.S. Environmental Protection Agency, Office of Water (4305), Washington, D.C.
138. Ward, W.C., and Jenkins, H.M. 1965. The display of information and the judgment of contingency. Can. J. Psychol. 19: 231-241.
139. Wickelgren, W.A. 1974. How to solve problems. Elements of a theory of problems and problem solving. Freeman, San Francisco.
140. Wiggins, J.S. 1973. Personality and prediction: principles of personality assessment. Addison-Wesley, Reading, Mass.
141. Wiggins, J.S. 1981. Clinical and statistical prediction: Where are we and where do we go from here? Clin. Psychol. Rev. 1: 3-18.
142. Williams, W.T., Clay, H.J., and Bunt, J.S. 1982. The analysis, in marine ecology, of three-dimensional data matrices with one dimension of variable length. J. Exp. Mar. Biol. Ecol. 60: 189-196.
143. Winkler, R.L. 1967. The assessment of prior distributions in Bayesian analysis. J. Am. Stat. Assoc. 62: 776-800.