Why and how to combine evidence in environmental assessments: Weighing evidence and building cases

Science of the Total Environment

Volumn 409, Issue 8, 2011, Pages 1406-1417

  Suter, Glenn W ^a   Cormier, Susan M ^a  

^a U S ENVIRONMENTAL PROTECTION AGENCY   (United States)

Author keywords

Ecological risk assessment; Environmental epidemiology; Kit fox; Lines of evidence; Risk assessment; Weight of evidence

Indexed keywords

ECOLOGICAL RISK ASSESSMENT; ENVIRONMENTAL EPIDEMIOLOGY; KIT FOX; LINES OF EVIDENCE; WEIGHT OF EVIDENCE;

DISEASES; ECOLOGY; POPULATION STATISTICS; RATING; WEIGHING;

RISK ASSESSMENT;

COMPLEXITY; CONTAMINATED LAND; ENVIRONMENTAL ASSESSMENT; ENVIRONMENTAL ISSUE; HYPOTHESIS TESTING; INTEGRATED APPROACH; RISK ASSESSMENT;

AQUATIC ENVIRONMENT; ARTICLE; DECISION SUPPORT SYSTEM; ENVIRONMENTAL IMPACT ASSESSMENT; ENVIRONMENTAL MANAGEMENT; ENVIRONMENTAL PLANNING; MATHEMATICAL MODEL; PRIORITY JOURNAL; PROBLEM SOLVING; QUALITATIVE ANALYSIS; RISK ASSESSMENT; RISK BENEFIT ANALYSIS; SCORING SYSTEM;

DECISION SUPPORT TECHNIQUES; ENVIRONMENTAL MONITORING; ENVIRONMENTAL POLLUTANTS; ENVIRONMENTAL POLLUTION; MODELS, THEORETICAL; RISK ASSESSMENT;

VULPES;

EID: 79951813767     PISSN: 00489697     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.scitotenv.2010.12.029     Document Type: Article

References (76)

1. Agency for Healthcare and Quality Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment 47 2002, U.S. Public Health Service, Washington, DC.
2. Akaike H. Information theory as an extension of the maximum likelihood principle. Second International Symposium on Information Theory 1973, 267-281. Akademia Kiado, Budapest. B.N. Petrov, F. Csaki (Eds.).
3. Anderson D.R. Model based inference in the life sciences: a primer on evidence 2008, Springer, New York, NY.
4. Bellucci C., Hoffman G., Cormier S. An iterative approach for identifying the causes of reduced benthic macroinvertebrate diversity in the Willimantic River, Connecticut. EPA/600/R-08/144 2010, U.S. Environmental Protection Agency, Cincinnati, OH.
5. Burkhardt-Holm P., Scheurer K. Application of the weight-of-evidence approach to assess the decline of brown trout (Salmo trutta) in Swiss rivers. Aquat Sci 2007, 69:51-70.
6. Burton G.A., Batley G.E., Chapman P.M., Forbes V.E., Smith E.P., Reynoldson T., et al. A weight-of-evidence framework for assessing sediment (or other) contamination: improving certainty in the decision-making process. Hum Ecol Risk Assess 2002, 8:1675-1696.
7. Cavalli-Sforza L. Genes, peoples, and languages 2000, North Point Press, New York.
8. Chapman P.M. The sediment quality triad approach to determining pollution-induced degradation. Sci Total Environ 1990, 97/98:815-825.
9. Chapman P.M. The sediment quality triad: then, now and tomorrow. Internat J Environ Pollut 2003, 13:351-356.
10. Chapman P.M. Traditional ecological knowledge (TEK) and scientific weight of evidence determination. Mar Pollut Bull 2007, 54:1839-1840.
11. Chapman P.M., McDonald B.G., Lawrence G.S. Weight-of-evidence issues and frameworks for sediment quality (and other) assessments. Hum Ecol Risk Assess 2002, 8:1489-1515.
12. Cormier S.M., Paul J.F., Spehar R.L., Berry W.J., Shaw-Allen P., Suter G.W. Using field data and weight of evidence to develop water quality criteria. Integr Environ Assess Manag 2008, 4:490-504.
13. Cormier S.M., Suter G.W., Norton S.B. Causal characteristics for ecoepidemiology. Hum Ecol Risk Assess 2010, 16:53-73.
14. Cormier S.M., Suter G.W. A framework for thoroughly integrating environmental assessments. Environ Manag 2008, 42:543-556.
15. Dagnino A., Sforzini S., Dondero F., Fenoglio S., Bona E., Jensen J., et al. A "weight-of-evidence" approach for the integration of environmental "triad" data to assess ecological risk and biological vulnerability. Integr Environ Assess Manag 2008, 4:314-326.
16. Dale V.H., Biddinger G.R., Newman M.C., Oris J.T., Suter G.W., Thompson T., et al. Enhancing the ecological risk assessment process. Integr Environ Assess Manag 2008, 4:306-313.
17. de Zwart D., Posthuma L., Gevrey M., von der Ohe P., de Dekere E. Diagnosis of ecosystem impairment in a multiple-stress context - how to formulate effective river basin management plans. Integr Environ Assess Manag 2009, 5:38-49.
18. Ebell M.H., Siwek J., Weiss B.D., Woolf S.H., Susman J., Ewigman B., et al. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. Am Fam Physician 2004, 69:548-556.
19. Forbes V.E., Calow P. Applying weight-of-evidence to retrospective ecological risk assessment when quantitative data are limited. Hum Ecol Risk Assess 2002, 8:1625-1639.
20. Fox G.A. Practical causal inference for ecoepidemiologists. J Toxicol Environ Health 1991, 33:359-373.
21. Garnett S.T., Brook B.W. Modelling to forestall extinction of Australian tropical birds. J Ornithol 2007, 148(Suppl. 2):S311-S320.
22. Gates S. Review of methodology of quantitative reviews using meta-analysis in ecology. J Anim Ecol 2002, 71:547-557.
23. Gerritsen J., Zheng L., Burton J., Boschen C., Wilkes S., Ludwig J., Cormier S. Inferring causes of biological impairment in the Clear Fork Watershed, West Virginia. EPA/600/R-08/146 2010, U.S. Environmental Protection Agency, Cincinnati, OH.
24. Gold J.I., Shadlen M.N. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 2002, 36:299-308.
25. Good I.J. A causal calculus. Brit J Philo Sci 1961, 11:305-318.
26. Good I.J. Good thinking: the foundations of probability and its applications 1983, U Minnesota Press, Minneapolis.
27. GRADE Working Group Grading quality of evidence and strength of recommendations. BMJ 2004, 328:1-8.
28. Gray G.M. Complete risk characterization. Risk Perspective 1994, 2:1-2.
29. Haake D.M., Wilton T., Krier K., Stewart A.J., Cormier S.M. Causal assessment of biological impairment in the Little Floyd River, Iowa, USA. Hum Ecol Risk Assess 2010, 16:116-148.
30. Hawkins C.P., Norris R.H., Hogue J.N., Feminella J.W. Development and evaluation of predictive models for measuring the biological integrity of streams. Ecol Appl 2000, 10:1456-1477.
31. Hicks M., Whittington K., Thomas J., Kurtz J., Stewart A., Suter G.W., et al. Causal assessment of biological impairment in the Bogue Homo River, Mississippi using the U.S. EPA's stressor identification methodology. EPA/600/R-08/143 2010, U.S. Environmental Protection Agency, Cincinnati, OH.
32. Higgins J.P.T., Green S. Cochrane handbook for systematic reviews of interventions. Version 5.0.1 2008, The Cochrane Collaboration, Cambridge, UK, Available at. http://www.mrc-bsu.cam.ac.uk/cochrane/handbook/.
33. Hilborn R., Mangel M. The ecological detective: confronting models with data 1997, Princeton U. Press, Princeton, NJ.
34. Hull R.N., Swanson S. Sequential analysis of lines of evidence - an advanced weight-of-evidence approach for ecological risk assessment. Integr Environ Assess Manag 2006, 2:302-311.
35. Jadad A.R., Moore R.A., Carroll D., Jenkinson C., Reynolds D.J.M., Gavaghan D.J., et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials 1996, 17:1-12.
36. Jones D.S., Barnthouse L.W., Suter G.W., Efroymson R.E., Field J.M., Beauchamp J.J. Ecological risk assessment of a large river-reservoir: 3. benthic invertebrates. Environ Toxicol Chem 1999, 18:599-609.
37. Krimsky S. The weight of scientific evidence in policy and law. Amer J Public Health 2005, 95:S129-S135.
38. Lee G.F., Jones-Lee A. Independent applicability of chemical and biologicalcriteria/standards and effluent toxicity tests. Nat Environ J 1995, 5:60-63.
39. Linkov I., Long E.B., Cormier S.M., Satterstrom F.K., Bridges T. Weight-of-evidence evaluation in environmental assessment: review of qualitative and quantitative approaches. Sci Total Environ 2009, 497:5199-5205.
40. Lohr K.N. Rating the strength of scientific evidence: relevance for quality improvement programs. Int J Qual Health Care 2004, 16:9-18.
41. Long E.R., Chapman P.M. A sediment quality triad: measures of sediment contamination, toxicity and infaunal community composition in Puget Sound. Mar Pollut Bull 1985, 16:405-415.
42. Marcomini A., Suter G.W., Critto A. Decision support systems for risk-based management for contaminated sites 2009, Springer, New York.
43. McDonald B.G., Chapman P.M. Selenium effects: a weight-of-evidence approach. Integr Environ Assess Manag 2007, 3:129-136.
44. McDonald B.G., DeBruyn A.M.H., Wernick B.G., Patterson L., Pellerin N., Chapman P.M. Design and application of a transparent and scalable weight-of-evidence framework: an example from Wabamun Lake, Alberta, Canada. Integr Environ Assess Manag 2007, 3:476-483.
45. McPherson C., Chapman P.M., DeBruyn A.M.H., Cooper L. The importance of benthos in weight of evidence sediment assessments - a case study. Sci Total Env 2008, 394:252-264.
46. Menzie C., Henning M.H., Cura J., Finkelstein K., Gentile J., Maughan J., et al. A weight-of-evidence approach for evaluating ecological risks: report of the Massachusetts weight-of-evidence work group. Hum Ecol Risk Assess 1996, 2:277-304.
47. Moher D., Schulz K.F., Altman D. The CONSORT Statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001, 285:1987-1991.
48. Mumtaz M.M., Furkin P.R. A weight-of-evidence scheme for assessing interactions in chemical mixtures. Toxicol Ind Health 1992, 8:377-406.
49. National Research Council Science and judgment in risk assessment 1994, National Academy Press, Washington, DC.
50. National Research Council Science and decisions: advancing risk assessment 2009, National Academies Press, Washington, DC.
51. Newman M.C., Zhao Y., Carriger J.F. Coastal and estuarine ecological risk assessment: the need for a more formal approach to stressor identification. Hydrobiolgia 2007, 577:31-40.
52. Pope C., Mays N., Popay J. Synthesizing qualitative and quantitative health evidence: a guide to methods 2007, Open University Press, Maidenhead, UK.
53. Smith E.P., Lipkovich I., Ye K. Weight-of-evidence (WOE): quantitative estimation of probability of impairment for individual and multiple lines of evidence. Hum Ecol Risk Assess 2002, 8:1585-1596.
54. Stahl R.G. Issues addressed and unaddressed in EPA's ecological risk guidelines. Risk Policy Report April 1998, 17:35-37.
55. Susser M. Rules of inference in epidemiology. Reg Toxicol Pharma 1986, 6:116-186.
56. Suter G.W. Risk characterization for ecological risk assessment of contaminated sites. ES/ER/TM-200 1996, Oak Ridge National Laboratory, Oak Ridge, TN.
57. Suter G.W., Barnthouse L.W., Efroymson R.E., Jager H. Ecological risk assessment of a large river-reservoir: 2. fish community. Environ Toxicol Chem 1999, 18:589-598.
58. Suter G.W., Efroymson R.A., Sample B.E., Jones D.S. Ecological risk assessment for contaminated sites 2000, Lewis Publishers, Boca Raton, FL.
59. Suter G.W., Norton S.B., Cormier S.M. A methodology for inferring the causes of observed impairments in aquatic ecosystems. Environ Toxicol Chem 2002, 21:1101-1111.
60. Suter G.W., Cormier S.M., Norton S.B. Ecological epidemiology and causal analysis. Ecological Risk Assessment 2007, p. 39-p. 68. CRC Press, Boca Raton, FL. 2nd ed. G.W. Suter (Ed.).
61. Suter G.W., Cormier S.M. When is a formal assessment process worth while?. Hum Ecol Risk Assess 2010, 16:1-3.
62. Risk assessment and risk management in regulatory decision-making 1997, The Presidential/Congressional Commission on Risk Assessment and Risk Management, U.S. Government Printing Office, Washington, D.C.
63. U.S.EPA Technical support document for water quality-based toxics control. EPA/505/2-90-001 1991, U.S. Environmental Protection Agency, Washington, D.C.
64. U.S.EPA ARCS assessment guidance document. EPA 905-B94-002 1994, Great Lakes National Program Office, Chicago, IL.
65. U.S.EPA Guidelines for ecological risk assessment. EPA/630/R-95/002F 1998, U.S. Environmental Protection Agency, Washington, DC.
66. U.S.EPA Stressor identification guidance document. EPA/822/B-00/025 2000, U.S. Environmental Protection Agency, Washington, DC.
67. EPA U.S. Guidelines for carcinogen risk assessment. EPA/630/P-03/001F 2005, U.S. Environmental Protection Agency, Washington, DC.
68. EPA U.S. Analysis of the causes of a decline in the San Joaquin kit fox population on the Elk Hills, Naval Petroleum Reserve #1, California. EPA/600/R-08/130 2009, U.S. Environmental Protection Agency, Cincinnati, OH.
69. EPA U.S. A field-based aquatic life benchmark for conductivity in central Appalachian streams. EPA/600/R-10/023A 2010, (external review draft)., U.S. Environmental Protection Agency, Cincinnati, OH.
70. Walker V.R. Risk characterization and the weight of evidence: adapting gatekeeping concepts from the courts. Risk Anal 1996, 16:793-799.
71. Weed D.L. Weight of evidence: a review of concept and method. Risk Anal 2005, 25:1545-1557.
72. Weed D.L. Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related. Int J Epidemiol 2000, 29:387-390.
73. Weinhold B. Building on success: assessment categories for experimental noncancer endpoints. Environ Health Persp 2009, 117:A113-A115.
74. Wiseman C.D., LeMoine M., Cormier S. Assessment of probable causes of reduced aquatic life in the Touchet River, Washington, USA. Hum Ecol Risk Assess 2010, 16:87-115.
75. Wright J.F., Sutcliffe D.W., Furse M.T. Assessing the biological quality of fresh waters: RIVPACS and other techniques 2000, Freshwater Biological Association, Ambleside.
76. Yoder C.O., Rankin E.T. The role of biological indicators in a state water quality management process. Environ Monit Assess 1998, 51:61-88.