Simplifying the selection of evidence synthesis methods to inform environmental decisions: A guide for decision makers and scientists

Biological Conservation

Volumn 213, Issue , 2017, Pages 135-145

  Cook, Carly N ^a,b   Nichols, Susan J ^a,c   Webb, J Angus ^a,d   Fuller, Richard A ^a,e   Richards, Rob M ^a,f  

^a Centre for Evidence Informed Policy and Practice   (Australia)

^b MONASH UNIVERSITY   (Australia)

^c UNIVERSITY OF CANBERRA   (Australia)

^d UNIVERSITY OF MELBOURNE   (Australia)

^e UNIVERSITY OF QUEENSLAND   (Australia)

^f Natural Decisions Pty Ltd   (Australia)

Author keywords

Aggregative methods; Configurative methods; Decision support; Environmental management; Evidence synthesis; Management decisions

Indexed keywords

DECISION MAKING; DECISION SUPPORT SYSTEM; ENVIRONMENTAL MANAGEMENT; ENVIRONMENTAL PLANNING; EUROPEAN COMMISSION; MANAGEMENT PRACTICE; NUMERICAL METHOD;

EID: 85030464107     PISSN: 00063207     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.biocon.2017.07.004     Document Type: Review

References (78)

1. Adams, S.M., Assessing cause and effect of multiple stressors on marine systems. Mar. Pollut. Bull. 51 (2005), 649–657.
2. Anderson, L.M., Petticrew, M., Rehfuess, E., Armstrong, R., Ueffing, E., Baker, P., Francis, D., Tugwell, P., Using logic models to capture complexity in systematic reviews. Res. Synth. Methods 2 (2011), 33–42.
3. Arnqvist, G., Wooster, D., Metaanalysis — synthesizing research findings in ecology and evolution. Trends Ecol. Evol. 10 (1995), 236–240.
4. Banks, G., Evidence-Based Policy Making: What Is It? How Do We Get It?. 2009, Productivity Commission, Canberra.
5. Bekkering, G.E., Harris, R.J., Thomas, S., Mayer, A.M.B., Beynon, R., Ness, A.R., Harbord, R.M., Bain, C., Smith, G.D., Sterne, J.A.C., How much of the data published in observational studies of the association between diet and prostate or bladder cancer is usable for meta-analysis?. Am. J. Epidemiol., 167, 2008.
6. Bilotta, G.S., Milner, A.M., Boyd, I., On the use of systematic reviews to inform environmental policies. Environ. Sci. Pol. 42 (2014), 67–77.
7. Bilotta, G.S., Milner, A.M., Boyd, I.L., How to increase the potential policy impact of environmental science research. Environ. Sci. Eur., 27, 2015, 9.
8. Bureau of Meterorology (BOM), Methods for Evidence-Based Conceptual Modelling in Environmental Accounting: A Technical Note. 2016, Bureau of Meteorology, Canberra, Australia, 99.
9. Cadotte, M.W., Mehrkens, L.R., Menge, D.N.L., Gauging the impact of meta-analysis on ecology. Evol. Ecol. 26 (2012), 1153–1167.
10. Campbell Collaboration (CC). Campbell Collaboration Systematic Reviews: Policies and Guidelines Version 11. 2015, Campbell Systematic Reviews, Oslo, Norway.
11. Canessa, S., Guillera-Arroita, G., Lahoz-Monfort, J.J., Southwell, D.M., Armstrong, D.P., Chadès, I., Lacy, R.C., Converse, S.J., When do we need more data? A primer on calculating the value of information for applied ecologists. Methods Ecol. Evol. 6 (2015), 1219–1228.
12. Centre for Evidence-Based Conservation (CEBC), Guidelines for systematic review in environmental management. Version 4.2. Environmental evidence. www.environmentalevidence.org/Authors.htm, 2013.
13. Cook, C.N., Mascia, M.B., Schwartz, M.W., Possingham, H.P., Fuller, R.A., Achieving conservation science that bridges the knowledge-action boundary. Conserv. Biol. 27 (2013), 669–678.
14. Cook, C.N., Possingham, H.P., Fuller, R.A., Contribution of systematic reviews to management decisions. Conserv. Biol. 27 (2013), 902–915.
15. Department of Environment and Heritage Protection (DEHP), Pictures Worth a Thousand Words: A Guide to Pictorial Conceptual Modelling. 2012, Department of Environment and Heritage Protection, Brisbane.
16. Dicks, L.V., Ashpole, J.E., Dänhardt, J., James, K., Jönsson, A., Randall, N., Showler, D.A., Smith, R.K., Turpie, S., Williams, D., Sutherland, W.J., Farmland Conservation: Evidence for the Effects of Interventions in Northern and Western Europe. 2013, Pelagic Publishing, Exeter.
17. Dicks, L.V., Walsh, J.C., Sutherland, W.J., Organising evidence for environmental management decisions: A ‘4S’ hierarchy. Trends Ecol. Evol. 29 (2014), 607–613.
18. Dicks, L.V., Wright, H.L., Ashpole, J.E., Hutchison, J., McCormack, C.G., Livoreil, B., Zulka, K.P., Sutherland, W.J., What works in conservation? Using expert assessment of summarised evidence to identify practices that enhance natural pest control in agriculture. Biodivers. Conserv. 25 (2016), 1383–1399.
19. Doerr, E.D., Dorrough, J., Davies, M.J., Doerr, V.A.J., McIntyre, S., Maximizing the value of systematic reviews in ecology when data or resources are limited. Austral Ecol. 40 (2015), 1–11.
20. Downes, B.J., Barmuta, L.A., Fairweather, P.G., Faith, D.P., Keough, M.J., Lake, P.S., Mapstone, B.D., Quinn, G.P., Monitoring Ecological Impacts: Concepts and Practice in Flowing Waters. 2002, Cambridge University Press, Cambridge.
21. Egger, M., Smith, G.D., Meta-analysis - bias in location and selection of studies. Br. Med. J. 316 (1998), 61–66.
22. Fazey, I., Salisbury, J.G., Lindenmayer, D.B., Maindonald, J., Douglas, R., Can methods applied in medicine be used to summarize and disseminate conservation research?. Environ. Conserv. 31 (2004), 190–198.
23. Fischenich, C., The Application of Conceptual Models to Ecosystem Restoration. U.S. Army Corps of Engineers, Environmental Advisory Board Washington D.C. 2008.
24. Fuller, R.A., Lee, J.R., Watson, J.E.M., Achieving open access to conservation science. Conserv. Biol. 28 (2014), 1550–1557.
25. Gough, D., Oliver, S., Thomas, J., An Introduction to Systematic Reviews. 2012, Sage Publications Ltd, London.
26. Graham, R., Mancher, M., Wolman, D.M., Greenfield, S., Steinberg, E., (eds.) Clinical Practice Guidelines We Can Trust, 2011, National Academies Press, Washington, D.C.
27. Greet, J., Webb, J.A., Cousens, R.D., The importance of seasonal flow timing for riparian vegetation dynamics: a systematic review using causal criteria analysis. Freshw. Biol. 56 (2011), 1231–1247.
28. Haddaway, N., Pullin, A.S., Evidence-based conservation and evidence-informed policy: a response to Adams & Sandbrook. Oryx 47 (2013), 336–338.
29. Haddaway, N.R., Woodcock, P., Macura, B., Collins, A., Making literature reviews more reliable through application of lessons from systematic reviews. Conserv. Biol. 29 (2015), 1596–1605.
30. Hansen, H.F., Rieper, O., The evidence movement: The development and consequences of methodologies in review practices. Evaluation 15 (2009), 141–163.
31. Hedges, L.V., Gurevitch, J., Curtis, P.S., The meta-analysis of response ratios in experimental ecology. Ecology 80 (1999), 1150–1156.
32. Heemskerk, M., Wilson, K., Pavao-Zuckerman, M., Conceptual models as tools for communication across disciplines. Conservation Ecology Series, 7, 2003 (article 8).
33. Higgins, J.P.T., Green, S., (eds.) Cochrane Handbook for Systematic Reviews of Interventions, 2011, Wily-Blackwell, Chichester, UK.
34. Hill, A.B., The environment and disease - association or causation?. Proc. R. Soc. Med.-Lond., 58, 1965.
35. James, K.L., Randall, N.P., Haddaway, N.R., A methodology for systematic mapping in environmental sciences. Environ. Evid. 5 (2016), 1–13.
36. Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., Moher, D., Evidence Summaries: The Evolution of a Rapid Review Approach. Systematic Reviews. 1, 2012, 1–9.
37. Kloprogge, P., van der Sluijs, J., Wardekker, A., Uncertainty Communication: Issues and Good Practice. 2007, Copernican Institute of Sustainable Development, and Innovation, Utrecht.
38. Livoreil, B., Geijzendorffer, I., Pullin, A.S., Schindler, S., Vandewalle, M., Nesshover, C., Biodiversity knowledge synthesis at the European scale: actors and steps. Biodivers. Conserv. 25 (2016), 1269–1284.
39. Margoluis, R., Stem, C., Salafsky, N., Brown, M., Using conceptual models as a planning and evaluation tool in conservation. Eval. Program Plann. 32 (2009), 138–147.
40. McKenzie, J.E., Salanti, G., Lewis, S.C., Altman, D.G., Meta-analysis and The Cochrane Collaboration: 20 years of the Cochrane Statistical Methods Group. Systematic Reviews. 2, 2013, 80.
41. McKinnon, M.C., Cheng, S.H., Dupre, S., Edmond, J., Garside, R., Glew, L., Holland, M.B., Levine, E., Masuda, Y.J., Miller, D.C., Oliveira, I., What are the effects of nature conservation on human well-being? A systematic map of empirical evidence from developing countries. Environ. Evid., 5, 2016, 1.
42. Miller, J., Rees, G., Warnaars, T., Young, G., C., D., Shrestha, A., What is the evidence for glacial shrinkage across the Himalayas? CEE review10-008. Collaboration for environmental evidence. www.environmentalevidence.org/SR10008.html, 2013.
43. Miller, K.A., Webb, J.A., de Little, S.C., Stewardson, M.J., Environmental flows can reduce the encroachment of terrestrial vegetation into river channels: a systematic literature review. Environ. Manag. 52 (2013), 1201–1212.
44. Moher, D., Pham, B., Jones, A., Cook, D.J., Jadad, A.R., Moher, M., Tugwell, P., Klassen, T.P., Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?. Lancet 352 (1998), 609–613.
45. Mulrow, C.D., Systematic reviews - rationale for systematic reviews. Br. Med. J. 309 (1994), 597–599.
46. Newton, A.C., Stewart, G.B., Diaz, A., Golicher, D., Pullin, A.S., Bayesian Belief Networks as a tool for evidence-based conservation management. J. Nat. Conserv. 15 (2007), 144–160.
47. Nichols, S.J., Peat, M., Webb, J.A., Challenges for evidence-based environmental management: What is acceptable and sufficient evidence of causation?. Freshwat. Sci., 36, 2017.
48. Nisbet, I.C.T., Bird conservation: global evidence for the effects of interventions. Book review. Waterbirds, 36, 2013, 390–392.
49. Norris, R.H., Liston, P., Mugodo, J., Nichols, S., Quinn, G.P., Cottingham, P., Metzeling, L., Perriss, S., Robinson, D., Tiller, D., Wilson, G., Multiple lines and levels of evidence for detecting ecological responses to management intervention. 4th Australian Stream Management Conference: Linking Rivers to Landscapes, 2005, Department of Primary Industries, Water and Environment, Launceston, Tasmania, 456–463.
50. Norris, R.H., Webb, J.A., Nichols, S.J., Stewartson, M.J., Harrison, E.T., Analyzing cause and effect in environmental assessments: Using weighted evidence from the literature. Freshwat. Sci. 31 (2012), 5–21.
51. Norton, S.B., Schofield, Conceptual model diagrams as evidence scaffolds for environmental assessment and management. Freshwat. Sci., 36, 2017.
52. Parker, T.H., Forstmeier, W., Koricheva, J., Fidler, F., Hadfield, J.D., Chee, Y.E., Kelly, C.D., Gurevitch, J., Nakagawa, S., Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31 (2016), 711–719.
53. Petticrew, M., Roberts, H., Systematic Reviews in the Social Sciences: A Practical Guide. 2008, Blackwell Publishing, Oxford, Malden, MA.
54. Pullin, A.S., Knight, T.M., Effectiveness in conservation practice: pointers from medicine and public health. Conserv. Biol. 15 (2001), 50–54.
55. Pullin, A.S., Knight, T.M., Support for decision making in conservation practice: an evidence-based approach. J. Nat. Conserv. 11 (2003), 83–90.
56. Pullin, A.S., Knight, T.M., Doing more good than harm - building an evidence-base for conservation and environmental management. Biol. Conserv. 142 (2009), 931–934.
57. Pullin, A.S., Stewart, G.B., Guidelines for systematic review in conservation and environmental management. Conserv. Biol. 20 (2006), 1647–1656.
58. Pullin, A., Frampton, G., Jongman, R., Kohl, C., Livoreil, B., Lux, A., Pataki, G., Petrokofsky, G., Podhora, A., Saarikoski, H., Santamaria, L., Schindler, S., Sousa-Pinto, I., Vandewalle, M., Wittmer, H., Selecting appropriate methods of knowledge synthesis to inform biodiversity policy. Biodivers. Conserv. 25 (2016), 1285–1300.
59. Roberts, P.D., Stewart, G.B., Pullin, A.S., Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine. Biol. Conserv. 132 (2006), 409–423.
60. Shah, N.B., Wang, P., Fraker, A., Gastfriend, D., Evaluations with Impact. Decision-Focused Impact Evaluation as a Practical Policymaking Tool (3ie Working Paper 25), 2015, International Initiative for Impact Evaluation, New Delhi.
61. Smith, R.K., Sutherland, W.J., Amphibian Conservation: Global Evidence for the Effects of Interventions. 2014, Pelagic Publishing, Exeter.
62. Smith, R.K., Pullin, A.S., Stewart, G.B., Sutherland, W.J., Is nest predator exclusion an effective strategy for enhancing bird populations?. Biol. Conserv. 144 (2011), 1–10.
63. Song, F., Eastwood, A.J., Gilbody, S., Duley, L., Sutton, A.J., Publication and related biases. Health Technol. Assess., 4, 2000, 1.
64. Susser, M., What is a cause and how do we know one - a grammar for pragmatic epidemiology. Am. J. Epidemiol. 133 (1991), 635–648.
65. Sutherland, W.J., Pullin, A.S., Dolman, P.M., Knight, T.M., The need for evidence-based conservation. Trends Ecol. Evol. 19 (2004), 305–308.
66. Sutherland, W.J., Dicks, L.V., Ockendon, N., Smith, R.K., (eds.) What Works in Conservation: 2017, 2017, Open Book Publishers, Cambridge, U.K.
67. Tugwell, B., Haynes, R.B., Assessing claims of causation. Haynes, R.B., Sackett, D.L., Guyatt, G.H., Tugwell, B., (eds.) Clinical Epidemiology: How to Do Clinical Practice Research, 2006, Lippincott Williams & Wilkins, Philadelphia, 356–387.
68. Tyler, C., Pullin, A.S., Stewart, G.B., Effectiveness of management interventions to control invasion by Rhododendron ponticum. Environ. Manag. 37 (2006), 513–522.
69. Walsh, J.C., Dicks, L.V., Sutherland, W.J., The effect of scientific evidence on conservation practitioners’ management decisions. Conserv. Biol. 29 (2015), 88–89.
70. Watt, A., Cameron, A., Sturm, L., Lathlean, T., Babidge, W., Blamey, S., Facey, K., Hailey, D., Norderhaug, I., Maddern, G., Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment. Int. J. Technol. Assess. Health Care 24 (2008), 133–139.
71. Webb, J.A., de Little, S.C., Miller, K.A., Stewardson, M.J., Rutherfurd, I.D., Sharpe, A.K., Poff, N.L., A general approach to predicting ecological responses to environmental flows: Making best use of the literature, expert knowledge, and monitoring data. River Res. Appl., 31, 2015.
72. Webb, J.A., Miller, K.A., de Little, S.C., Stewardson, M.J., Nichols, S.J., Wealands, S.R., An online database and desktop assessment software to simplify systematic reviews in environmental science. Environ. Model. Softw. 64 (2015), 72–79.
73. Webb, J.A., Schofield, K., Peat, M., Norton, S.B., Nichols, S.J., Melcher, A., Weaving the common threads in environmental causal assessment methods: towards an ideal method for rapid evidence synthesis. Freshwat. Sci., 36, 2017.
74. Weed, D.L., On the use of causal criteria. Int. J. Epidemiol. 26 (1997), 1137–1141.
75. Williams, D.R., Pople, R.G., Showler, D.A., Dicks, L.V., Child, M.F., Ermgassen, E.K.H.J., Sutherland, W.J., Bird Conservation: Global Evidence for the Effects of Interventions. 2012, Pelagic Publishing, Exeter.
76. Woodcock, P., Pullin, A.S., Kaiser, M.J., Evaluating and improving the reliability of evidence syntheses in conservation and environmental science: a methodology. Biol. Conserv. 176 (2014), 54–62.
77. Young, K.D., Van Aarde, R.J., Science and elephant management decisions in South Africa. Biol. Conserv. 144 (2011), 876–885.
78. Ziegler, C.R., Webb, J.A., Norton, S.B., Pullin, A.S., Melcher, A., Digital repository of associations between environmental variables: a new resource to facilitate knowledge synthesis. Ecol. Indic., 53, 2015.