Making literature reviews more reliable through application of lessons from systematic reviews

Conservation Biology

Volumn 29, Issue 6, 2015, Pages 1596-1605

  Haddaway, N R ^a   Woodcock, P ^b   Macura, B ^c,d   Collins, A ^e  

^a ROYAL SWEDISH ACADEMY OF SCIENCES   (Sweden)

^b JNCC   (United Kingdom)

^c UNIVERSITY OF PADOVA   (Italy)

^d BANGOR UNIVERSITY   (United Kingdom)

^e IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Evidence assessments; Evidence reviews; Meta analysis; Rapid assessments; Rapid reviews

Indexed keywords

ASSESSMENT METHOD; CONSERVATION MANAGEMENT; FEASIBILITY STUDY; LITERATURE REVIEW; META-ANALYSIS; RELIABILITY ANALYSIS; RESOURCE ALLOCATION; STRATEGIC APPROACH; THEORETICAL STUDY;

ENVIRONMENTAL PROTECTION; LITERATURE; PROCEDURES;

CONSERVATION OF NATURAL RESOURCES; REVIEW LITERATURE AS TOPIC;

EID: 84955195002     PISSN: 08888892     EISSN: 15231739     Source Type: Journal    
DOI: 10.1111/cobi.12541     Document Type: Article

References (43)

1. Bernes C, Bråthen KA, Forbes BC, Hofgaard A, Moen J, Speed JD. 2013a. What are the impacts of reindeer/caribou (Rangifer tarandus L.) on arctic and alpine vegetation? A systematic review protocol. Environmental Evidence 2:http://www.environmentalevidencejournal.org/content/2/1/6.
2. Bernes C, Carpenter SR, Gårdmark A, Larsson P, Persson L, Skov C, Van Donk E. 2013b. What is the influence on water quality in temperate eutrophic lakes of a reduction of planktivorous and benthivorous fish? A systematic review protocol. Environmental Evidence 2: http://www.environmentalevidencejournal.org/content/2/1/9.
3. Bilotta GS, Milner AM, Boyd IL. 2014. Quality assessment tools for evidence from environmental science. Environmental Evidence 3: http://www.environmentalevidencejournal.org/content/3/1/14.
4. Borenstein M, Hedges LV, Higgins JPT, Rothstein H. 2009. Introduction to meta-analysis. Wiley, New York.
5. Civil Service. 2010. Rapid evidence assessment toolkit index. Civil Service, London. Available from http://www.civilservice.gov.uk/networks/gsystematicreview/resources-and-guidance/rapid-evidence-assessment (accessed September 2014).
6. Collaboration for Environmental Evidence (CEE). 2013. Guidelines for systematic review and evidence synthesis in environmental management. Version 4.2. Environmental Evidence http://environmentalevidence.org/wp-content/uploads/2014/06/Review-guidelines-version-4.2-final.pdf.
7. Collins A, Miller J, Coughlin D, Kirk S. 2014. The production of quick scoping reviews and rapid evidence assessments a how to guide. Beta version 2 (April 2014). Joint Water Evidence Group A, Defra, London. Available from https://connect.innovateuk.org/documents/3058188/3918930/The%20Production%20of%20QSRs%20and%20REAs-%20A%20How%20to%20guide.pdf?version=1.0 (accessed September 2014).
8. DfID (Department for International Development). 2014. How to guide: assessing the strength of evidence. Available from https://www.gov.uk/government/publications/how-to-note-assessing-the-strength-of-evidence (accessed September 2014).
9. Dickersin K. 1990. The existence of publication bias and risk factors for its occurrence. JAMA 263:1385-1389.
10. Egger M, Smith GD. 1998. Bias in location and selection of studies. BMJ 316:61-66.
11. Englund G, Sarnelle O, Cooper SD. 1999. The importance of data-selection criteria: meta-analyses of stream predation experiments. Ecology 80:1132-1141.
12. Gates S. 2002. Review of methodology of quantitative reviews using meta-analysis in ecology. Journal of Animal Ecology 71:547-557.
13. Gough D, Oliver S, Thomas J. 2012. An introduction to systematic reviews. Sage Publications, London.
14. Haddaway NR. 2015. A call for better reporting of conservation research data for use in meta-analyses. Conservation Biology DOI: 10.1111/cobi.12449.
15. Haddaway NR, Burden A, Evans CD, Healey JR, Jones DL, Dalrymple SE, Pullin AS. 2014a. Evaluating effects of land management on greenhouse gas fluxes and carbon balances in boreo-temperate lowland peatland systems. Environmental Evidence 3: http://www.environmentalevidencejournal.org/content/3/1/5
16. Haddaway NR, Styles D, Pullin AP. 2014b. Evidence on the environmental impacts of farm land abandonment in high altitude/mountain regions: a systematic map. Environmental Evidence 3:17. DOI: 10.1186/2047-2382-3-17.
17. Haddaway NR, Collins A, Coughlin D, Kirk S, Cooke S, Richards R, Stewart R, Johansson S, Knight TM. 2015. Proposal for establishment of CEE rapid evidence review methods. The Collaboration for Environmental Evidence: in press.
18. Halme P, Toivanen T, Honkanen M, Kotiaho JS, Mönkkönen M, Timonen J. 2010. Flawed meta-analysis of biodiversity effects of forest management. Conservation Biology 24:1154-1156.
19. Harrison F. 2011. Getting started with meta-analysis. Methods in Ecology & Evolution 2:1-10.
20. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JAC. 2011. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343:d5928.
21. Huntington BE. 2011. Confronting publication bias in marine reserve meta-analyses. Frontiers in Ecology and the Environment 9:375-376.
22. Koricheva J, Gurevitch J. 2014. Uses and misuses of meta-analysis in plant ecology. Journal of Ecology 102:828-844.
23. Koricheva J, Gurevitch J, Mengersen K. 2013. Handbook of meta-analysis in ecology and evolution. Princeton University Press, New Jersey.
24. Larsen PO, von Ins M. 2010. The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics 84:575-603.
25. Lortie CJ, Aarssen LW, Budden AE, Koricheva JK, Leimu R, Tregenza T. 2007. Publication bias and merit in ecology. Oikos 116:1247-1253.
26. Macura B, Secco L, Pullin AS. 2013. Does the effectiveness of forest protected areas differ conditionally on their type of governance? Environmental Evidence 2 DOI: 10.1186/2047-2382-2-14.
27. McGowan J, Sampson M. 2005. Systematic reviews need systematic searchers. JMLA 93:74-80.
28. Megill A. 2007. Historical knowledge, historical error. The University of Chicago Press, Chicago.
29. Michels C, Schmoch U. 2012. The growth of science and database coverage. Scientometrics 93:831-846.
30. Møller AP, Jennions MD. 2001. Testing and adjusting for publication bias. Trends in Ecology & Evolution 16:580-586.
31. Pautasso M. 2012. Publication growth in biological sub-fields: patterns, predictability and sustainability. Sustainability 4:3234-3247.
32. Petticrew M. 2001. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ 322:98-101.
33. Petticrew M, Roberts H. 2008. Systematic reviews in the social sciences: a practical guide. Blackwell Publishing, Oxford, Malden, MA.
34. Philibert A, Loyce C, Makowski D. 2012. Assessment of the quality of meta-analysis in agronomy. Agriculture, Ecosystems & Environment 148:72-82.
35. Pullin AS, et al. 2013. Human well-being impacts of terrestrial protected areas. Environmental Evidence 2: DOI:10.1186/2047-2382-2-19.
36. Pullin AS, Stewart GB. 2006. Guidelines for systematic review in conservation and environmental management. Conservation Biology 20:1647-1656.
37. Randall NP, James KL. 2012. The effectiveness of integrated farm management, organic farming and agri-environment schemes for conserving biodiversity in temperate Europe - a systematic map. Environmental Evidence 1: DOI: 10.1186/2047-2382-1-4.
38. Roberts PD, Stewart GB, Pullin AS. 2006. Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine. Biological Conservation 132:409-423.
39. Spencer L, Ritchie J, Lewis J, Dillon L. 2003. Quality in qualitative evaluation: a framework for assessing research evidence. Cabinet Office http://www.civilservice.gov.uk/wp-content/uploads/2011/09/a_quality_framework_tcm6-38740.pdf (accessed September 2014).
40. Sterne JA, Egger M, Smith GD. 2001. Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis. BMJ 323:101-105.
41. The Cochrane Collaboration. 2011. Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. Higgins J, Green S, editors. Available from www.cochrane-handbook.org (accessed September 2014).
42. Torgerson CJ. 2006. Publication bias: The Achilles' heel of systematic reviews? British Journal of Educational Studies 54:89-102.
43. Woodcock P, Pullin AS, Kaiser MJ. 2014. Evaluating and improving the reliability of evidence syntheses in conservation and environmental science: a methodology. Biological Conservation 176:54-62.