How to critically read ecological meta-analyses

Research Synthesis Methods

Volumn 6, Issue 2, 2015, Pages 124-133

  Lortie, Christopher J ^a   Stewart, Gavin ^b   Rothstein, Hannah ^c   Lau, Joseph ^d  

^a YORK UNIVERSITY   (Canada)

^b UNIVERSITY OF YORK   (United Kingdom)

^c CITY UNIVERSITY OF NEW YORK   (United States)

^d TUFTS MEDICAL CENTER   (United States)

Author keywords

And synthesis; Criteria; Ecology; Guidelines; Interpretation; Meta analysis; Reading

Indexed keywords

DATA MINING; ECOLOGY; MEASUREMENT ACCURACY; META ANALYSIS (TOPIC); METHODOLOGY; PROCEDURES; READING; STATISTICAL ANALYSIS;

DATA ACCURACY; DATA INTERPRETATION, STATISTICAL; DATA MINING; ECOLOGY; META-ANALYSIS AS TOPIC; READING; RESEARCH DESIGN;

EID: 84938676792     PISSN: None     EISSN: 17592887     Source Type: Journal    
DOI: 10.1002/jrsm.1109     Document Type: Article

References (78)

1. Andersson P, Lofstedt C, Hamback PA. 2013. How insects sense olfactory patches-the spatial scaling of olfactory information. Oikos.
2. Arnquist G, Wooster D. 1995. Meta-analysis: synthesizing research findings in ecology and evolution. Trends in Ecology and Evolution 10: 236-240.
3. Barto EK, Rillig M. 2012. Dissemination biases in ecology: effect sizes matter more than quality. Oikos 121: 228-235.
4. Blum MGB, Francois O, Janson S. 2006. The mean, variance and limiting distribution of two statistics sensitive to phylogenetic tree balance. The Annals of Applied Probability 16: 2195-2214.
5. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. 2010. A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods 1: 97-111.
6. Cadotte MW, Mehrkens LR, Menge DNL. 2012. Gauging the impact of meta-analysis on ecology. Evolutionary Ecology 26: 1153-67.
7. Carpenter SR, Armbrust VE, Arzberger PW, Chapin FS, Elser JJ, Hackett EJ, Ives AR, Kareiva PM, Leibold MA, Lundberg P, Mangel M, Merchant N, Murdoch WM, Palmer MA, Peters D, Pickett S, Smith KK, Wall DH, Zimmerman AS. 2009. Accelerate synthesis in ecology and environmental sciences. Bioscience 59: 699-701.
8. Castellanos MC, Verdu M. 2012. Meta-analysis of meta-analyses in plant evolutionary ecology. Evolutionary Ecology 26: 1187-1196.
9. Chamberlain SA, Hovick SM, Dibble CJ, Rasmussen NL, Van Allen BG, Maitner BS, Ahern JR, Bell-Dereske LP, Roy CL, Meza-Lopez M, Carrillo J, Siemann E, Lajeunesse MJ, Whitney KD 2012. Does phylogeny matter? Assessing the impact of phylogenetic information in ecological meta-analysis. Ecology Letters 15: 627-636.
10. Chaudhary VB, Walters LL, Bever JD 2010. Advancing synthetic ecology: a database system to facilitate complex ecological meta-analyses. Bulletin of the Ecological Society of America 91: 235-43.
11. Cohen J. 1992. A power primer. Psychological Bulletin 112: 155-159.
12. Cote IM, Jennions MD. 2013. The procedure of meta-analysis in a nutshell. In Handbook of Meta-analysis in Ecology and Evolution, Koricheva J, Gurevitch J, Mengersen K (eds.). Princeton University Press; 14-26.
13. Cote IM, Curtis PS, Rothstein HR, Stewart G 2013. Gathering data: searching literature and criteria selection. In Handbook of Meta-analysis in Ecology and Evolution, Koricheva J, Gurevitch J, Mengersen K (eds). Princeton University Press: Oxford; 37-51.
14. Ford ED. 2000. Scientific Method for Ecological Research. Cambridge University Press: Cambridge.
15. Ford ED, Ishii H. 2001. The method of synthesis in ecology. Oikos 93: 153-160.
16. Garcia-Closas M, Maestre F, Kattge J, Wall DH 2013. Climate and litter quality differently modulate the effects of soil fauna on litter decomposition across biomes. Ecology Letters 16: 1045-1053.
17. Gates S. 2002. Review of methodology of quantitative reviews using meta-analysis in ecology. Journal of Animal Ecology 71: 547-557.
18. Gurevitch J, Hedges LV 1993. Meta-analysis: combining the results of independent experiments. In Design and analysis of ecological experiments, Gurevitch J (ed.). Chapman and Hall: New York.
19. Gurevitch J, Morrow LL, Wallace A, Walsh, JS. 1992. A meta-analysis of competition in field experiments. American Naturalist 140: 539-572.
20. Hadfield JD, Nakagawa S. 2009. General quantitative genetic methods for comparative biology: phylogenies, taxonomies and multi-trait models for continuous and categorical characters. Journal of Evolutionary Biology 23: 494-508.
21. Hampton SE, Strasser CA, Tewksbury JJ, Gram WK, Budden AE, Batcheller AL, Duke CS, Porter JH. 2013. Big data and the future of ecology. Frontiers in Ecology & the Environment 11: 156-162.
22. Harrison F. 2011. Getting started with meta-analysis. Methods in Ecology and Evolution 2: 1-10.
23. He Q, Bertness MD, Altieri AH. 2013. Global shifts towards positive species interactions with increasing environmental stress. Ecology Letters 16: 695-706.
24. Higgins JPT, Green S. 2006. Cochrane Handbook for Systematic Reviews of Interventions 4.2.6. The Cochrane Library 4.
25. Higgins JPT, Green S. 2011. Cochrane handbook for systematic reviews of interventions 5.1.0. The Cochrane Library 5.1.
26. Higgins JPT, Thompson SG. 2002. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 21: 1539-1558.
27. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. 2003. Measuring inconsistency in meta-analyses. British Medical Journal 327: 557-560.
28. Humphrey SE. 2011. What does a great meta-analysis look like? Organizational Psychology Review 1: 99-103.
29. Ioannidis PA. 2005. Why most published findings are false. Plos Medicine 2: e124.
30. Ioannidis PA, Lau J. 2001. Evolution of treatment effects over time: empirical insight from recursive metaanlyses. Proceedings of the National Academy of Science 98: 831-836.
31. Ioannidis PA, Cappelleri JC, Lau J. 1998. Issues in comparisons between meta-analyses and large trials. Journal of American Medical Association 279: 1089-1093.
32. Jennions MD, Moller AP. 2002. Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution. Proceedings of the Royal Society of London B 269: 43-48.
33. Jennions MD, Lortie CJ, Koricheva J. 2013a. Role of meta-analyses in interpreting the scientific literature. In Handbook of Meta-analysis in Ecology and Evolution, Koricheva J, Gurevitch J, Mengersen K (eds). Princeton University Press; 364-380.
34. Jennions MD, Lortie CJ, Rosenberg MS, Rothstein HR. 2013b. Publication and related biases. In Handbook of Meta-analysis in Ecology and Evolution, Koricheva J, Gurevitch J, Mengersen K (eds). Princeton University Press; 207-236.
35. Juni P, Witschi A, Bloch R, Egger M. 1999. The hazards of scoring the quality of clinical trials for meta-analysis. Journal of the American Medical Association 282: 1054-1060.
36. Kelly CD. 2006. Replicating empirical research in behavioral ecology: how and why it should be done but rarely ever is. The Quarterly Review of Biology 81: 221-236.
37. Koricheva J 2003. Non-significant results in ecology: a burden or a blessing in disguise? Oikos 102: 397-401.
38. Koricheva J, Gurevitch J. 2013. Place of meta-analysis among other methods of research synthesis. In Handbook of Meta-analysis in Ecology and Evolution, Koricheva J, Gurevitch J, Mengersen K. (eds). Princeton University Press; 3-13.
39. Koricheva J, Gurevitch J, Mengersen K. 2013. Handbook of Meta-analysis in Ecology and Evolution. Princeton University Press.
40. Kotiaho JS, Tomkins JL. 2002. Meta-analysis, can it ever fail? Oikos 96: 551-553.
41. Lajeunesse M. 2011. phyloMeta: a program for phylogenetic comparative analyses with meta-analysis. Bioinformatics 27: 2603-2604.
42. Lajeunesse M, Rosenberg MS, Jennions MD. 2013. Phylogenetic nonindependence and meta-analysis. In Handbook of Meta-analysis in Ecology and Evolution, Koricheva J, Gurevitch J, Mengersen K (eds). Princeton University Press; 284-299.
43. Lamarque LJ, Delzon S, Lortie CJ. 2011. Tree invasions: a comparative test of the dominant hypotheses and predictor traits. Biological Invasions 13: 1969-1989.
44. Leimu R, Koricheva J. 2004. Cumulative meta-analysis: a new tool for detection of temporal trends and publication bias in ecology. Proceedings of the Royal Society of London B 271: 1961-1966.
45. Leucht S, Kissling W, Davis JM. 2009. How to read and understand and use systematic reviews and meta-analyses. Acta Psychiatrica Scandinavica 119: 443-450.
46. Lipsey MW, Wilson DB. 2001. Practical Meta-analysis. Sage Publications Inc: Thousand Oaks, California.
47. Lortie CJ, Callaway RM. 2006. Re-analysis of meta-analysis: a test of the stress-gradient hypothesis. Journal of Ecology 94: 7-16.
48. Lortie CJ, Lau J, Lajeunesse M. 2013. Graphical presentation of results. In Handbook of meta-analysis in ecology and evolution, Koricheva J, Gurevitch J, Mengersen K (eds). Princeton University Press; 339-347.
49. Maestre FT, Cortina J. 2004. Do positive interactions increase with abiotic stress? A test from a semi-arid steppe. Proc. R. Soc. Lond. B 271: s331-s333.
50. Michalet R, Schob C, Lortie CJ, Brooker RW, Callaway RM. 2013. Partitioning net interactions among plants along altitudinal gradients to study community responses to climate change. Functional Ecology. doi: 10.1111/1365- 2435.12136.
51. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. 1995. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Controlled Clinical Trials 16: 62-73.
52. Moher D, Jadad AR, Tugwell P. 1996. Assessing the quality of randomized controlled trials: current issues and future directions. International Journal of Technology Assessment in Health Care 12: 195-208.
53. Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP. 2009. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339: b2535.
54. Moller AP, Jennions MD. 2001. Testing and adjusting for publication bias. Trends in Ecology and Evolution 16: 580-586.
55. Moller AP, Jennions MD. 2002. How much variance can be explained by ecologists and evolutionary biologists? Oecologia 132: 492-500.
56. Nakagawa S, Cuthill IC. 2007. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological Reviews 82: 591-605.
57. Nakagawa S, Poulin R. 2012. meta-analytic insights into evolutionary ecology: an introduction and synthesis. Evolutionary Ecology 26: 1085-1099.
58. Nakagawa S, Santos ES. 2012. Methodological issues and advances in biological meta-analysis. Evolutionary Ecology 26: 1253-1274.
59. Osenberg CW, Sarnelle O, Cooper SD, Holt RD. 1999. Resolving ecological questions through meta-analysis: goals, metrics, and models. Ecology 80: 1105-1117.
60. Paine RT. 2002. Advances in ecological understanding: by Kuhnian revolution or conceptual evolution. Ecology 83: 1553-1559.
61. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. 2007. Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity. Statistics in Medicine 26: 4544-4562.
62. Poon AFY, Walker LW, Murray H, McCloskey RM, Harrigan PR, Liang RH. 2013. Mapping the shapes of phylogenetic trees from human and zoonotic RNA viruses. PLoS ONE 8: e78122.
63. Price J, Partel M. 2013. Can limiting similarity increase invasion resistance? A meta-analysis of experimental studies. Oikos 122: 649-656.
64. Pullin AS, Stewart GB 2006. Guidelines for Systematic Review in Conservation and Environmental Management. Conservation Biology 20: 1647-1656.
65. Ried K 2006. Interpreting and understanding meta-analysis graphs. Australian Family Physician 35: 635-638.
66. Robinson SA, Lajeunesse M, Forbes MR. 2012. Sex differences in mercury contamination of birds: testing multiple hypotheses with meta-analysis. Environmental Science & Technology 46: 7094-7101.
67. Root TL, Price JT, Hall KR, Schneider SH, Rosenzweig C, Pounds JA, 2003. Fingerprints of global warming on wild animals and plants. Nature 421: 57-60.
68. Rothstein HR, Lortie CJ, Stewart G. 2013. Quality standards for research syntheses. In Handbook of Meta-analysis in Ecology and Evolution, Koricheva J, Gurevitch J, Mengersen, K (eds). Princeton University Press; 323-228.
69. Russo MW. 2007. How to review a meta-analysis. Gastroenterology & Hepatology 3: 637-642.
70. Sidlauskas B, Ganapathy G, Jenkins KP, Lapp H, McCall LW, Price S, Scherle R, Spaeth PA, Kidd DM. 2009. Linking big: the continuing promise of evolutionary synthesis. Evolution 64: 871-880.
71. Sterne JAC, Smith GD. 2001. Sifting the evidence-what's wrong with significance tests? British Medical Journal 322: 226-231.
72. Stewart G. 2010. Meta-analysis in applied ecology. Biology Letters 6: 78-81.
73. Stroup DF, Berlin JA, Morton SC, et al. 2000. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA 283: 2008-2012.
74. Tomkins JL, Kotiaho JS. 2004. Publication bias in meta-analysis: seeing the wood for the trees. Oikos 104: 194-196.
75. Treadwell JR, Tregar SJ, Reston JT, Turkelson CM. 2007. A system for rating the stability and strength of medical evidence. BMC Medical Research Methodology 6: 52.
76. Vetter D, Rucker G, Storch I. 2013. Meta-analysis: A need for well-defined usage in ecology and conservation biology. Ecosphere 4: 74.
77. Wacholder S, Chancock S, Garcia-Closas M, El ghormli L, Rothman N. 2004. Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. National Cancer Institute 96: 434-442.
78. Walker E, Hernandez AV, Kattan MW 2008. Meta-analysis: its strengths and limitations. Cleveland Clinic Journal of Medicine 75: 431-439.