Research Weaving: Visualizing the Future of Research Synthesis

Trends in Ecology and Evolution

Volumn 34, Issue 3, 2019, Pages 224-238

  Nakagawa, Shinichi ^a,b   Samarasinghe, Gihan ^a   Haddaway, Neal R ^c,d   Westgate, Martin J ^e   O'Dea, Rose E ^a,b   Noble, Daniel W A ^a,f   Lagisz, Malgorzata ^a  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

^b GARVAN INSTITUTE OF MEDICAL RESEARCH   (Australia)

^c STOCKHOLM ENVIRONMENT INSTITUTE   (Sweden)

^d UNIVERSITY OF JOHANNESBURG   (South Africa)

^e AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^f AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

big data; data visualization; evidence synthesis; meta analysis; meta research; systematic review

Indexed keywords

DATA ACQUISITION; DATA ASSIMILATION; DETECTION METHOD; FUTURE PROSPECT; LITERATURE REVIEW; MAPPING METHOD; META-ANALYSIS; RESEARCH METHOD;

BIBLIOMETRICS; ECOLOGY; EVOLUTION; LITERATURE; META ANALYSIS (TOPIC); METHODOLOGY; PROCEDURES;

BIBLIOMETRICS; BIOLOGICAL EVOLUTION; ECOLOGY; META-ANALYSIS AS TOPIC; RESEARCH DESIGN; REVIEW LITERATURE AS TOPIC; SYSTEMATIC REVIEWS AS TOPIC;

EID: 85058691870     PISSN: 01695347     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tree.2018.11.007     Document Type: Review

References (95)

1. Jennions, M.D., et al. Role of meta-analysis in interpreting the scientific literature. Koricheva, J., (eds.) The Handbook of Meta-analysis in Ecology and Evolution, 2013, Princeton University Press, 364–380.
2. Arnqvist, G., Wooster, D., Meta-analysis: synthesizing research findings in ecology and evolution. Trends Ecol. Evol. 10 (1995), 236–240.
3. Shadish, W.R., Lecy, J.D., The meta-analytic big bang. Res. Synth. Methods 6 (2015), 246–264.
4. Gurevitch, J., et al. Meta-analysis and the science of research synthesis. Nature 555 (2018), 175–182.
5. Nakagawa, S., et al. Meta-evaluation of meta-analysis: ten appraisal questions for biologists. BMC Biol., 15, 2017, 18.
6. Glass, G.V., Meta-analysis at middle age: a personal history. Res. Synth. Methods 6 (2015), 221–231.
7. Koricheva, J., et al. The Handbook of Meta-analysis in Ecology and Evolution. 2013, Princeton University Press.
8. Vetter, D., et al. Meta-analysis: a need for well-defined usage in ecology and conservation biology. Ecosphere, 4, 2013, 74.
9. Bates, S., et al. Systematic maps to support the evidence base in social care. Evid. Policy 3 (2007), 539–551.
10. Haddaway, N.R., et al. The benefits of systematic mapping to evidence-based environmental management. Ambio 45 (2016), 613–620.
11. James, K.L., et al. A methodology for systematic mapping in environmental sciences. Environ. Evid., 5, 2016, 7.
12. Miake-Lye, I.M., et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst. Rev., 5, 2016, 28.
13. McKinnon, M.C., et al. Sustainability: map the evidence. Nature 528 (2015), 185–187.
14. O'Leary, B.C., et al. Evidence maps and evidence gaps: evidence review mapping as a method for collating and appraising evidence reviews to inform research and policy. Environ. Evid., 6, 2017, 19.
15. Dicks, L.V., et al. Organising evidence for environmental management decisions: a ‘4S' hierarchy. Trends Ecol. Evol. 29 (2014), 607–613.
16. Sutherland, W.J., Wordley, C.F.R., A fresh approach to evidence synthesis. Nature 558 (2018), 364–366.
17. Cobo, M.J., et al. Science mapping software tools: review, analysis, and cooperative study among tools. J. Am. Soc. Inf. Sci. Technol. 62 (2011), 1382–1402.
18. Zupic, I., Cater, T., Bibliometric methods in management and organization. Organ. Res. Methods 18 (2015), 429–472.
19. van Eck, N.J., Waltman, L., CitNetExplorer: a new software tool for analyzing and visualizing citation networks. J. Informetr. 8 (2014), 802–823.
20. Ding, Y., et al. (eds.) Measuring Scholarly Impact: Methods and Practice, 2014, Springer.
21. Westgate, M.J., et al. Text analysis tools for identification of emerging topics and research gaps in conservation science. Conserv. Biol. 29 (2015), 1606–1614.
22. Booth, A., et al. Systematic Approaches to a Successful Literature Review. 2012, SAGE.
23. Gough, D., et al. An Introduction to Systematic Reviews (2nd edn). 2017, SAGE.
24. Gough, D., et al. Clarifying differences between review designs and methods. Syst. Rev., 1, 2012, 28.
25. Nakagawa, S., Poulin, R., Meta-analytic insights into evolutionary ecology: an introduction and synthesis. Evol. Ecol. 26 (2012), 1085–1099.
26. Higgins, J.P.T., Green, S., Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. 2011, Wiley.
27. Liberati, A., et al. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies that evaluate health care interventions: explanation and elaboration. PLoS Med., 6, 2009, e1000100.
28. Moher, D., et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann. Intern. Med. 151 (2009), 264–269.
29. Haddaway, N.R., et al. ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ. Evid., 7, 2018, 7.
30. Schmidt, F.L., Oh, I.-S., Methods for second order meta-analysis and illustrative applications. Organ. Behav. Hum. Decis. Process. 121 (2013), 204–218.
31. Sigman, M., A meta-analysis of meta-analyses. Fertil. Steril. 96 (2011), 11–14.
32. Ioannidis, J.P.A., Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. Can. Med. Assoc. J. 181 (2009), 488–493.
33. Randall, N.P., James, K.L., The effectiveness of integrated farm management, organic farming and agri-environment schemes for conserving biodiversity in temperate Europe-A systematic map. Environ. Evid., 1, 2012, 4.
34. Neaves, L.E., et al. The fitness consequences of inbreeding in natural populations and their implications for species conservation – a systematic map. Environ. Evid., 4, 2015, 5.
35. Berger-Tal, O., et al. Systematic reviews and maps as tools for applying behavioral ecology to management and policy. Behav. Ecol., 2018, 10.1093/beheco/ary130 Published online October 8, 2018.
36. Aria, M., Cuccurullo, C., bibliometrix: an R-tool for comprehensive science mapping analysis. J. Informetr. 11 (2017), 959–975.
37. Vincenot, C.E., How new concepts become universal scientific approaches: insights from citation network analysis of agent-based complex systems science. Proc. R. Soc. B Biol. Sci., 285, 2018 20172360.
38. Haddaway, N.R., et al. How is the term ‘ecotechnology’ used in the research literature? A systematic review with thematic synthesis. Ecohydrol. Hydrobiol. 18 (2018), 247–261.
39. Haddaway, N.R., et al. The multifunctional roles of vegetated strips around and within agricultural fields. Environ. Evid., 7, 2018, 14.
40. van Eck, N.J., et al. Automatic term identification for bibliometric mapping. Scientometrics 82 (2010), 581–596.
41. Nunez-Mir, G.C., et al. Automated content analysis: addressing the big literature challenge in ecology and evolution. Methods Ecol. Evol. 7 (2016), 1262–1272.
42. LeCun, Y., et al. Deep learning. Nature 521 (2015), 436–444.
43. Schmidhuber, J., Deep learning in neural networks: an overview. Neural Netw. 61 (2015), 85–117.
44. Fidler, F., et al. Metaresearch for evaluating reproducibility in ecology and evolution. Bioscience 67 (2017), 282–289.
45. Ioannidis, J.P.A., Meta-research: why research on research matters. PLoS Biol., 16, 2018, e2005468.
46. Ioannidis, J.P.A., et al. Meta-research: evaluation and improvement of research Methods and practices. PLoS Biol., 13, 2015, e1002264.
47. Ioannidis, J.P.A., Meta-research: the art of getting it wrong. Res. Synth. Methods 1 (2010), 169–184.
48. Parker, T.H., et al. Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31 (2016), 711–719.
49. Nosek, B.A., et al. Promoting an open research culture. Science 348 (2015), 1422–1425.
50. Nakagawa, S., Parker, T.H., Replicating research in ecology and evolution: feasibility, incentives, and the cost-benefit conundrum. BMC Biol., 13, 2015, 88.
51. Chavalarias, D., Ioannidis, J.P.A., Science mapping analysis characterizes 235 biases in biomedical research. J. Clin. Epidemiol. 63 (2010), 1205–1215.
52. Fanelli, D., Glanzel, W., Bibliometric evidence for a hierarchy of the sciences. PLoS One, 8, 2013, e66938.
53. Fanelli, D., et al. Meta-assessment of bias in science. Proc. Natl. Acad. Sci. U. S. A. 114 (2017), 3714–3719.
54. Fanelli, D., Ioannidis, J.P.A., US studies may overestimate effect sizes in softer research. Proc. Natl. Acad. Sci. U. S. A. 110 (2013), 15031–15036.
55. Fortunato, S., et al. Science of science. Science, 359, 2018, eaao0185.
56. Latour, B., Woolgar, S., Laboratory Life: The Construction of Scientific Facts (2nd edn). 1986, Princeton University Press.
57. Barlow, J., et al. On the extinction of the single-authored paper: the causes and consequences of increasingly collaborative applied ecological research. J. Appl. Ecol. 55 (2018), 1–4.
58. Donnelly, C.A., et al. Four principles for synthesizing evidence. Nature 558 (2018), 361–364.
59. Noble, D.W.A., et al. Nonindependence and sensitivity analyses in ecological and evolutionary meta-analyses. Mol. Ecol. 26 (2017), 2410–2425.
60. Greenberg, S.A., How citation distortions create unfounded authority: analysis of a citation network. Br. Med. J., 339, 2009, b2680.
61. de Vries, Y.A., et al. The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression. Psychol. Med. 48 (2018), 2453–2455.
62. Duyx, B., et al. Scientific citations favor positive results: a systematic review and meta-analysis. J. Clin. Epidemiol. 88 (2017), 92–101.
63. Moller, A.P., Jennions, M.D., Testing and adjusting for publication bias. Trends Ecol. Evol. 16 (2001), 580–586.
64. Owens, I.P.F., Where is behavioural ecology going?. Trends Ecol. Evol. 21 (2006), 356–361.
65. Roberts, B.E.I., et al. Taxonomic and geographic bias in conservation biology research: a systematic review of wildfowl demography studies. PLoS One, 11, 2016, e0153908.
66. Troudet, J., et al. Taxonomic bias in biodiversity data and societal preferences. Sci. Rep., 7, 2017, 9132.
67. Allen, W.L., Visual brokerage: communicating data and research through visualisation. Public Underst. Sci., 2018, 0963662518756853.
68. Tsafnat, G., et al. Systematic review automation technologies. Syst. Rev., 3, 2014, 74.
69. Westgate, M.J., et al. Software support for environmental evidence synthesis. Nat. Ecol. Evol., 2, 2018, 588.
70. Kohl, C., et al. Online tools supporting the conduct and reporting of systematic reviews and systematic maps: a case study on CADIMA and review of existing tools. Environ. Evid., 7, 2018, 8.
71. Pullin, A.S., Stewart, G.B., Guidelines for systematic review in conservation and environmental management. Conserv. Biol. 20 (2006), 1647–1656.
72. Lajeunesse, M.J., Facilitating systematic reviews, data extraction and meta-analysis with the metagear package for R. Methods Ecol. Evol. 7 (2016), 323–330.
73. Polderman, T.J.C., et al. Meta-analysis of the heritability of human traits based on fifty years of twin studies. Nat. Genet. 47 (2015), 702–709.
74. Elliott, J.H., et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med., 11, 2014, e1001603.
75. Elliott, J.H., et al. Living systematic review: 1. Introduction-the why, what, when, and how. J. Clin. Epidemiol. 91 (2017), 23–30.
76. Mongeon, P., Paul-Hus, A., The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics 106 (2016), 213–228.
77. Waltman, L., A review of the literature on citation impact indicators. J. Informetr. 10 (2016), 365–391.
78. Haddaway, N.R., et al. The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLoS One, 10, 2015, e0138237.
79. Haddaway, N.R., et al. A rapid method to increase transparency and efficiency in web-based searches. Environ. Evid., 6, 2017, 1.
80. Hampton, S.E., Parker, J.N., Collaboration and productivity in scientific synthesis. Bioscience 61 (2011), 900–910.
81. De Bellis, N., Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics. 2009, Scarecrow Press.
82. Hirsch, J.E., An index to quantify an individual's scientific research output. Proc. Natl. Acad. Sci. U. S. A. 102 (2005), 16569–16572.
83. Garfield, E., Citation indexes to science: a new dimension in documentation through association of ideas. Science 22 (1955), 108–111.
84. Kelly, C.D., Jennions, M.D., The h index and career assessment by numbers. Trends Ecol. Evol. 21 (2006), 167–170.
85. Small, H., Co-citation in the scientific literature: a new measure of the relationship between two documents. J. Assoc. Inf. Sci. Technol. 24 (1973), 265–269.
86. Martyn, J., Bibliographic coupling. J. Doc., 20, 1964 236–236.
87. Hassan-Montero, Y., et al. Graphical interface of the Scimago journal and country rank: an interactive approach to accessing bibliometric information. Prof. Inf. 23 (2014), 272–278.
88. Ioannidis, J.P.A., The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 94 (2016), 485–514.
89. Koricheva, J., Gurevitch, J., Uses and misuses of meta-analysis in plant ecology. J. Ecol. 102 (2014), 828–844.
90. Nakagawa, S., Santos, E.S.A., Methodological issues and advances in biological meta-analysis. Evol. Ecol. 26 (2012), 1253–1274.
91. Nakagawa, S., et al. Comparative and meta-analytic insights into life extension via dietary restriction. Aging Cell 11 (2012), 401–409.
92. Janssens, A.C.J.W., Gwinn, M., Novel citation-based search method for scientific literature: application to meta-analyses. BMC Med. Res. Methodol., 15, 2015, 84.
93. Manning, C.D., et al. An Introduction to Information Retrieval. 2008, Cambridge University Press.
94. Leydesdorff, L., Nerghes, A., Co-word maps and topic modeling: a comparison using small and medium-sized corpora (n < 1,000). J. Assoc. Inf. Sci. Technol. 68 (2017), 1024–1035.
95. Cheng, S., et al. Using machine learning to advance synthesis and use of conservation and environmental evidence. Conserv. Biol. 32 (2018), 762–764.