A primer on systematic reviews in toxicology

Archives of Toxicology

Volumn 91, Issue 7, 2017, Pages 2551-2575

  Hoffmann, Sebastian ^a,l   de Vries, Rob B M ^b   Stephens, Martin L ^a   Beck, Nancy B ^c   Dirven, Hubert A A M ^d   Fowle, John R ^e   Goodman, Julie E ^f   Hartung, Thomas ^a   Kimber, Ian ^g   Lalu, Manoj M ^h   Thayer, Kristina ⁱ   Whaley, Paul ^j   Wikoff, Daniele ^k   Tsaioun, Katya ^a  

^a JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

^b RADBOUD UNIVERSITY MEDICAL CENTER   (Netherlands)

^c American Chemistry Council   (United States)

^d NORWEGIAN INSTITUTE OF PUBLIC HEALTH   (Norway)

^e Science to Inform LLC   (United States)

^f GRADIENT   (United States)

^g UNIVERSITY OF MANCHESTER   (United Kingdom)

^h OTTAWA HOSPITAL RESEARCH INSTITUTE   (Canada)

ⁱ U S ENVIRONMENTAL PROTECTION AGENCY   (United States)

^j LANCASTER UNIVERSITY   (United Kingdom)

^k ToxStrategies Inc   (United States)

^l Seh Consulting + Services   (Germany)

more..

Author keywords

Evidence synthesis; Evidence based toxicology; Narrative review; Review steps; Systematic review

Indexed keywords

HUMAN; NONHUMAN; PRIORITY JOURNAL; PUBLISHING; REVIEW; SYSTEMATIC REVIEW (TOPIC); TOXICOLOGY; META ANALYSIS (TOPIC); PROCEDURES;

META-ANALYSIS AS TOPIC; TOXICOLOGY;

EID: 85019269575     PISSN: 03405761     EISSN: 14320738     Source Type: Journal    
DOI: 10.1007/s00204-017-1980-3     Document Type: Review

References (68)

1. AHRQ (2014) Methods guide for effectiveness and comparative effectiveness reviews. https://effectivehealthcare.ahrq.gov/ehc/products/60/318/CER-Methods-Guide-140109.pdf. Accessed 13 Feb 2017
2. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH (2011) GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 64(4):401–406. doi:10.1016/j.jclinepi.2010.07.015
3. Bero L, Anglemyer A, Vesterinen H, Krauth D (2015) The relationship between study sponsorship, risks of bias, and research outcomes in atrazine exposure studies conducted in non-human animals: systematic review and meta-analysis. Environ Int 92–93:597–604. doi:10.1016/j.envint.2015.10.011
4. Birnbaum LS, Thayer KA, Bucher JR, Wolfe MS (2013) Implementing systematic review at the National toxicology program: status and next steps. Environ Health Perspect 121(4):A108–A109. doi:10.1289/ehp.1306711
5. Cedergreen N (2014) Quantifying synergy: a systematic review of mixture toxicity studies within environmental toxicology. PLoS One 9(5):e96580. doi:10.1371/journal.pone.0096580
6. CEE (2013) Guidelines for systematic reviews in environmental management. Version 4.2, http://www.environmentalevidence.org/wp-content/uploads/2017/01/Review-guidelines-version-4.2-final-update.pdf. Accessed 13 Feb 2017
7. CRD (2009) Systematic reviews: CRD’s guidance for undertaking systematic reviews in health care. https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf. Accessed 13 Feb 2017
8. de Vries RB, Leenaars M, Tra J, Huijbregtse R, Bongers E, Jansen JA, Gordijn B, Ritskes-Hoitinga M (2013) The potential of tissue engineering for developing alternatives to animal experiments: a systematic review. J Tissue Eng Regen Med 9(7):771–778. doi:10.1002/term.1703
9. de Vries RB, Wever KE, Avey MT, Stephens ML, Sena ES, Leenaars M (2014) The usefulness of systematic reviews of animal experiments for the design of preclinical and clinical studies. ILAR J 55(3):427–437. doi:10.1093/ilar/ilu043
10. de Vries RB, Hooijmans CR, Langendam MW, van Luijk J, Leenaars M, Ritskes-Hoitinga M, Wever KE (2015) A protocol format for the preparation, registration and publication of systematic reviews of animal intervention studies. Evid Based Preclin Med 2(1):1–9. doi:10.1002/ebm2.7
11. Dello SA, Neis EP, de Jong MC, van Eijk HM, Kicken CH, Olde Damink SW, Dejong CH (2013) Systematic review of ophthalmate as a novel biomarker of hepatic glutathione depletion. Clin Nutr 32(3):325–330. doi:10.1016/j.clnu.2012.10.008
12. EFSA (2010) Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA J 8(6):1637
13. Efsa, ANS Panel (EFSA Panel on Food Additives and Nutrient Sources added to Food) (2015) Scientific opinion on the risk assessment for peri- and post-menopausal women taking food supplements containing isolated isoflavones. EFSA J 13(10):4246. doi:10.2903/j.efsa.2015.4246 (pp342)
14. Goodman M, LaVerda N, Clarke C, Foster ED, Iannuzzi J, Mandel J (2002) Neurobehavioural testing in workers occupationally exposed to lead: systematic review and meta-analysis of publications. Occup Environ Med 59(4):217–223
15. Goodman JE, Petito Boyce C, Sax SN, Beyer LA, Prueitt RL (2015) Rethinking meta-analysis: applications for air pollution data and beyond. Risk Anal 35(6):1017–1039. doi:10.1111/risa.12405
16. Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knottnerus A (2011) GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol 64(4):380–382. doi:10.1016/j.jclinepi.2010.09.011
17. Guzelian PS, Victoroff MS, Halmes NC, James RC, Guzelian CP (2005) Evidence-based toxicology: a comprehensive framework for causation. Hum Exp Toxicol 24:161–201
18. Haddaway NR, Land M, Macura B (2017) A little learning is a dangerous thing”: a call for better understanding of the term ‘systematic review. Environ Int 99:356–360. doi:10.1016/j.envint.2016.12.020
19. Higgins JPT, Green S (editors) (2011) Cochrane Handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The cochrane collaboration
20. Hoffmann S, Hartung T (2006) Toward an evidence-based toxicology. Hum Exp Toxicol 25(9):497–513
21. Hooijmans CR, Leenaars M, Ritskes-Hoitinga M (2010) A gold standard publication checklist to improve the quality of animal studies, to fully integrate the Three Rs, and to make systematic reviews more feasible. Altern Lab Anim 38(2):167–182
22. Hooijmans CR, Rovers MM, de Vries RB, Leenaars M, Ritskes-Hoitinga M, Langendam MW (2014a) SYRCLE’s risk of bias tool for animal studies. BMC Med Res Methodol 26:14–43. doi:10.1186/1471-2288-14-43
23. Hooijmans CR, IntHout J, Ritskes-Hoitinga M, Rovers MM (2014b) Meta-analyses of animal studies: an introduction of a valuable instrument to further improve healthcare. ILAR J 55(3):418–426. doi:10.1093/ilar/ilu042
24. Hopewell S, McDonald S, Clarke M, Egger M (2007) Gray literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev 18(2):MR000010
25. IOM (2011) Finding what works in health care: Standards for systematic reviews. Free download at https://www.nap.edu/catalog/13059/finding-what-works-in-health-care-standards-for-systematic-reviews. Accessed 13 Feb 2017
26. Johnson PI, Sutton P, Atchley DS, Koustas E, Lam J, Sen S, Robinson KA, Axelrad DA, Woodruff TJ (2014) The Navigation Guide—evidence-based medicine meets environmental health: systematic review of human evidence for PFOA effects on fetal growth. Environ Health Perspect 122(10):1028–1039. doi:10.1289/ehp.1307893
27. Kilkenny C, Browne W, Cuthill I, Emerson M, Altman D (2010) Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. J Pharmacol Pharmacother 1(2):94–99. doi:10.4103/0976-500X.72351
28. Koustas E, Lam J, Sutton P, Johnson PI, Atchley DS, Sen S, Robinson KA, Axelrad DA, Woodruff TJ (2014) The Navigation Guide—evidence-based medicine meets environmental health: systematic review of nonhuman evidence for PFOA effects on fetal growth. Environ Health Perspect 122(10):1015–1027. doi:10.1289/ehp.1307177
29. Krauth D, Woodruff TJ, Bero L (2013) Instruments for assessing risk of bias and other methodological criteria of published animal studies: a systematic review. Environ Health Perspect 121(9):985–992. doi:10.1289/ehp.1206389
30. Lalu MM, Sullivan KJ, Mei SH, Moher D, Straus A, Fergusson DA, Stewart DJ, Jazi M, MacLeod M, Winston B, Marshall J, Hutton B, Walley KR, McIntyre L (2016) Evaluating mesenchymal stem cell therapy for sepsis with preclinical meta-analyses prior to initiating a first-in-human trial. Elife 5:e17850. doi:10.7554/eLife.17850
31. Lam J, Koustas E, Sutton P, Johnson PI, Atchley DS, Sen S, Robinson KA, Axelrad DA, Woodruff TJ (2014) The navigation guide—evidence-based medicine meets environmental health: integration of animal and human evidence for PFOA effects on fetal growth. Environ Health Perspect 122(10):1040–1051. doi:10.1289/ehp.1307923
32. Lam J, Sutton P, Kalkbrenner A, Windham G, Halladay A, Koustas E, Lawler C, Davidson L, Daniels N, Newschaffer C, Woodruff T (2016) A systematic review and meta-analysis of multiple airborne pollutants and autism spectrum disorder. PLoS One 11(9):e0161851. doi:10.1371/journal.pone.0161851
33. Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, Crystal RG, Darnell RB, Ferrante RJ, Fillit H, Finkelstein R, Fisher M, Gendelman HE, Golub RM, Goudreau JL, Gross RA, Gubitz AK, Hesterlee SE, Howells DW, Huguenard J, Kelner K, Koroshetz W, Krainc D, Lazic SE, Levine MS, Macleod MR, McCall JM, Moxley RT 3rd, Narasimhan K, Noble LJ, Perrin S, Porter JD, Steward O, Unger E, Utz U, Silberberg SD (2012) A call for transparent reporting to optimize the predictive value of preclinical research. Nature 490(7419):187–191. doi:10.1038/nature11556
34. Levac D, Colquhoun H, O’Brien KK (2010) Scoping studies: advancing the methodology. Implement Sci 5:69. doi:10.1186/1748-5908-5-69
35. McGowan J, Sampson M (2005) Systematic reviews need systematic searchers. J Med Libr Assoc 93(1):74–80
36. Meyer-Baron M (2005) A meta-analytical approach to neurobehavioural effects of occupational toluene exposure. Environ Toxicol Pharmacol 19(3):651–657. doi:10.1016/j.etap.2004.12.032
37. Meyer-Baron M, Knapp G, Schäper M, van Thriel C (2009) Performance alterations associated with occupational exposure to manganese: a meta-analysis. NeuroToxicology 30(4):487–496. doi:10.1016/j.neuro.2009.05.001
38. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(6):e1000097. doi:10.1371/journal.pmed1000097
39. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA (2015) Preferred Reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 4(1):1. doi:10.1186/2046-4053-4-1
40. Morgan RL, Thayer KA, Bero L, Bruce N, Falck-Ytter Y, Ghersi D, Guyatt G, Hooijmans C, Langendam M, Mandrioli D, Mustafa RA, Rehfuess EA, Rooney AA, Shea B, Silbergeld EK, Sutton P, Wolfe MS, Woodruff TJ, Verbeek JH, Holloway AC, Santesso N, Schünemann HJ (2016) GRADE: assessing the quality of evidence in environmental and occupational health. Environ Int 92–93:611–616. doi:10.1016/j.envint.2016.01.004
41. National Research Council (NRC) (2014) Review of the EPA’s Integrated Risk Information System (IRIS) Process. National Academies Press, Washington
42. National Toxicology Program (2015) Handbook for conducting a literature-based health assessment using office of health assessment and translation (OHAT) approach for systematic review and evidence integration. https://ntp.niehs.nih.gov/ntp/ohat/pubs/handbookjan2015_508.pdf. Accessed 13 Feb 2017
43. National Toxicology Program (2016) Systematic review of the effects of fluoride on learning and memory in animal studies. NTP Research Report 1. https://ntp.niehs.nih.gov/ntp/ohat/pubs/ntp_rr/01fluoride_508.pdf. Accessed 20 Feb 2017
44. Navas-Acien A, Guallar E, Silbergeld EK, Rothenberg SJ (2007) Lead exposure and cardiovascular disease: a systematic review. Environ Health Perspect 115(3):472–482
45. Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB (2015) Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc 13(3):141–146. doi:10.1097/XEB.0000000000000050
46. Rhomberg L (2015) Hypothesis-based weight-of-evidence: an approach to assessing causation and its application to regulatory toxicology. Risk Anal 35(6):1114–1124. doi:10.1111/risa.12206
47. Rooney AA, Boyles AL, Wolfe MS, Bucher JR, Thayer KA (2014) Systematic review and evidence integration for literature-based environmental health science assessments. Environ Health Perspect 122(7):711–718. doi:10.1289/ehp.1307972
48. Rudén C (2001) Interpretations of primary carcinogenicity data in 29 trichloroethylene risk assessments. Toxicology 169(3):209–225
49. Samuel GO, Hoffmann S, Wright RA, Lalu MM, Patlewicz G, Becker RA, DeGeorge GL, Fergusson D, Hartung T, Lewis RJ, Stephens ML (2016) Guidance on assessing the methodological and reporting quality of toxicologically relevant studies: a scoping review. Environ Int 92–93:630–646. doi:10.1016/j.envint.2016.03.010
50. Schreider J, Barrow C, Birchfield N, Dearfield K, Devlin D, Henry S, Kramer M, Schappelle S, Solomon K, Weed DL, Embry MR (2010) Enhancing the credibility of decisions based on scientific conclusions: transparency is imperative. Toxicol Sci 116(1):5–7. doi:10.1093/toxsci/kfq102
51. Segal D, Makris SL, Kraft AD, Bale AS, Fox J, Gilbert M, Bergfelt DR, Raffaele KC, Blain RB, Fedak KM, Selgrade MK, Crofton KM (2015) Evaluation of the ToxRTool’s ability to rate the reliability of toxicological data for human health hazard assessments. Regul Toxicol Pharmacol 72(1):94–101. doi:10.1016/j.yrtph.2015.03.005
52. Sena ES, Currie GL, McCann SK, Macleod MR, Howells DW (2014) Systematic reviews and meta-analysis of preclinical studies: why perform them and how to appraise them critically. J Cereb Blood Flow Metab 34(5):737–742. doi:10.1038/jcbfm.2014.28
53. Silbergeld E, Scherer RW (2013) Evidence-based toxicology: strait is the gate, but the road is worth taking. Altex 30(1):67–73
54. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, Hing C, Kwok CS, Pang C, Harvey I (2014) Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess 14(8):1–193. doi:10.3310/hta14080
55. Stephens ML, Andersen M, Becker RA, Betts K, Boekelheide K, Carney E, Chapin R, Devlin D, Fitzpatrick S, Fowle JR 3rd, Harlow P, Hartung T, Hoffmann S, Holsapple M, Jacobs A, Judson R, Naidenko O, Pastoor T, Patlewicz G, Rowan A, Scherer R, Shaikh R, Simon T, Wolf D, Zurlo J (2013) Evidence-based toxicology for the 21st century: opportunities and challenges. Altex 30(1):74–103
56. Stephens ML, Betts K, Beck NB, Cogliano V, Dickersin K, Fitzpatrick S, Freeman J, Gray G, Hartung T, McPartland J, Rooney AA, Scherer RW, Verloo D, Hoffmann S (2016) The emergence of systematic review in toxicology. Toxicol Sci 152(1):10–16. doi:10.1093/toxsci/kfw059
57. TCEQ (Texas Commission on Environmental Quality) (2014) TCEQ recommendations for systematic review and evidence integration. http://www.tceq.com/assets/public/implementation/tox/dsd/proposed/nov14/systematic_review.pdf. Accessed 13 Feb 2017
58. Thayer KA, Wolfe MS, Rooney AA, Boyles AL, Bucher JR, Birnbaum LS (2014) Intersection of systematic review methodology with the NIH reproducibility initiative. Environ Health Perspect 122(7):A176–A177. doi:10.1289/ehp.1408671
59. Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E (2014) Systematic review automation technologies. Systematic reviews 3:74. doi:10.1186/2046-4053-3-74
60. Tsuji JS, Perez V, Garry MR, Alexander DD (2014) Association of low-level arsenic exposure in drinking water with cardiovascular disease: a systematic review and risk assessment. Toxicology 323:78–94. doi:10.1016/j.tox.2014.06.008
61. Vesterinen HM, Sena ES, Egan KJ, Hirst TC, Churolov L, Currie GL, Antonic A, Howells DW, Macleod MR (2014) Meta-analysis of data from animal studies: a practical guide. J Neurosci Method. 221:92–102. doi:10.1016/j.jneumeth.2013.09.010
62. Wandall B, Hansson SO, Rudén C (2007) Bias in toxicology. Arch Toxicol 81(9):605–617
63. Wever KE, Menting TP, Rovers M, van der Vliet JA, Rongen GA, Masereeuw R, Ritskes-Hoitinga M, Hooijmans CR, Warlé M (2012) Ischemic preconditioning in the animal kidney, a systematic review and meta-analysis. PLoS One 7(2):e32296. doi:10.1371/journal.pone.0032296
64. Wever KE, Hooijmans CR, Riksen NP, Sterenborg TB, Sena ES, Ritskes-Hoitinga M, Warlé MC (2015) Determinants of the efficacy of cardiac ischemic preconditioning: a systematic review and meta-analysis of animal studies. PLoS One 10(11):e0142021. doi:10.1371/journal.pone.0142021
65. Whaley P, Halsall C, Ågerstrand M, Aiassa E, Benford D, Bilotta G, Coggon D, Collins C, Dempsey C, Duarte-Davidson R, FitzGerald R, Galay-Burgos M, Gee D, Hoffmann S, Lam J, Lasserson T, Levy L, Lipworth S, Ross SM, Martin O, Meads C, Meyer-Baron M, Miller J, Pease C, Rooney A, Sapiets A, Stewart G, Taylor D (2015) Implementing systematic review techniques in chemical risk assessment: challenges, opportunities and recommendations. Environ Int 92–93:556–564. doi:10.1016/j.envint.2015.11.002
66. Whaley P, Letcher RJ, Covaci A, Alcock R (2016) Raising the standard of systematic reviews published in Environment International. Environ Int 97:274–276. doi:10.1016/j.envint.2016.08.007
67. Wikoff DS, Britt JK (2016) The Role of systematic review in the practice of toxicology and risk assessment—an appreciation for the primary tool in evidence-based practice. Toxicol open access 2:110
68. Woodruff TJ, Sutton P (2014) The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes. Environ Health Perspect 122(10):1007–1014. doi:10.1289/ehp.1307175