Perspective: NutriGrade: A scoring system to assess and judge the meta-evidence of randomized controlled trials and cohort studies in nutrition research

Advances in Nutrition

Volumn 7, Issue 6, 2016, Pages 994-1004

  Schwingshackl, Lukas ^a   Knüppel, Sven ^a   Schwedhelm, Carolina ^a   Hoffmann, Georg ^b   Missbach, Benjamin ^b   Stelmach Mardas, Marta ^a,c   Dietrich, Stefan ^a   Eichelmann, Fabian ^a   Kontopanteils, Evangelos ^d   Iqbal, Khalid ^a   Aleksandrova, Krasimira ^a   Lorkowski, Stefan ^e,f   Leitzmann, Michael F ^g   Kroke, Anja ^h   Boeing, Heiner ^a  

^a GERMAN INSTITUTE OF HUMAN NUTRITION   (Germany)

^b UNIVERSITY OF VIENNA   (Austria)

^c POZNAN UNIVERSITY OF MEDICAL SCIENCES   (Poland)

^d UNIVERSITY OF MANCHESTER   (United Kingdom)

^e FRIEDRICH SCHILLER UNIVERSITY JENA   (Germany)

^f Competence Cluster of Nutrition and Cardiovascular Health ^*  (Germany)

^g UNIVERSITY OF REGENSBURG   (Germany)

^h UNIVERSITY OF APPLIED SCIENCES   (Germany)

Author keywords

Meta analysis; Meta evidence; Nutrition; Reliability; Scoring

Indexed keywords

COHORT ANALYSIS; EVALUATION STUDY; HUMAN; MEDICAL RESEARCH; META ANALYSIS (TOPIC); METHODOLOGY; NUTRITIONAL SCIENCE; RANDOMIZED CONTROLLED TRIAL (TOPIC); REPRODUCIBILITY; STANDARDS;

BIOMEDICAL RESEARCH; COHORT STUDIES; EVALUATION STUDIES AS TOPIC; HUMANS; META-ANALYSIS AS TOPIC; NUTRITIONAL SCIENCES; RANDOMIZED CONTROLLED TRIALS AS TOPIC; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 84997142690     PISSN: 21618313     EISSN: 21565376     Source Type: Journal    
DOI: 10.3945/an.116.013052     Document Type: Article

References (70)

1. Mann JI. Evidence-based nutrition: does it differ from evidence-based medicine? Ann Med 2010;42:475-86.
2. Higgins JPT, Green S (editors) [Internet]. Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011 [cited 2015 Dec 8]. Available from: www.cochrane-handbook.org.
3. Oxford Centre for Evidence-Based Medicine [Internet]. Levels of evidence [cited 2015 Nov 5]. Available from: http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/.
4. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, Guyatt GH, Harbour RT, Haugh MC, Henry D, et al. Grading quality of evidence and strength of recommendations. BMJ 2004;328:1490.
5. de Souza RJ, Mente A, Maroleanu A, Cozma AI, Ha V, Kishibe T, Uleryk E, Budylowski P, Schunemann H, Beyene J, et al. Intake of saturated and trans unsaturated fatty acids and risk of all-cause mortality, cardiovascular disease, and type 2 diabetes: systematic review and meta-analysis of observational studies. BMJ 2015;351:h3978.
6. Hooper L, Martin N, Abdelhamid A, Davey Smith G. Reduction in saturated fat intake for cardiovascular disease. Cochrane Database Syst Rev 2015;6:CD011737.
7. Satija A, Yu E, Willett WC, Hu FB. Understanding nutritional epidemiology and its role in policy. Adv Nutr 2015;6:5-18.
8. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, et al. GRADE guidelines:3. Rating the quality of evidence. J Clin Epidemiol 2011;64:401-6.
9. StroupDF, Berlin JA, Morton SC, Olkin I, WilliamsonGD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008-12.
10. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med 2009;151:W65-94.
11. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009;151(4):W65-94.
12. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and metaanalysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1.
13. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007;7:10.
14. Whiting P, Savovic J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016;69:225-34.
15. Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol 2007;36:666-76.
16. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 1995;16:62-73.
17. Schwingshackl L, Missbach B, Dias S, Konig J, Hoffmann G. Impact of different training modalities on glycaemic control and blood lipids in patients with type 2 diabetes: a systematic review and network metaanalysis. Diabetologia 2014;57:1789-97.
18. Bjelakovic G, Nikolova D, Gluud LL, Simonetti RG, Gluud C. Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. Cochrane Database Syst Rev 2012;3:CD007176.
19. Savović J, Jones HE, Altman DG, Harris RJ, Juni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 2012;157:429-38.
20. Higgins JP, AltmanDG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.
21. Higgins JPT, Thompson SG. Quantifying heterogeneity in a metaanalysis. Stat Med 2002;21:1539-58.
22. Mittlböck M, Heinzl H. A simulation study comparing properties of heterogeneity measures in meta-analyses. Stat Med 2006;25:4321-33.
23. Thompson SG, Turner RM, Warn DE. Multilevel models for metaanalysis, and their application to absolute risk differences. Stat Methods Med Res 2001;10:375-92.
24. Gøtzsche PC, Hrobjartsson A, Maric K, Tendal B. Data extraction errors in meta-analyses that use standardized mean differences. JAMA 2007;298:430-7.
25. GuyattGH, OxmanAD, Kunz R, Woodcock J, Brozek J, HelfandM, Alonso-Coello P, Falck-Ytter Y, JaeschkeR, Vist G, et al.GRADE guidelines: 8. Rating the quality of evidence-indirectness. J Clin Epidemiol 2011;64:1303-10.
26. Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. Publication and related biases. Health Technol Assess 2000;4:1-115.
27. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2007; (2):MR000005.
28. Sterne JA, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol 2001;54:1046-55.
29. Harbord RM, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med 2006;25:3443-57.
30. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA 2006;295:676-80.
31. Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315:629-34.
32. Lau J, Ioannidis JPA, Terrin N, Schmid CH, Olkin I. The case of the misleading funnel plot. BMJ 2006;333:597-600.
33. Terrin N, Schmid CH, Lau J. In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J Clin Epidemiol 2005;58:894-901.
34. Nestle M. Food company sponsorship of nutrition research and professional activities: a conflict of interest? Public Health Nutr 2001;4:1015-22.
35. Rowe S, Alexander N, Clydesdale FM, Applebaum RS, Atkinson S, Black RM, Dwyer JT, Hentges E, Higley NA, Lefevre M, et al. Funding food science and nutrition research: financial conflicts and scientific integrity. Am J Clin Nutr 2009;89:1285-91.
36. Lesser LI, Ebbeling CB, Goozner M, Wypij D, Ludwig DS. Relationship between funding source and conclusion among nutrition-related scientific articles. PLoS Med 2007;4:e5.
37. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ 1996;312:71-2.
38. Hernán MA, Hernandez-Diaz S, Robins JM. Randomized trials analyzed as observational studies. Ann Intern Med 2013;159:560-2.
39. Kroke A, Boeing H, Rossnagel K, Willich SN. History of the concept of 'levels of evidence' and their current status in relation to primary prevention through lifestyle interventions. Public Health Nutr 2004;7:279-84.
40. Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello P, Atkins D, Kunz R, Brozek J, Montori V, et al. GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol 2011;64:1311-6.
41. Hill AB. The environment and disease: association or causation? 1965. J R Soc Med 2015;108:32-7.
42. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74.
43. Fleiss JL. The design and analysis of clinical experiments. New York: John Wiley & Sons Inc.; 1986.
44. R-package [Internet]. [cited 2016 Feb 1]. Available from: https://cran.rproject. org/web/packages/ICC/index.html; https://cran.r-project.org/web/packages/irr/irr.pdf.
45. Mente A, de Koning L, Shannon HS, Anand SS. A systematic review of the evidence supporting a causal link between dietary factors and coronary heart disease. Arch Intern Med 2009;169:659-69.
46. Manheimer EW, van Zuuren EJ, Fedorowicz Z, Pijl H. Paleolithic nutrition for metabolic syndrome: systematic review and meta-analysis. Am J Clin Nutr 2015;102:922-32.
47. Santesso N, Akl EA, Bianchi M, Mente A, Mustafa R, Heels-Ansdell D, Schunemann HJ. Effects of higher-versus lower-protein diets on health outcomes: a systematic review and meta-analysis. Eur J Clin Nutr 2012;66:780-8.
48. Schwingshackl L, Hoffmann G. Comparison of the long-term effects of high-fat v. low-fat diet consumption on cardiometabolic risk factors in subjects with abnormal glucose metabolism: a systematic review and meta-analysis. Br J Nutr 2014;111:2047-58.
49. Naude CE, Schoonees A, Senekal M, Young T, Garner P, Volmink J. Low carbohydrate versus isoenergetic balanced diets for reducing weight and cardiovascular risk: a systematic review and meta-analysis. PLoS One 2014;9:e100652.
50. Zhou Q, Luo ML, Li H, Li M, Zhou JG. Coffee consumption and risk of endometrial cancer: a dose-response meta-analysis of prospective cohort studies. Sci Rep 2015;5:13410.
51. Aburto NJ, Ziolkovska A, Hooper L, Elliott P, Cappuccio FP, Meerpohl JJ. Effect of lower sodium intake on health: systematic review and metaanalyses. BMJ 2013;346:f1326.
52. Te Morenga L, Mallard S, Mann J. Dietary sugars and body weight: systematic review and meta-analyses of randomised controlled trials and cohort studies. BMJ 2013;346:e7492.
53. Aburto NJ, Hanson S, Gutierrez H, Hooper L, Elliott P, Cappuccio FP. Effect of increased potassium intake on cardiovascular risk factors and disease: systematic review and meta-analyses. BMJ 2013;346:f1378.
54. Lucas M. Conflicts of interest in nutritional sciences: The forgotten bias in meta-analysis. World J Methodol 2015;5:175-8.
55. Maki KC, Slavin JL, Rains TM, Kris-Etherton PM. Limitations of observational evidence: implications for evidence-based dietary recommendations. Adv Nutr 2014;5:7-15.
56. Ankarfeldt MZ. Comment on "Limitations of observational evidence: implications for evidence-based dietary recommendations". Adv Nutr 2014;5:293.
57. Dreyer NA, Tunis SR, Berger M, Ollendorf D, Mattox P, Gliklich R. Why observational studies should be among the tools used in comparative effectiveness research. Health Aff (Millwood) 2010;29:1818-25.
58. Pollock A, Farmer SE, Brady MC, Langhorne P, Mead GE, Mehrholz J, van Wijck F, Wiffen PJ. An algorithm was developed to assign GRADE levels of evidence to comparisons within systematic reviews. J Clin Epidemiol 2016;70:106-10.
59. Berkman ND, Lohr KN, Morgan LC, Kuo TM, Morton SC. Interrater reliability of grading strength of evidence varies with the complexity of the evidence in systematic reviews. J Clin Epidemiol 2013;66(10):1105-1117e1.
60. Schwingshackl L, Missbach B, Hoffmann G. An umbrella review of garlic intake and risk of cardiovascular disease. Phytomedicine 2015 Nov 14 (Epub ahead of print; DOI: 10.1016/j.phymed.2015.10.015).
61. Schwingshackl L, Hoffmann G. Monounsaturated fatty acids and risk of cardiovascular disease: synopsis of the evidence available from systematic reviews and meta-analyses. Nutrients 2012;4:1989-2007.
62. Mozaffarian D. Dietary and Policy Priorities for Cardiovascular Disease, Diabetes, and Obesity: A Comprehensive Review. Circulation 2016;133:187-225.
63. Murad MH, Mustafa R, Morgan R, Sultan S, Falck-Ytter Y, Dahm P. Rating the quality of evidence is by necessity a matter of judgment. J Clin Epidemiol 2016 Nov 14 (Epub ahead of print; DOI: 10.1016/j. phymed.2015.10.015).
64. The 2015 Dietary Guidelines Advisory Committee [Internet]. In Scientific report of the 2015 Dietary Guidelines Advisory Committee [cited 2016 Mar 2]. Available from: https://health.gov/dietaryguidelines/2015-scientific-report/PDFs/Scientific-Report-of-the-2015-Dietary-Guidelines-Advisory-Committee.pdf.
65. Huedo-Medina TB, Garcia M, Bihuniak JD, Kenny A, Kerstetter J. Methodologic quality of meta-analyses and systematic reviews on the Mediterranean diet and cardiovascular disease outcomes: a review. Am J Clin Nutr 2016;103:841-50.
66. Hartling L, Milne A, HammMP, Vandermeer B, Ansari M, Tsertsvadze A, Dryden DM. Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers. J Clin Epidemiol 2013;66:982-93.
67. Hartling L, Hamm MP, Milne A, Vandermeer B, Santaguida PL, AnsariM, Tsertsvadze A, Hempel S, Shekelle P, Dryden DM. Testing the risk of bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs. J Clin Epidemiol 2013;66:973-81.
68. Armijo-Olivo S, Ospina M, da Costa BR, Egger M, Saltaji H, Fuentes J, Ha C, Cummings GG. Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane risk of bias tool in physical therapy trials. PLoS One 2014;9:e96920.
69. Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol 2010;25:603-5.
70. Weed DL. Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related. Int J Epidemiol 2000;29: 387-90.