Perspective: Limiting dependence on nonrandomized studies and improving randomized trials in human nutrition research: Why and how

Advances in Nutrition

Volumn 9, Issue 4, 2018, Pages 367-377

  Trepanowski, John F ^a   Ioannidis, John P A ^a,b  

^a STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Epidemiology; Nutritional sciences; Observational study; Randomized controlled trial; Research design

Indexed keywords

ADULT; ARTICLE; CLINICAL OUTCOME; CONTROLLED STUDY; DIETARY COMPLIANCE; EFFECT SIZE; FEMALE; FOLLOW UP; HUMAN; MALE; MEASUREMENT ERROR; MENDELIAN RANDOMIZATION ANALYSIS; NUTRIENT; NUTRITIONAL SCIENCE; OBSERVATIONAL STUDY; OUTCOME ASSESSMENT; RANDOMIZED CONTROLLED TRIAL; RISK FACTOR; SAMPLE SIZE; DIET; DIET THERAPY; MEDICAL RECORD; META ANALYSIS (TOPIC); METHODOLOGY; NUTRITION; RANDOMIZED CONTROLLED TRIAL (TOPIC); REGISTER; STATISTICAL BIAS;

BIAS; DIET; DIET RECORDS; HUMANS; META-ANALYSIS AS TOPIC; NUTRITION THERAPY; NUTRITIONAL PHYSIOLOGICAL PHENOMENA; OBSERVATIONAL STUDIES AS TOPIC; RANDOMIZED CONTROLLED TRIALS AS TOPIC; REGISTRIES; RESEARCH DESIGN;

EID: 85053232819     PISSN: 21618313     EISSN: 21565376     Source Type: Journal    
DOI: 10.1093/ADVANCES/NMY014     Document Type: Article

References (100)

1. Trinquart L, Johns DM, Galea S. Why do we think we know what we know? A metaknowledge analysis of the salt controversy. Int J Epidemiol 2016;45(1):251-60.
2. Kanter MM, Kris-Etherton PM, Fernandez ML, Vickers KC, Katz DL. Exploring the factors that affect blood cholesterol and heart disease risk: is dietary cholesterol as bad for you as history leads us to believe? Adv Nutr 2012;3(5):711-7.
3. Benziger CP, Roth GA, Moran AE. The Global Burden of Disease Study and the preventable burden of NCD. Glob Heart 2016;11(4):393-7.
4. Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA 2005;294(2):218-28.
5. Young SS, Karr A. Deming, data and observational studies. Significance 2011;8(3):116-20.
6. Mente A, de Koning L, Shannon HS, Anand SS. A systematic review of the evidence supporting a causal link between dietary factors and coronary heart disease. Arch Intern Med 2009;169(7):659-69.
7. Miller PE, Perez V. Low-calorie sweeteners and body weight and composition: a meta-analysis of randomized controlled trials and prospective cohort studies. Am J Clin Nutr 2014;100(3):765-77.
8. US Department of Health and Human Services; USDA. 2015-2020 Dietary guidelines for Americans. 8th ed. December 2015. Available from: https://health.gov/dietaryguidelines/2015/guidelines/, Last accessed April 2, 2018.
9. Siontis GC, Ioannidis JP. Risk factors and interventions with statistically significant tiny effects. Int J Epidemiol 2011;40(5):1292-307.
10. Ioannidis JP, Loy EY, Poulton R, Chia KS. Researching genetic versus nongenetic determinants of disease: a comparison and proposed unification. Sci Transl Med 2009;1(7):7ps8.
11. Patel CJ, Cullen MR, Ioannidis JP, Butte AJ. Systematic evaluation of environmental factors: persistent pollutants and nutrients correlated with serum lipid levels. Int J Epidemiol 2012;41(3):828-43.
12. Archer E, Pavela G, Lavie CJ. The inadmissibility of What We Eat in America and NHANES dietary data in nutrition and obesity research and the scientific formulation of national dietary guidelines.MayoClin Proc 2015;90(7):911-26.
13. Patel CJ, Ioannidis JP. Placing epidemiological results in the context of multiplicity and typical correlations of exposures. J Epidemiol Community Health 2014;68(11):1096-100.
14. Ioannidis JP. We need more randomized trials in nutrition-preferably large, long-term, and with negative results. Am J Clin Nutr 2016;103(6):1385-6.
15. Dal-Ré R, Bracken MB, Ioannidis JP. Call to improve transparency of trials of non-regulated interventions. BMJ 2015;350:h1323.
16. Ioannidis JP. Implausible results in human nutrition research. BMJ 2013;347:f6698.
17. Schoenfeld JD, Ioannidis JP. Is everything we eat associated with cancer? A systematic cookbook review. Am J Clin Nutr 2013;97(1):127-34.
18. Ioannidis JP, Siontis GC. Re: Fruit and vegetable intake and overall cancer risk in the European Prospective Investigation into Cancer and Nutrition. J Natl Cancer Inst 2011;103(3):279-80.
19. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ. Rating quality of evidence and strength of recommendations: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336(7650):924.
20. Khoury MJ, Ioannidis JP. Big data meets public health. Science 2014;346(6213):1054-5.
21. Bouvard V, Loomis D, Guyton KZ, Grosse Y, El Ghissassi F, Benbrahim-Tallaa L, Guha N, Mattock H, Straif K. Carcinogenicity of consumption of red and processed meat. Lancet Oncol 2015;16(16):1599-600.
22. Chan DS, Lau R, Aune D, Vieira R, Greenwood DC, Kampman E, Norat T. Red and processed meat and colorectal cancer incidence: meta-analysis of prospective studies. PLoS One 2011;6(6):e20456.
23. Ioannidis JP, Polyzos NP, Trikalinos TA. Selective discussion and transparency in microarray research findings for cancer outcomes. Eur J Cancer 2007;43(13):1999-2010.
24. Patel CJ, Burford B, Ioannidis JP.Assessment of vibration of effects due tomodel specification can demonstrate the instability of observational associations. J Clin Epidemiol 2015;68(9):1046-58.
25. Brown AW, Ioannidis JP, CopeMB, BierDM, AllisonDB.Unscientific beliefs about scientific topics in nutrition. Adv Nutr 2014;5(5):563-5.
26. Chavalarias D, Wallach JD, Li AHT, Ioannidis JP. Evolution of reporting P values in the biomedical literature, 1990-2015. JAMA 2016;315(11):1141-8.
27. Johnson VE. Revised standards for statistical evidence. Proc Natl Acad Sci USA 2013;110(48):19313-7.
28. Belbasis L, Bellou V, Evangelou E, Ioannidis JP, Tzoulaki I. Environmental risk factors and multiple sclerosis: an umbrella review of systematic reviews and meta-analyses. Lancet Neurol 2015;14(3):263-73.
29. SchuemieMJ, HripcsakG, Ryan PB,MadiganD, SuchardMA. Robust empirical calibration of p-values using observational data. Stat Med 2016;35(22):3883-8.
30. Prasad V, Jena AB. Prespecified falsification end points: can they validate true observational associations? JAMA 2013;309(3):241-2.
31. Patel CJ, Ji J, Sundquist J, Ioannidis JP, Sundquist K. Systematic assessment of pharmaceutical prescriptions in association with cancer risk: a method to conduct a population-wide medication-wide longitudinal study. Sci Rep 2016;6:31308.
32. Fewell Z, SmithGD, Sterne JA. The impact of residual and unmeasured confounding in epidemiologic studies: a simulation study. Am J Epidemiol 2007;166(6):646-55.
33. Patel CJ, Manrai AK. Development of exposome correlation globes to map out environment-wide associations. Pac Symp Biocomput 2015;20:231-42.
34. Patel CJ, Ioannidis JP. Studying the elusive environment in large scale. JAMA 2014;311(21):2173-4.
35. Tatsioni A, Bonitsis NG, Ioannidis JP. Persistence of contradicted claims in the literature. JAMA 2007;298(21):2517-26.
36. Gorber SC, Schofield-Hurwitz S, Hardt J, Levasseur G, Tremblay M. The accuracy of self-reported smoking: a systematic review of the relationship between self-reported and cotinine-assessed smoking status. Nicotine Tob Res 2009;11(1):12-24.
37. Athersuch TJ. The role of metabolomics in characterizing the human exposome. Bioanalysis 2012;4(18):2207-12.
38. Mischak H, Allmaier G, Apweiler R, Attwood T, BaumannM, Benigni A, Bennett SE, Bischoff R, Bongcam-Rudloff E, Capasso G et al. Recommendations for biomarker identification and qualification in clinical proteomics. Sci Transl Med 2010;2(46):46ps2.
39. Naci H, Ioannidis JP. Evaluation of wellness determinants and interventions by citizen scientists. JAMA 2015;314(2):121-2.
40. Dietary Guidelines Advisory Committee. Scientific report of the 2015 Dietary Guidelines Advisory Committee. Washington (DC): USDA, US Department of Health and Human Services; 2015.
41. Archer E, Hand GA, Blair SN. Validity of US nutritional surveillance: National Health and Nutrition Examination Survey caloric energy intake data, 1971-2010. PLoS One 2013;8(10):e76632.
42. Bellach B, Kohlmeier L. Energy adjustment does not control for differential recall bias in nutritional epidemiology. J Clin Epidemiol 1998;51(5):393-8.
43. BernsteinDM, Loftus EF. The consequences of falsememories for food preferences and choices. Perspect Psychol Sci 2009;4(2):135-9.
44. Straube B. An overview of the neuro-cognitive processes involved in the encoding, consolidation, and retrieval of true and false memories. Behav Brain Funct 2012;8(1):35.
45. Johnson MK. Memory and reality. Am Psychol 2006;61(8):760-71.
46. Centers for Disease Control and Prevention. NAHNES ES MEC inperson dietary interviewers procedures manual. Atlanta, GA: CDC; 2008.
47. Dwan K, Gamble C, Williamson PR, Kirkham JJ. Systematic review of the empirical evidence of study publication bias and outcome reporting bias-an updated review. PLoS One 2013;8(7):e66844.
48. Ioannidis JP. The importance of potential studies that have not existed and registration of observational data sets. JAMA 2012;308(6): 575-6.
49. Dal-Ré R, Ioannidis JP, Bracken MB, Buffler PA, Chan A-W, Franco EL, La Vecchia C, Weiderpass E. Making prospective registration of observational research a reality. Sci Transl Med 2014; 6(224):224cm1.
50. Estruch R, Ros E, Salas-Salvadó J, Covas M-I, Corella D, Arós F, Gómez-Gracia E, Ruiz-Gutiérrez V, Fiol M, Lapetra J. Primary prevention of cardiovascular diseasewith aMediterranean diet.NEngl J Med 2013;368(14):1279-90.
51. Ioannidis JP, Lau J. The impact of high-risk patients on the results of clinical trials. J Clin Epidemiol 1997;50(10):1089-98.
52. Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med 1996;125(7):605-13.
53. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES, Munafò MR. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 2013;14(5):365-76.
54. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 2015;350:h2147.
55. Calvo G, McMurray JJ, Granger CB, Alonso-García á, Armstrong P, Flather M, Gómez-Outes A, Pocock S, Stockbridge N, Svensson A. Large streamlined trials in cardiovascular disease. Eur Heart J 2014;35(9):544-8.
56. Lauer MS, D'Agostino RB Sr. The randomized registry trial-the next disruptive technology in clinical research? N Engl J Med 2013;369(17):1579-81.
57. Lissner L, Majem LS, de Almeida MDV, Berg C, Hughes R, Cannon G, Thorsdottir I, Kearney J, Gustafsson J-° A, Rafter J. The Women's Health Initiative: what is on trial: nutrition and chronic disease? Or misinterpreted science, media havoc and the sound of silence from peers? Public Health Nutr 2006;9(02):269-72.
58. Van Spall HG, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 2007;297(11):1233-40.
59. Murthy VH, Krumholz HM, Gross CP. Participation in cancer clinical trials: race-, sex-, and age-based disparities. JAMA2004;291(22):2720-6.
60. RosenthalR,RosnowRL. The volunteer subject. Artifacts in behavioral research, Oxford University Press, Oxford, 2009:48-92.
61. Sainani K.Misleading comparisons: the fallacy of comparing statistical significance. PM R 2010;2(6):559-62.
62. Hernández AV, Boersma E, Murray GD, Habbema JDF, Steyerberg EW. Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading? Am Heart J 2006;151(2):257-64.
63. Sun X, Briel M, Busse JW, You JJ, Akl EA, Mejza F, Bala MM, Bassler D, Mertz D, Diaz-Granados N. Credibility of claims of subgroup effects in randomised controlled trials: systematic review. BMJ 2012; 344:e1553.
64. Relton C, Torgerson D, O'Cathain A, Nicholl J. Rethinking pragmatic randomised controlled trials: introducing the "cohort multiple randomised controlled trial" design. BMJ 2010;340:c1066.
65. Illner AK, Freisling H, Boeing H, Huybrechts I, Crispim SP, Slimani N. Review and evaluation of innovative technologies for measuring diet in nutritional epidemiology. Int J Epidemiol 2012;41(4): 1187-203.
66. Ioannidis JPA, Bossuyt PMM. Waste, leaks, and failures in the biomarker pipeline. Clin Chem 2017;63(5):963-72.
67. Walter SD, Turner R, Macaskill P, McCaffery KJ, Irwig L. Beyond the treatment effect: evaluating the effects of patient preferences in randomised trials. Stat Methods Med Res 2017;26(1):489-507.
68. Lei H, Nahum-Shani I, Lynch K, Oslin D, Murphy SA. A "SMART" design for building individualized treatment sequences. Annu Rev Clin Psychol 2012;8:21-48.
69. Vickers AJ, Scardino PT. The clinically-integrated randomized trial: proposed novel method for conducting large trials at low cost. Trials 2009;10(1):14.
70. LiG, Sajobi TT,Menon BK, Korngut L, LowerisonM, JamesM,Wilton SB, Williamson T, Gill S, Drogos LL. Registry-based randomized controlled trials-what are the advantages, challenges, and areas for future research? J Clin Epidemiol 2016;80:16-24.
71. James S, Rao SV, Granger CB. Registry-based randomized clinical trials-a new clinical trial paradigm. Nat Rev Cardiol 2015;12(5):312-6.
72. Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group. The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers. N Engl J Med 1994;330(15):1029-35.
73. Virtamo J, Pietinen P, Huttunen JK, Korhonen P, Malila N, Virtanen MJ, Albanes D, Taylor PR, Albert P; ATBC Study Group. Incidence of cancer and mortality following alpha-tocopherol and beta-carotene supplementation: a post-intervention follow-up. JAMA 2003;290(4):476-85.
74. Virtamo J, Taylor PR, Kontto J, Männistö S, Utriainen M, Weinstein SJ, Huttunen J, Albanes D. Effects of a-tocopherol and ß-carotene supplementation on cancer incidence and mortality: 18-year postintervention follow-up of the Alpha-tocopherol, Beta-carotene Cancer Prevention Study. Int J Cancer. 2014;135(1):178-85.
75. Jeon YJ, Myung SK, Lee EH, Kim Y, Chang YJ, Ju W, Cho HJ, Seo HG, Huh BY. Effects of beta-carotene supplements on cancer prevention:meta-analysis of randomized controlled trials.Nutr Cancer 2011;63(8):1196-207.
76. Bjelakovic G, Nikolova D, Gluud LL, Simonetti RG, Gluud C. Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. Cochrane Database Syst Rev 2012;3:CD007176.
77. Ye Y, Li J, Yuan Z. Effect of antioxidant vitamin supplementation on cardiovascular outcomes: a meta-analysis of randomized controlled trials. PLoS One 2013;8(2):e56803.
78. Bjelakovic G, Nikolova D, Gluud C. Meta-regression analyses, metaanalyses, and trial sequential analyses of the effects of supplementation with beta-carotene, vitamin A, and vitamin E singly or in different combinations on all-cause mortality: do we have evidence for lack of harm? PLoS One 2013;8(9):e74558.
79. Piantadosi S. Clinical trials: a methodological approach. Hoboken, NJ: Wiley; 1997.
80. Ioannidis JP, Adami H-O. Nested randomized trials in large cohorts and biobanks: studying the health effects of lifestyle factors. Epidemiology 2008;19(1):75-82.
81. Kriska AM, Bayles C, Cauley JA, LaPorte RE, Sandler RB, Pambianco G. A randomized exercise trial in older women: increased activity over two years and the factors associated with compliance. Med Sci Sports Exerc 1986;18(5):557-62.
82. Irwin ML, Tworoger SS, Yasui Y, Rajan B, McVarish L, LaCroix K, Ulrich CM, Bowen D, Schwartz RS, Potter JD. Influence of demographic, physiologic, and psychosocial variables on adherence to a yearlong moderate-intensity exercise trial in postmenopausal women. Prev Med 2004;39(6):1080-6.
83. Courneya KS, Segal RJ, Reid RD, Jones LW, Malone SC, Venner PM, Parliament MB, Scott CG, Quinney HA, Wells GA. Three independent factors predicted adherence in a randomized controlled trial of resistance exercise training among prostate cancer survivors. J Clin Epidemiol 2004;57(6):571-9.
84. Women's Health Initiative Study Group. Dietary adherence in the Women'sHealth InitiativeDietaryModification Trial. JAmDietAssoc 2004;104(4):654.
85. Punja S, Xu D, Schmid CH, Hartling L, Urichuk L, Nikles CJ, Vohra S. N-of-1 trials can be aggregated to generate group mean treatment effects: a systematic review and meta-analysis. J Clin Epidemiol 2016;76:65-75.
86. Punja S, Bukutu C, Shamseer L, Sampson M, Hartling L, Urichuk L, Vohra S. N-of-1 trials are a tapestry of heterogeneity. J Clin Epidemiol 2016;76:47-56.
87. Hall KD, Bemis T, Brychta R, Chen KY, Courville A, Crayner EJ, Goodwin S,Guo J,Howard L,KnuthND.Calorie for calorie, dietary fat restriction results in more body fat loss than carbohydrate restriction in people with obesity. Cell Metab 2015;22(3):427-36.
88. Hall KD, Chen KY, Guo J, LamYY, Leibel RL,Mayer LE, ReitmanML, Rosenbaum M, Smith SR, Walsh BT. Energy expenditure and body composition changes after an isocaloric ketogenic diet in overweight and obese men. Am J Clin Nutr 2016;104(2):324-33.
89. Smith GD, Ebrahim S. Mendelian randomization: prospects, potentials, and limitations. Int J Epidemiol 2004;33(1):30-42.
90. Smith GD, Timpson N, Ebrahim S. Strengthening causal inference in cardiovascular epidemiology throughMendelian randomization. Ann Med 2008;40(7):524-41.
91. Theodoratou E, Timofeeva M, Li X, Meng X, Ioannidis JPA. Nature, nurture, and cancer risks: genetic and nutritional contributions to cancer. Annu Rev Nutr 2017;37:293-320.
92. Li X, Meng X, Timofeeva M, Tzoulaki I, Tsilidis KK, Ioannidis JP, Campbell H, Theodoratou E. Serum uric acid levels and multiple health outcomes: umbrella review of evidence from observational studies, randomised controlled trials, and Mendelian randomisation studies. BMJ. 2017;357:j2376.
93. Maki KC, Slavin JL, Rains TM, Kris-Etherton PM. Limitations of observational evidence: implications for evidence-based dietary recommendations. Adv Nutr 2014;5:7-15.
94. Schwingshackl L, Knüppel S, Schwedhelm C, Hoffmann G, Missbach B, Stelmach-Mardas M, Dietrich S, Eichelmann F, Kontopantelis E, Iqbal K et al. NutriGrade: a scoring system to assess and judge the meta-evidence of randomized controlled trials and cohort studies in nutrition research [perspective]. Adv Nutr 2016;7(6):994-1004.
95. Yang C, Pinart M, Kolsteren P, Van Camp J, De Cock N, Nimptsch K, Pischon T, Laird E, Perozzi G, Canali R et al. Essential study quality descriptors for data from nutritional epidemiologic research [perspective]. Adv Nutr 2017;8(5): 639-51.
96. Magni P, Bier DM, Pecorelli S, Agostoni C, Astrup A, Brighenti F, Cook R, Folco E, Fontana L, Gibson RA et al. Improving nutritional guidelines for sustainable health policies: current status and perspectives [perspective]. Adv Nutr 2017;8(4):532-45.
97. Temple NJ. How reliable are randomised controlled trials for studying the relationship between diet and disease?Anarrative review. Br JNutr 2016; 116(3):381-9.
98. Ankarfeldt MZ. Comment on "Limitations of observational evidence: implications for evidence-based dietary recommendations". Adv Nutr 2014;5(3):293.
99. Munafò MR, Nosek BA, Bishop DV, Button KS, Chambers CD, du Sert NP, Simonsohn U, Wagenmakers E-J, Ware JJ, Ioannidis JP. A manifesto for reproducible science. Nat Hum Behav 2017; 1:0021.
100. Ebrahim S, Montoya L, el Din MK, Sohani ZN, Agarwal A, Bance S, Saquib J, Saquib N, Ioannidis JP. Randomized trials are frequently fragmented in multiple secondary publications. J Clin Epidemiol 2016;79:130-9.