Common scientific and statistical errors in obesity research

Obesity

Volumn 24, Issue 4, 2016, Pages 781-790

  George, Brandon J ^a   Beasley, T Mark ^a   Brown, Andrew W ^a   Dawson, John ^b   Dimova, Rositsa ^c   Divers, Jasmin ^d   Goldsby, Tashauna U ^a   Heo, Moonseong ^e   Kaiser, Kathryn A ^a   Keith, Scott W ^f   Kim, Mimi Y ^e   Li, Peng ^a   Mehta, Tapan ^a   Oakes, J Michael ^g   Skinner, Asheley ^h   Stuart, Elizabeth ⁱ   Allison, David B ^a  

^a UNIVERSITY OF ALABAMA AT BIRMINGHAM   (United States)

^b TEXAS TECH UNIVERSITY   (United States)

^c UNIVERSITY AT BUFFALO   (United States)

^d WAKE FOREST SCHOOL OF MEDICINE   (United States)

^e ALBERT EINSTEIN COLLEGE OF MEDICINE   (United States)

^f THOMAS JEFFERSON UNIVERSITY   (United States)

^g UNIVERSITY OF MINNESOTA   (United States)

^h IL   (United States)

ⁱ JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANALYTICAL ERROR; CLUSTER ANALYSIS; CONFIRMATION BIAS; EFFECT SIZE; HUMAN; INFORMATION PROCESSING; INTERPERSONAL COMMUNICATION; MEDICAL RESEARCH; METHODOLOGY; MISSING DATA; NULL HYPOTHESIS; OBESITY; QUALITY CONTROL; REVIEW; SCIENTIFIC ERROR; STATISTICAL ANALYSIS; STATISTICAL ERROR; STATISTICAL SIGNIFICANCE; STUDY DESIGN; STANDARDS; STATISTICAL BIAS;

BIAS (EPIDEMIOLOGY); BIOMEDICAL RESEARCH; DATA INTERPRETATION, STATISTICAL; HUMANS; OBESITY; RESEARCH DESIGN;

EID: 84962110833     PISSN: 19307381     EISSN: 1930739X     Source Type: Journal    
DOI: 10.1002/oby.21449     Document Type: Review

References (111)

1. Li P, Brown AW, Oakes JM, Allison DB., Comment on "Intervention Effects of a School-Based Health Promotion Programme on Obesity Related Behavioural Outcomes". J Obes 2015; 2015: 2.
2. Brown AW, Sievenpiper JL, Kyle TA, Kaiser KA., Communication of randomized controlled trial results must match the study focus. J Nutr 2015; 145: 1027-1029.
3. Brown AW, Li P, Bohan Brown MM, et al., Best (but often forgotten) practices: designing, analyzing, and reporting cluster randomized controlled trials. Am J Clin Nutr 2015; 102: 241-248.
4. Levitsky DA, Brown AW, Hansen BC, et al., An unjustified conclusion from self-report-based estimates of energy intake. Am J Med 2014; 127: e33.
5. Dhurandhar NV, Schoeller D, Brown AW, et al., Energy balance measurement: when something is not better than nothing. Int J Obes 2014; 39 (7): 1109-1113.
6. Casazza K, Brown AW, Astrup A, et al., Weighing the evidence of common beliefs in obesity research. Crit Rev Food Sci Nutr 2014. DOI: 10.1080/10408398.2014.922044.
7. Brown AW, Ioannidis JP, Cope MB, Bier DM, Allison DB., Unscientific beliefs about scientific topics in nutrition. Adv Nutr 2014; 5: 563-565.
8. Brown AW, Hall KD, Thomas D, Dhurandhar NV, Heymsfield SB, Allison DB., Order of magnitude misestimation of weight effects of children's meal policy proposals. Childhood Obes 2014; 10: 542-545.
9. Bohan Brown MM, Brown AW, Allison DB., Linear extrapolation results in erroneous overestimation of plausible stressor-related yearly weight changes. Biol Psychiatry 2015; 78 (4): e10-1.
10. Casazza K, Fontaine KR, Astrup A, et al., Myths, presumptions, and facts about obesity. N Engl J Med 2013; 368: 446-454.
11. Brown AW, Bohan Brown MM, Allison DB., Belief beyond the evidence: using the proposed effect of breakfast on obesity to show 2 practices that distort scientific evidence. Am J Clin Nutr 2013; 98: 1298-1308.
12. Brown AW, Allison DB., Unintended consequences of obesity-targeted health policy. Virtual Mentor 2013; 15: 339-346.
13. Schoeller D, Archer E, Dawson JA, Heymsfield S., Implausible results from the use of invalid methods. J Nutr 2015; 145: 150.
14. Li P, Brown AW, Oakes JM, Allison DB., School-based obesity prevention intervention in chilean children: effective in controlling, but not reducing obesity. J Obes 2015. DOI: 10.1155/2015/183528.
15. Lewis DW Jr, Fields DA, Allison DB., Inconsistencies and inaccuracies in reporting on choice of endpoints and of statistical results in RCT of maternal diet. Pediatr Obes 2015. DOI: 10.1111/ijpo.12030.
16. Brown AW, Sievenpiper JL, Kyle TA, Kaiser KA, Communication of Randomized Controlled Trial Results Must Match the Study Focus. Journal of Nutrition 2015;145:1027-1029.
17. O'Neill D, Sweetman O., The consequences of measurement error when estimating the impact of obesity on income. IZA J Labor Econ 2013. DOI: 10.1186/2193-0000-0000-0000.
18. Lyons R., The spread of evidence-poor medicine via flawed social-network analysis. Stat Politics Policy 2011. DOI: 10.2202/2151-0000.1024.
19. Wake M, Lycett K., Let's call it as it is: on results, reach, and resolution in population-based obesity trials. Pediatrics 2014; 134: e846-e847.
20. Veličkovic VM., Opinion: statistical misconceptions. Scientist. The Scientist 2013. Available from: http://www.the-scientist.com/?articles.view/articleNo/36781/title/Opinion - Statistical-Misconceptions/
21. Torres A., That big childhood obesity decline may have been a statistical error. National Review 2014. Available from: http://www.nationalreview.com/corner/373624/big-childhood-obesity-decline-may-have-been-statistical-error-alec-torres.
22. Johns DM., Disconnected? Slate 2011. Available from: http://www.slate.com/articles/health-and-science/science/2011/07/disconnected.html.
23. Lykken DT., Statistical significance in psychological research. Psychol Bull 1968; 70: 151-159.
24. Casella G, Berger RL., Statistical Inference, 2nd ed. Pacific Grove: Duxbury; 2002.
25. Cohen J., The earth is round (P < 0.05). Am Psychol 1994; 49: 997-1003.
26. Schervish MJ., P values: what they are and what they are not. Am Stat 1996; 50: 203-206.
27. Fisher RA., The Design of Experiments. London: Oliver and Boyd; 1935.
28. Blackwelder W., "Proving the null hypothesis" in clinical trials. Control Clin Trials 1982; 3: 345-353.
29. Koopman KE, Caan MW, Nederveen AJ, et al., Hypercaloric diets with increased meal frequency, but not meal size, increase intrahepatic triglycerides: a randomized controlled trial. Hepatology 2014; 60: 545-553.
30. Schoenfeld JD, Ioannidis JP., Is everything we eat associated with cancer? A systematic cookbook review. Am J Clin Nutr 2013; 97: 127-134.
31. Cohen J., The concepts of power analysis. In:, Cohen J, editor. Statistical Power Analysis for the Behavioral Sciences. Hillsdale: Academic Press, Inc.; 1998. pp 1-17.
32. Cassani RS, Fassini PG, Silvah JH, Lima CM, Marchini JS., Impact of weight loss diet associated with flaxseed on inflammatory markers in men with cardiovascular risk factors: a clinical study. Nutr J 2015. DOI: 10.1186/1475-0000-0000-0000.
33. Dimova RB, Allison DB., Inappropriate statistical method in a parallel-group randomized controlled trial results in unsubstantiated conclusions. Nutr J, in press.
34. Boutron I, Dutton S, Ravaud P, Altman DG., Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 2010; 303: 2058-2064.
35. Bland JM, Altman DG., Comparisons against baseline within randomized groups are often used and can be highly misleading. Trials 2011; 12: 1-7.
36. Allison DB, Gorman BS, Primavera LH., The most common questions asked of statistical consultants: our favorite responses and recommended readings. Genet Soc Gen Psychol Monogr 1993; 119: 153-185.
37. Huck SW, McLean RA., Using a repeated measures ANOVA to analyze the data from a pretest-posttest design: a potentially confusing task. Psychol Bull 1975; 82: 511-518.
38. Myers JL, Well AD., Research Design and Statistical Analysis, 1st ed. New York: HarperCollins; 1991.
39. Allison DB., When is it worth measuring a covariate in a randomized clinical trial? J Consult Clin Psychol 1995; 63: 339-343.
40. Albert PS., Tutorial in biostatistics: longitudinal data analysis (repeated measures) in clinical trials. Stat Med 1999; 18: 1707-1732.
41. Kirk RE., Experimental Design: Procedures for the Behavioral Sciences, 2nd ed. Pacific Grove: Brooks/Cole; 1982.
42. Liu S, Rovine MJ, Molenaar PC., Selecting a linear mixed model for longitudinal data: repeated measures analysis of variance, covariance pattern model, and growth curve approaches. Psychol Methods 2012; 17: 15-30.
43. Shaffer JP., Multiple hypothesis testing. Ann Rev Psychol 1995; 46: 561-584.
44. Mehta TS, Zakharkin SO, Gadbury GL, Allison DB., Epistemological issues in omics and high-dimensional biology: give the people what they want. Physiol Genom 2006; 28: 24-32.
45. Mudge JF, Baker LF, Edge CB, Houlahan JE., Setting an optimal α that minimizes errors in null hypothesis significance tests. PLoS One 2012; 7:e32734. DOI: 10.1371/journal.pone.0032734.
46. Bland JPL, Chen L, Cui X, et al., An adaptive alpha spending algorithm improves the power of statistical inference in microarray data analysis. Bioinformation 2007; 1: 384-389.
47. Elliott TR., Registering randomized clinical trials and the case for CONSORT. Exp Clin Psychopharmacol 2007; 15: 511-518.
48. Schulz KF, Altman DG, Moher D., CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. PLoS Med 2010; 7: e1000251.
49. von Elm E, Altman DG, Egger M, et al., The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Bull World Health Organ 2007; 85: 867-872.
50. Williams RJ, Tse T, Harlan WR, Zarin DA., Registration of observational studies: is it time? CMAJ 2010; 182: 1638-1642.
51. Mills JL., Data torturing. N Engl J Med 1993; 329: 1196-1199.
52. Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD., The extent and consequences of p-hacking in science. PLoS One 2015; 13:e1002106. DOI: 10.1371/journal.pbio.1002106.
53. Motulsky HJ., Common misconceptions about data analysis and statistics. Pharmacol Res Perspect 2015; 3:e00093. DOI: 10.1002/prp2.93.
54. Muller K, Fetterman B., Regression and ANOVA an Integrated Approach Using SAS Software. Cary: SAS Institute; 2002.
55. Ivanescu AE, Li P, George B, et al., The importance of prediction model validation and assessment in obesity and nutrition research. Int J Obesity 2015. DOI: 10.1038/ijo.2015.214.
56. Campbell MJ, Donner A, Klar N., Developments in cluster randomized trials and statistics in medicine. Stat Med 2007; 26: 2-19.
57. Hannan PJ., Experimental social epidemiology: controlled community trials. In:, Oakes JM, Kaufman JS, editors. Methods in Social Epidemiology. San Francisco: Jossey-Bass/Wiley; 2006. pp 335-364.
58. Eldridge SM, Ukoumunne OC, Carlin JB., The intra-cluster correlation coefficient in cluster randomized trials: a review of definitions. Int Stat Rev 2009; 77: 378-394.
59. Feng ZF, Diehr P, Peterson AV, McLerran D., Selected statistical issues in group randomized trials. Annu Rev Public Health 2001; 22: 167-187.
60. Koziol JA, Reid N., On the asymptotic equivalence of two ranking methods for K-sample linear rank statistics. Ann Stat 1977; 5: 1099-1106.
61. Agresti A, Pendergast J., Comparing mean ranks for repeated measures data. Commun Stat TheoryMethod 1986; 15: 1417-1433.
62. Beasley TM., Multivariate aligned rank test for interactions in multiple group repeated measures designs. Multivariate Behav Res 2002; 37: 197-226.
63. Datta S, Satten GA., Rank-sum tests for clustered data. J Am Stat Assoc 2005; 100: 908-915.
64. Lehmann EL., Nonparametrics: Statistical Methods Based on Ranks. New York: Springer; 1975.
65. Cliff N., Dominance statistics: ordinal analyses to answer ordinal questions. Psychol Bull 1993; 114: 494-509.
66. Vargha A, Delaney HD., The Kruskal-Wallis test and stochastic homogeneity. J Educ Behav Stat 1998; 23: 170-192.
67. Marascuilo LA, McSweeney M., Nonparametric and Distribution-Free Methods for the Social Sciences. Monterey: Brooks-Cole; 1977.
68. Zimmerman DW., A note on homogeneity of variance of scores and ranks. J Exp Educ 1996; 64: 351-362.
69. Zimmerman DW, Zumbo BD., Parametric alternatives to the student t test under violation of normality and homogeneity of variance. Percept Mot Skills 1992; 74: 835-844.
70. Sawilowsky S, Blair RC., A more realistic look at the robustness and type II error properties of the t-test to departures from population normality. Psychol Bull 1992; 111: 353-360.
71. National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials, Panel on Handling Missing Data in Clinical Trials, Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academies Press; 2010.
72. Greenland S, Finkle WD., A critical look at methods for handling missing covariates in epidemiologic regression analyses. Am J Epidemiol 1995; 142: 1255-1264.
73. Rubin DB., Multiple Imputation for Nonresponse in Surveys. New York: Wiley; 1987.
74. White IR, Royston P, Wood AM., Multiple imputation using chained equations: issues and guidance for practice. Stat Med 2011; 30: 377-399.
75. Carpenter JR, Kenward MG., Multiple Imputation and its Application. Hoboken: Wiley; 2013.
76. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR., Converting among effect sizes. Introduction to Meta-Analysis. West Sussex: Wiley; 2009, pp 45-49.
77. Gorman BS, Primavera LH, Allison DB., POWPAL: a program for estimating effect sizes, statistical power, and sample sizes. Educ Psychol Measure 1995; 55: 773-776.
78. Bonett DG., Transforming odds ratios into correlations for meta-analytic research. Am Psychol 2007; 62: 254-255.
79. Gøtzsche PC, Hróbjartsson A, Maric K, Tendal B., Data extraction errors in meta-analyses that use standardized mean differences. JAMA 2007; 298: 430-437.
80. Kaiser KA, Brown AW, Allison DB., Comment on PMID 25168465: Systematic Review and Meta-Analysis of the Effect of Increased Vegetable and Fruit Consumption on Body Weight and Energy Intake. Bethesda (MD): National Library of Medicine; 2015.
81. Durant N, Baskin ML, Thomas O, Allison DB., School-based obesity treatment and prevention programs: all in all, just another brick in the wall? Int J Obesity 2008; 32: 1747-1751.
82. Pigott TD., Handling missing data. In:, Cooper H HL, Valentine JC, editors. The Handbook of Research Synthesis and Meta-Analysis. New York: Russell Sage Foundation; 2009. pp 399-416.
83. Allison DB, Faith MS., Hypnosis as an adjunct to cognitive-behavioral psychotherapy for obesity: a meta-analytic reappraisal. J Consult Clin Psychol 1996; 64: 513-516.
84. Perry P., Realities of the effect size calculation process: considerations for beginning meta-analysts. In:, Bukoski W, editor. Meta-analysis of Drug Abuse Prevention Programs. Derby: Diane Publishing; 1997. pp 120-129.
85. Faraone SV., Interpreting estimates of treatment effects: implications for managed care. Pharm Ther 2008; 33: 700-711.
86. Feingold A., Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychol Methods 2009; 14: 43-53.
87. Morris SB, DeShon RP., Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychol Methods 2002; 7: 105-125.
88. Hedges LV., Effect sizes in cluster-randomized designs. J Educ Behav Stat 2007; 32: 341-370.
89. Allison DB, Keating K, Kaiser K, Shikany J., Re: dietary sugars and body weight: systematic review and meta-analyses of randomised controlled trials and cohort studies. BMJ 2013; 346:e7492. DOI: http://dx.doi.org/10.1136/bmj.e7492.
90. Galton F., Regression towards mediocrity in hereditary stature. J Anthropol Inst Great Britain Ireland 1886;15: 246-263.
91. Barnett AG, van der Pols JC, Dobson AJ., Regression to the mean: what it is and how to deal with it. Int J Epidemiol 2005; 34: 215-220.
92. Chobanian AV, Bakris GL, Black HR, et al., Seventh report of the joint national committee on prevention, detection, evaluation, and treatment of high blood pressure. Hypertension 2003; 42: 1206-1252.
93. Senn S., Francis Galton and regression to the mean. Significance 2011; 8: 124-126.
94. Cadzow RB, Chambers MK, Sandell AM., School-based obesity intervention associated with three year decrease in student weight status in a low-income school district. J Community Health 2015; 40: 709-713.
95. Sandercock GR, Cohen DD, Griffin M., Evaluation of a multicomponent intervention to improve weight status and fitness in children: upstarts. Pediatr Int 2012; 54: 911-917.
96. King MH, Lederer AM, Sovinski D, et al., Implementation and evaluation of the HEROES initiative a tri-state coordinated school health program to reduce childhood obesity. Health Promot Pract 2014; 15: 395-405.
97. Kim N, Seo DC, King MH, Lederer AM, Sovinski D., Long-term predictors of blood pressure among adolescents during an 18-month school-based obesity prevention intervention. J Adolesc Health 2014; 55: 521-527.
98. Castro DC, Samuels M, Harman AE., Growing healthy kids: a community garden-based obesity prevention program. Am J Prev Med 2013; 44: S193-S199.
99. Skinner AC, Heymsfield SB, Pietrobelli A, Faith MS, Allison DB., Ignoring regression to the mean leads to unsupported conclusion about obesity. Int J Behav Nutr Phys Activity 2015; 12: 56.
100. Allison DB, Loebel AD, Lombardo I, Romano SJ, Siu CO., Understanding the relationship between baseline BMI and subsequent weight change in antipsychotic trials: effect modification or regression to the mean? Psychiatry Res 2009; 170: 172-176.
101. Yudkin P, Stratton I., How to deal with regression to the mean in intervention studies. Lancet 1996; 347: 241-243.
102. Cook TD, Campbell DT, Day A., Quasi-Experimentation: Design & Analysis Issues for Field Settings. Boston: Houghton Mifflin; 1979.
103. Smith R., David Sackett Physician, trialist, and teacher. BMJ 2015; 350:h2639. DOI: 10.1136/bmj.h2639.
104. Sackett DL., Bias in analytic research. J Chronic Dis 1979; 32: 51-63.
105. Collins FS, Tabak LA., NIH plans to enhance reproducibility. Nature 2014; 505: 612-613.
106. Popper K., The Logic of Scientific Discovery. New York: Routledge; 2002.
107. Simmons JP, Nelson LD, Simonsohn U., False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 2011; 22: 1359-1366.
108. Gelman A., "False-positive psychology". Available from: http://andrewgelman.com/2012/02/16/false-positive-psychology/.
109. Gelman A., Too Good to Be True. Slate 2013. Available from: http://www.slate.com/articles/health-and-science/science/2013/07/statistics-and-psychology-multiple-comparisons-give-spurious-results.html
110. Hayes AF., Beyond Baron and Kenny: statistical mediation analysis in the new millennium. Commun Monogr 2009; 76: 408-420.
111. Moher D, Liberati A, Tetzlaff J, Altman DG., The Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6: e1000097.