Issues with data and analyses: Errors, underlying themes, and potential solutions

Proceedings of the National Academy of Sciences of the United States of America

Volumn 115, Issue 11, 2018, Pages 2563-2570

  Brown, Andrew W ^a,b   Kaiser, Kathryn A ^a   Allison, David B ^a,b  

^a UNIVERSITY OF ALABAMA AT BIRMINGHAM   (United States)

^b INDIANA UNIVERSITY   (United States)

Author keywords

Data analysis; Quality control; Reproducibility; Rigor; Statistical errors

Indexed keywords

ANALYTICAL ERROR; ARTICLE; DATA ANALYSIS; INFORMATION PROCESSING; MATHEMATICAL ANALYSIS; MEASUREMENT ERROR; MEDICAL RESEARCH; PREVALENCE; PRIORITY JOURNAL; PROFESSIONAL KNOWLEDGE; PUBLICATION; PUBLISHING; SCIENTIFIC MISCONDUCT; SCIENTIST; STATISTICAL ANALYSIS; STUDY DESIGN; EXPERIMENTAL ERROR; HUMAN; METHODOLOGY; QUALITY CONTROL; REPRODUCIBILITY; SCIENCE; STANDARDS; STATISTICS; STATISTICS AND NUMERICAL DATA;

DATA COLLECTION; HUMANS; QUALITY CONTROL; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; SCIENCE; SCIENTIFIC EXPERIMENTAL ERROR; STATISTICS AS TOPIC;

EID: 85043781083     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1708279115     Document Type: Article

References (50)

1. Hamilton A (1961) The Papers of Alexander Hamilton (Columbia Univ Press, New York).
2. Ioannidis JPA (2012) Why science is not necessarily self-correcting. Perspect Psychol Sci 7:645–654.
3. Salsburg D (2017) Errors, Blunders, and Lies (CRC, Boca Raton, FL).
4. Perkel J (2012) Should Linus Pauling’s erroneous 1953 model of DNA be retracted? Retraction Watch. Available at retractionwatch.com/2012/06/27/should-linus-paulings-erroneous-1953-model-of-dna-be-retracted/. Accessed September 21, 2017.
5. Burchfield JD (1990) Lord Kelvin and the Age of the Earth (Univ of Chicago Press, Chicago).
6. Galton F (1897) The average contribution of each several ancestor to the total heritage of the offspring. Proc R Soc Lond 61:401–413.
7. Livio M (2013) Brilliant Blunders: From Darwin to Einstein—Colossal Mistakes by Great Scientists that Changed our Understanding of Life and the Universe (Simon & Schuster, New York), 1st Simon & Schuster hardcover ed, p 341.
8. Secrist H (1925) An Introduction to Statistical Methods (Macmillan, New York).
9. Ellenberg J (2014) How Not to Be Wrong: The Power of Mathematical Thinking (Penguin, New York).
10. “Student” (1931) The Lanarkshire milk experiment. Biometrika 23:398–406.
11. Allison DB, Brown AW, George BJ, Kaiser KA (2016) Reproducibility: A tragedy of errors. Nature 530:27–29.
12. Carlisle JB (2017) Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals. Anaesthesia 72:944–952.
13. Ercan I, et al. (2017) Examination of published articles with respect to statistical errors in veterinary sciences. Acta Veterniaria 67:33–42.
14. Nuijten MB, Hartgerink CH, van Assen MA, Epskamp S, Wicherts JM (2016) The prevalence of statistical reporting errors in psychology (1985-2013). Behav Res Methods 48:1205–1226.
15. National Academies of Sciences Engineering, & Medicine (2017) Fostering Integrity in Research (The National Academies Press, Washington, DC).
16. Bouter LM, Tijdink J, Axelsen N, Martinson BC, ter Riet G (2016) Ranking major and minor research misbehaviors: Results from a survey among participants of four World Conferences on Research Integrity. Res Integr Peer Rev 1:17.
17. Firestein S (2016) Failure: Why Science Is So Successful (Oxford Univ Press, Oxford).
18. Allison DB, et al. (2015) Goals in nutrition science 2015-2020. Front Nutr 2:26.
19. Dhurandhar NV, et al.; Energy Balance Measurement Working Group (2015) Energy balance measurement: When something is not better than nothing. Int J Obes 39: 1109–1113.
20. Beaudoin R, Mayer J (1953) Food intakes of obese and non-obese women. J Am Diet Assoc 29:29–33.
21. Sebastiani P, et al. (2010) Genetic signatures of exceptional longevity in humans. Science, 10.1126/science.1190532, and retraction (2011) 333:404.
22. Hughes P, Marshall D, Reid Y, Parkes H, Gelber C (2007) The costs of using unauthenticated, over-passaged cell lines: How much more data do we need? Bio-techniques 43:575, 577–586, and erratum (2008) 44:47.
23. Verhulst B, Eaves L, Hatemi PK (2012) Correlation not causation: The relationship between personality traits and political ideologies. Am J Pol Sci 56:34–51, and erratum (2016) 60:E3–E4.
24. Bent S, Tiedt TN, Odden MC, Shlipak MG (2003) The relative safety of ephedra compared with other herbal products. Ann Intern Med 138:468–471, and correction (2003) 138:1012.
25. Gawande A (2010) The Checklist Manifesto: How to Get Things Right (Metropolitan Books, New York).
26. George BJ, Brown AW, Allison DB (2015) Errors in statistical analysis and questionable randomization lead to unreliable conclusions. J Paramed Sci 6:153–154.
27. Li P, et al. (2015) Concerning Sichieri R, Cunha DB: Obes facts 2014; 7:221–232. The assertion that controlling for baseline (pre-randomization) covariates in randomized controlled trials leads to bias is false. Obes Facts 8:127–129.
28. Anonymous (2015) Retraction statement: LA sprouts randomized controlled nutrition, cooking and gardening program reduces obesity and metabolic risk in Latino youth. Obesity (Silver Spring) 23:2522.
29. Zalewski BM, et al. (2015) Correction of data errors and reanalysis of “The effect of glucomannan on body weight in overweight or obese children and adults: A systematic review of randomized controlled trials”. Nutrition 31:1056–1057.
30. Gelman A, Stern H (2006) The difference between “significant” and “not significant” is not itself statistically significant. Am Stat 60:328–331.
31. George BJ, et al. (2016) Common scientific and statistical errors in obesity research. Obesity (Silver Spring) 24:781–790.
32. Serlie MJ, Ter Horst KW, Brown AW (2016) Addendum: Hypercaloric diets with high meal frequency, but not increased meal size, increase intrahepatic triglycerides: A randomized controlled trial. Hepatology 64:1814–1816.
33. Allison DB, Williams MS, Hand GA, Jakicic JM, Fontaine KR (2015) Conclusion of “Nordic walking for geriatric rehabilitation: A randomized pilot trial” is based on faulty statistical analysis and is inaccurate. Disabil Rehabil 37:1692–1693.
34. Brown AW, Bohan Brown MM, Allison DB (2013) Belief beyond the evidence: Using the proposed effect of breakfast on obesity to show 2 practices that distort scientific evidence. Am J Clin Nutr 98:1298–1308.
35. Cofield SS, Corona RV, Allison DB (2010) Use of causal language in observational studies of obesity and nutrition. Obes Facts 3:353–356.
36. Micha R, et al. (2017) Association between dietary factors and mortality from heart disease, stroke, and type 2 diabetes in the United States. JAMA 317:912–924.
37. Mozaffarian D (2017). “Our nation’s nutrition crisis: nearly 1,000 cardiovascular & diabetes deaths each day (!) due to poor diet. https://t.co/oeCZQRIJQA.” Available at https://twitter.com/Dmozaffarian/status/839227095565815808. Accessed March 7, 2017.
38. Trasande L, Lind PM, Lampa E, Lind L (2017) Dismissing manufactured uncertainties, limitations and competing interpretations about chemical exposures and diabetes. J Epidemiol Community Health 71:942.
39. American Council on Science and Health (1991) From Mice to Men: The Benefits and Limitations of Animal Testing in Predicting Human Cancer Risk (American Council on Science and Health, New York), 3rd Ed.
40. Louis-Sylvestre J, et al. (1989) Learned caloric adjustment of human intake. Appetite 12:95–103.
41. Bohan Brown MM, Brown AW, Allison DB (2015) Linear extrapolation results in erroneous overestimation of plausible stressor-related yearly weight changes. Biol Psychiatry 78:e10–e11.
42. Thomas DM, et al. (2013) Can a weight loss of one pound a week be achieved with a 3500-kcal deficit? Commentary on a commonly accepted rule. Int J Obes 37: 1611–1613.
43. Schwingshackl L, et al. (2015) Fruit and vegetable consumption and changes in anthropometric variables in adult populations: A systematic review and meta-analysis of prospective cohort studies. PLoS One 10:e0140846.
44. Gelman A, Loken E (2013) The garden of forking paths. Available at www.stat.columbia.edu/∼gelman/research/unpublished/p_hacking.pdf. Accessed July 5, 2017.
45. Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 22:1359–1366.
46. Gadbury GL, Allison DB (2012) Inappropriate fiddling with statistical analyses to obtain a desirable p-value: Tests to detect its presence in published literature. PLoS One 7:e46363.
47. Begley S (2012) In cancer science, many “discoveries” don’t hold up. Reuters Science News. Available at www.reuters.com/article/us-science-cancer-idUSBRE82R12P20120328. Accessed July 5, 2017.
48. Kerr NL (1998) HARKing: Hypothesizing after the results are known. Pers Soc Psychol Rev 2:196–217.
49. Pires AM, Branco JA (2010) A statistical model to explain the Mendel–Fisher controversy. Stat Sci 25:545–565.
50. Feyerabend P (1975) How to defend society against science. Radical Philos 2:5–14.