Best (but oft-forgotten) practices: Sensitivity analyses in randomized controlled trials

American Journal of Clinical Nutrition

Volumn 103, Issue 1, 2016, Pages 5-17

  De Souza, Russell J ^a,b,c   Eisen, Rebecca B ^a   Perera, Stefan ^a   Bantoto, Bianca ^a   Bawor, Monica ^a   Dennis, Brittany B ^a   Samaan, Zainab ^a,d,h   Thabane, Lehana ^a,e,f,g  

^a Faculty of Health Sciences   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

^c ST MICHAEL'S HOSPITAL   (Canada)

^d Department of Psychiatry and Behavioral Neurosciences ^*  (Canada)

^e Center for Evaluation of Medicine   (Canada)

^f MCMASTER UNIVERSITY   (Canada)

^g MCMASTER UNIVERSITY   (Canada)

^h MCMASTER UNIVERSITY   (Canada)

Author keywords

Analysis; Commentary; Sensitivity; Statistics; Tutorial

Indexed keywords

CLINICAL TRIAL; CONTROLLED CLINICAL TRIAL; HUMAN; MODEL; NUTRITION; RANDOM SAMPLE; RANDOMIZED CONTROLLED TRIAL; VALIDITY; METHODOLOGY; NUTRITIONAL SCIENCE; RANDOMIZED CONTROLLED TRIAL (TOPIC); STATISTICAL BIAS;

BIAS (EPIDEMIOLOGY); HUMANS; NUTRITIONAL SCIENCES; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN;

EID: 84953923652     PISSN: 00029165     EISSN: 19383207     Source Type: Journal    
DOI: 10.3945/ajcn.115.121848     Document Type: Review

References (71)

1. Thabane L, Mbuagbaw L, Zhang S, Samaan Z, Marcucci M, Ye C, Thabane M, Giangregorio L, Dennis B, Kosa D, et al. A tutorial on sensitivity analyses in clinical trials: the what, why, when and how. BMC Med Res Methodol 2013;13:92.
2. Vickers AJ. The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: a simulation study. BMC Med Res Methodol 2001;1:6.
3. Haynes RB, Dantes R. Patient compliance and the conduct and in-terpretation of therapeutic trials. Control Clin Trials 1987;8:12-9.
4. Bendtsen P, McCambridge J, Bendtsen M, Karlsson N, Nilsen P. Effectiveness of a proactive mail-based alcohol Internet intervention for university students: dismantling the assessment and feedback components in a randomized controlled trial. J Med Internet Res 2012;14: e142.
5. Gueorguieva R, Krystal JH. Move over ANOVA: progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Arch Gen Psychiatry 2004;61:310-7.
6. Keselman HJ, Algina J, Kowalchuk RK. The analysis of repeated measures designs: a review. Br J Math Stat Psychol 2001;54:1-20.
7. NINDS NET-PD Investigators. A randomized, double-blind, futility clinical trial of creatine and minocycline in early Parkinson disease. Neurology 2006;66:664-71.
8. Dixon WJ. Processing data for outliers. Biometrics 1953;9:74-89.
9. Andrews DF, Pregibon D. Finding the outliers that matter. J Royal Statistical Society Series B (Methodological) 1978;40:85-93.
10. Ware JH. Interpreting incomplete data in studies of diet and weight loss. N Engl J Med 2003;348:2136-7.
11. Little RJ, D'Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA, et al. The prevention and treatment of missing data in clinical trials. N Engl J Med 2012;367:1355-60.
12. Sun X, Ioannidis JP, Agoritsas T, Alba AC, Guyatt G. How to use a subgroup analysis: users' guide to the medical literature. JAMA 2014;311:405-11.
13. Kasenda B, Schandelmaier S, Sun X, von Elm E, You J, Blumle A, Tomonaga Y, Saccilotto R, Amstutz A, Bengough T, et al. Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications. BMJ 2014;349:g4539.
14. Erratum in: BMJ. 2014;349:4921.
15. Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine-reporting of subgroup analyses in clinical trials. N Engl J Med 2007;357:2189-94.
16. Porta MS, Greenland S, Hernán M, dos Santos Silva I, Last JM. A dictionary of epidemiology. New York: Oxford University Press; 2014.
17. Schneeweiss S. Sensitivity analysis and external adjustment for un-measured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf 2006;15:291-303.
18. International conference on harmonisation; guidance on statistical principles for clinical trials; availability-FDA. Notice. Fed Regist 1998;63:49583-98.
19. Estruch R, Ros E, Salas-Salvado J, Covas MI, Corella D, Aros F, Gomez-Gracia E, Ruiz-Gutierrez V, Fiol M, Lapetra J, et al. Primary prevention of cardiovascular disease with a Mediterranean diet. N Engl J Med 2013;368:1279-90.
20. Sacks FM, Bray GA, Carey VJ, Smith SR, Ryan DH, Anton SD, McManus K, Champagne CM, Bishop LM, Laranjo N, et al. Comparison of weight-loss diets with different compositions of fat, protein, and carbohydrates. N Engl J Med 2009;360:859-73.
21. Goldberg GR, Jarjou LM, Cole TJ, Prentice A. Randomized, placebo-controlled, calcium supplementation trial in pregnant Gambian women accustomed to a low calcium intake: effects on maternal blood pressure and infant growth. Am J Clin Nutr 2013;98:972-82.
22. Belenchia AM, Tosh AK, Hillman LS, Peterson CA. Correcting vitamin D insufficiency improves insulin sensitivity in obese adolescents: a randomized controlled trial. Am J Clin Nutr 2013;97:774-81.
23. Economos CD, Moore CE, Hyatt RR, Kuder J, Chen T, Meydani SN, Meydani M, Klein E, Biancuzzo RM, Holick MF. Multinutrient-fortified juices improve vitamin D and vitamin E status in children: a randomized controlled trial. J Acad Nutr Diet 2014;114:709-17.
24. Harris Jackson K, West SG, Vanden Heuvel JP, Jonnalagadda SS, Ross AB, Hill AM, Grieger JA, Lemieux SK, Kris-Etherton PM. Effects of whole and refined grains in a weight-loss diet on markers of metabolic syndrome in individuals with increased waist circumference: a randomized controlled-feeding trial. Am J Clin Nutr 2014;100:577-86.
25. Laborie S, Denis A, Dassieu G, Bedu A, Tourneux P, Pinquier D, Kermorvant E, Millet V, Klosowski S, Patural H, et al. Shielding parenteral nutrition solutions from light: a randomized controlled trial. JPEN J Parenter Enteral Nutr 2015;39:729-37.
26. Lindsay KL, Kennelly M, Culliton M, Smith T, Maguire OC, Shanahan F, Brennan L, McAuliffe FM. Probiotics in obese pregnancy do not reduce maternal fasting glucose: a double-blind, placebo-controlled, randomized trial (Probiotics in Pregnancy Study). Am J Clin Nutr 2014;99:1432-9.
27. Murray K, Placidi E, Schuring EA, Hoad CL, Koppenol W, Arnaudov LN, Blom WA, Pritchard SE, Stoyanov SD, Gowland PA, et al. Aerated drinks increase gastric volume and reduce appetite as assessed by MRI: a randomized, balanced, crossover trial. Am J Clin Nutr 2015;101:270-8.
28. Neufingerl N, Zebregs YE, Schuring EA, Trautwein EA. Effect of cocoa and theobromine consumption on serum HDL-cholesterol concentrations: a randomized controlled trial. Am J Clin Nutr 2013;97:1201-9.
29. Sadiya A, Ahmed SM, Carlsson M, Tesfa Y, George M, Ali SH, Siddieg HH, Abusnana S. Vitamin D supplementation in obese type 2 diabetes subjects in Ajman, UAE: a randomized controlled double-blinded clinical trial. Eur J Clin Nutr 2015;69:707-11.
30. Sheppard L, Kristal AR, Kushi LH. Weight loss in women participating in a randomized trial of low-fat diets. Am J Clin Nutr 1991;54:821-8.
31. Juanola-Falgarona M, Salas-Salvado J, Ibarrola-Jurado N, Rabassa-Soler A, Diaz-Lopez A, Guasch-Ferre M, Hernandez-Alonso P, Balanza R, Bullo M. Effect of the glycemic index of the diet on weight loss, modulation of satiety, inflammation, and other metabolic risk factors: a randomized controlled trial. Am J Clin Nutr 2014;100:27-35.
32. Liu ZH, Huang MJ, Zhang XW, Wang L, Huang NQ, Peng H, Lan P, Peng JS, Yang Z, Xia Y, et al. The effects of perioperative probiotic treatment on serum zonulin concentration and subsequent postoperative infectious complications after colorectal cancer surgery: a double-center and double-blind randomized clinical trial. Am J Clin Nutr 2013;97: 117-26.
33. Luley C, Blaik A, Gotz A, Kicherer F, Kropf S, Isermann B, Stumm G, Westphal S. Weight loss by telemonitoring of nutrition and physical activity in patients with metabolic syndrome for 1 year. J Am Coll Nutr 2014;33:363-74.
34. Mirza NM, Palmer MG, Sinclair KB, McCarter R, He J, Ebbeling CB, Ludwig DS, Yanovski JA. Effects of a low glycemic load or a low-fat dietary intervention on body weight in obese Hispanic American children and adolescents: a randomized controlled trial. Am J Clin Nutr 2013;97:276-85.
35. Wolfenden L, Wyse R, Campbell E, Brennan L, Campbell KJ, Fletcher A, Wiggers J, Bowman J, Heard TR. Randomized controlled trial of a telephone-based intervention for child fruit and vegetable intake: long-term follow-up. Am J Clin Nutr 2014;99:543-50.
36. Weber M, Grote V, Closa-Monasterolo R, Escribano J, Langhendries JP, Dain E, Giovannini M, Verduci E, Gruszfeld D, Socha P, et al. Lower protein content in infant formula reduces BMI and obesity risk at school age: follow-up of a randomized trial. Am J Clin Nutr 2014;99:1041-51.
37. Jesudason DR, Pedersen E, Clifton PM. Weight-loss diets in people with type 2 diabetes and renal disease: a randomized controlled trial of the effect of different dietary protein amounts. Am J Clin Nutr 2013; 98:494-501.
38. Dangour AD, Allen E, Clarke R, Elbourne D, Fletcher AE, Letley L, Richards M, Whyte K, Uauy R, Mills K. Effects of vitamin B-12 supplementation on neurologic and cognitive function in older people: a randomized controlled trial. Am J Clin Nutr 2015;102:639-47.
39. Wang L, Sesso HD, Glynn RJ, Christen WG, Bubes V, Manson JE, Buring JE, Gaziano JM. Vitamin E and C supplementation and risk of cancer in men: posttrial follow-up in the Physicians' Health Study II randomized trial. Am J Clin Nutr 2014;100:915-23.
40. Giammanco M, Bavetta S, Greco R, Di Majo D, La Guardia M. Therapeutic effectiveness of nutrition therapy in pediatric patients with chronic liver diseases awaiting liver transplantation. J Food Res 2013; 2:179-90.
41. Iannotti LL, Dulience SJ, Green J, Joseph S, Francois J, Antenor ML, Lesorogol C, Mounce J, Nickerson NM. Linear growth increased in young children in an urban slum of Haiti: a randomized controlled trial of a lipid-based nutrient supplement. Am J Clin Nutr 2014;99:198-208.
42. Women's Health Initiative Study Group. Dietary adherence in the Women's Health Initiative Dietary Modification Trial. J Am Diet Assoc 2004;104:654-8.
43. Skov AR, Toubro S, Raben A, Astrup A. A method to achieve control of dietary macronutrient composition in ad libitum diets consumed by free-living subjects. Eur J Clin Nutr 1997;51:667-72.
44. Appel LJ, Sacks FM, Carey VJ, Obarzanek E, Swain JF, Miller ER 3rd, Conlin PR, Erlinger TP, Rosner BA, Laranjo NM, et al. Effects of protein, monounsaturated fat, and carbohydrate intake on blood pres-sure and serum lipids: results of the OmniHeart randomized trial. JAMA 2005;294:2455-64.
45. Appel LJ, Moore TJ, Obarzanek E, Vollmer WM, Svetkey LP, Sacks FM, Bray GA, Vogt TM, Cutler JA, Windhauser MM, et al. A clinical trial of the effects of dietary patterns on blood pressure. DASH Col-laborative Research Group. N Engl J Med 1997;336:1117-24.
46. Bray GA, Smith SR, de Jonge L, Xie H, Rood J, Martin CK, Most M, Brock C, Mancuso S, Redman LM. Effect of dietary protein content on weight gain, energy expenditure, and body composition during overeating: a randomized controlled trial. JAMA 2012;307:47-55.
47. Anderson JW, Story L, Sieling B, Chen WJ, Petro MS, Story J. Hy-pocholesterolemic effects of oat-bran or bean intake for hypercholes-terolemic men. Am J Clin Nutr 1984;40:1146-55.
48. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869.
49. Astrup A, Brand-Miller J. Reply: totality of evidence needed for nu-trition recommendations. Nat Rev Endocrinol 2014;10:310.
50. Montori VM, Guyatt GH. Intention-to-treat principle. CMAJ 2001;165: 1339-41.
51. Jenkins DJ, Kendall CW, Marchie A, Faulkner DA, Wong JM, de Souza R, Emam A, Parker TL, Vidgen E, Lapsley KG, et al. Effects of a dietary portfolio of cholesterol-lowering foods vs lovastatin on serum lipids and C-reactive protein. JAMA 2003;290:502-10.
52. Jenkins DJ, Jones PJ, Lamarche B, Kendall CW, Faulkner D, Cermakova L, Gigleux I, Ramprasath V, de Souza R, Ireland C, et al. Effect of a dietary portfolio of cholesterol-lowering foods given at 2 levels of intensity of dietary advice on serum lipids in hyperlipidemia: a randomized controlled trial. JAMA 2011;306:831-9.
53. Thiese MS, Arnold ZC, Walker SD. The misuse and abuse of statistics in biomedical research. Biochem Med (Zagreb) 2015;25:5-11.
54. McKnight PE, McKnight KM, Sidani S, Figueredo AJ. Missing data: a gentle introduction. New York: Guildford; 2007.
55. Joseph R, Sim J, Ogollah R, Lewis M. A systematic review finds variable use of the intention-to-treat principle in musculoskeletal randomized controlled trials with missing data. J Clin Epidemiol 2015;68:15-24.
56. Little RJ. A test of missing completely at random for multivariate data with missing values. J Am Stat Assoc 1988;83:1198-202.
57. Pi-Sunyer X, Astrup A, Fujioka K, Greenway F, Halpern A, Krempf M, Lau DC, le Roux CW, Violante Ortiz R, Jensen CB, et al. A Randomized, Controlled Trial of 3.0 mg of Liraglutide in Weight Management. N Engl J Med 2015;373:11-22.
58. McManus K, Antinoro L, Sacks F. A randomized controlled trial of a moderate-fat, low-energy diet compared with a low fat, low-energy diet for weight loss in overweight adults. Int J Obes Relat Metab Disord 2001;25:1503-11.
59. Chen L, Appel LJ, Loria C, Lin PH, Champagne CM, Elmer PJ, Ard JD, Mitchell D, Batch BC, Svetkey LP, et al. Reduction in consumption of sugar-sweetened beverages is associated with weight loss: the PREMIER trial. Am J Clin Nutr 2009;89:1299-306.
60. St-Onge MP, Bosarge A. Weight-loss diet that includes consumption of medium-chain triacylglycerol oil leads to a greater rate of weight and fat mass loss than does olive oil. Am J Clin Nutr 2008;87:621-6.
61. Chang KT, Lampe JW, Schwarz Y, Breymeyer KL, Noar KA, Song X, Neuhouser ML. Low glycemic load experimental diet more satiating than high glycemic load diet. Nutr Cancer 2012;64:666-73.
62. Tate DF, Turner-McGrievy G, Lyons E, Stevens J, Erickson K, Polzien K, Diamond M, Wang X, Popkin B. Replacing caloric beverages with water or diet beverages for weight loss in adults: main results of the Choose Healthy Options Consciously Everyday (CHOICE) randomized clinical trial. Am J Clin Nutr 2012;95:555-63.
63. Katcher HI, Legro RS, Kunselman AR, Gillies PJ, Demers LM, Bag-shaw DM, Kris-Etherton PM. The effects of a whole grain-enriched hypocaloric diet on cardiovascular disease risk factors in men and women with metabolic syndrome. Am J Clin Nutr 2008;87:79-90.
64. Mason C, Xiao L, Imayama I, Duggan C, Wang CY, Korde L, McTiernan A. Vitamin D3 supplementation during weight loss: a double-blind randomized controlled trial. Am J Clin Nutr 2014;99:1015-25.
65. Kramer MS, Matush L, Vanilovich I, Platt RW, Bogdanovich N, Sevkovskaya Z, Dzikovich I, Shishko G, Collet JP, Martin RM, et al. Effects of prolonged and exclusive breastfeeding on child height, weight, adiposity, and blood pressure at age 6.5 y: evidence from a large randomized trial. Am J Clin Nutr 2007;86:1717-21.
66. de Souza RJ, Bray GA, Carey VJ, Hall KD, LeBoff MS, Loria CM, Laranjo NM, Sacks FM, Smith S.R. Effects of 4 weight-loss diets differing in fat, protein, and carbohydrate on fat mass, lean mass, visceral adipose tissue, and hepatic fat: results from the POUNDS LOST trial. Am J Clin Nutr 2012;95:614-25.
67. Singal AG, Higgins PD, Waljee AK. A primer on effectiveness and efficacy trials. Clin Transl Gastroenterol 2014;5:e45.
68. Nakai M, Chen D-G, Nishimura KYM. Comparative study of four methods in missing value imputations under missing completely at random mechanism. Open J Stat 2014;4:27-37.
69. Armijo-Olivo S, Warren S, Magee DJ. Intention to treat analysis, compliance, drop-outs and how to deal with missing data in clinical research: a review. Phys Ther Rev 2009;14:36-49.
70. Efird J. Blocked randomization with randomly selected block sizes. Int J Environ Res Public Health 2011;8:15-20.
71. D'Agostino R, Stephens M.A. Goodness-of-fit techniques. New York: Marcel Dekker Inc.; 1986.