Statistical challenges in assessing potential efficacy of complex interventions in pilot or feasibility studies

Statistical Methods in Medical Research

Volumn 25, Issue 3, 2016, Pages 997-1009

  Wilson, Duncan T ^a   Walwyn, Rebecca E A ^a   Brown, Julia ^a   Farrin, Amanda J ^a   Brown, Sarah R ^a  

^a UNIVERSITY OF LEEDS   (United Kingdom)

Author keywords

Complex interventions; feasibility; phase II; pilot; sample size

Indexed keywords

ARTICLE; BAYES THEOREM; CLINICAL EFFECTIVENESS; CLINICAL TRIAL (TOPIC); COMPLEX INTERVENTION; DATA ANALYSIS; DECISION MAKING; DRUG EFFICACY; EVALUATION STUDY; FEASIBILITY STUDY; PATIENT CARE; PILOT STUDY; SAMPLE SIZE; STATISTICAL ANALYSIS; STATISTICAL DISTRIBUTION; STUDY DESIGN; HUMAN; METHODOLOGY; PHASE 2 CLINICAL TRIAL (TOPIC); PROCEDURES;

BAYES THEOREM; CLINICAL TRIALS, PHASE II AS TOPIC; FEASIBILITY STUDIES; HUMANS; PILOT PROJECTS; RESEARCH DESIGN; SAMPLE SIZE;

EID: 84970967867     PISSN: 09622802     EISSN: 14770334     Source Type: Journal    
DOI: 10.1177/0962280215589507     Document Type: Article

References (74)

1. Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ 2008; 337: a1655-a1655.
2. Walwyn RAE. Therapist variation within meta-analyses of psychotherapy trials. University of Manchester, UK, 2010.
3. Petticrew M,. When are complex interventions 'complex'? When are simple interventions 'simple'? Eur J Public Health 2011; 21: 397-398.
4. Campbell M, Fitzpatrick R, Haines A, et al. Framework for design and evaluation of complex interventions to improve health. BMJ 2000; 321: 694-696.
5. Barkun JS, Aronson JK, Feldman LS, et al. Evaluation and stages of surgical innovations. Lancet 2009; 374: 1089-1096.
6. Collins LM, Murphy SA, Nair VN, et al. A strategy for optimizing and evaluating behavioral interventions. Ann Behav Med 2005; 30: 65-73.
7. Collins LM, Chakraborty B, Murphy SA, et al. Comparison of a phased experimental approach and a single randomized clinical trial for developing multicomponent behavioral interventions. Clin Trials 2009; 6: 5-15.
8. Arain M, Campbell MJ, Cooper CL, et al. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol 2010; 10: 67-67.
9. Thabane L, Ma J, Chu R, et al. A tutorial on pilot studies: The what, why and how. BMC Med Res Methodol 2010; 10: 1-1.
10. Billingham S, Whitehead A, Julious S,. An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database. BMC Med Res Methodol 2013; 13: 104-104.
11. Julious SA,. Sample size of 12 per group rule of thumb for a pilot study. Pharmaceut Stat 2005; 4: 287-291.
12. Browne RH,. On the use of a pilot sample for sample size determination. Stat Med 1995; 14: 1933-1940.
13. Lancaster GA, Dodd S, Williamson PR,. Design and analysis of pilot studies: Recommendations for good practice. J Eval Clin Pract 2004; 10: 307-312.
14. Arnold DM, Burns KE, Adhikari NK, et al. The design and interpretation of pilot trials in clinical research in critical care. Crit Care Med 2009; 37: S69-S74.
15. Lee E, Whitehead AL, Jacques RM, et al. The statistical interpretation of pilot trials: Should significance thresholds be reconsidered? BMC Med Res Methodol 2014; 14: 41-41.
16. Bland JM,. Cluster randomised trials in the medical literature: Two bibliometric surveys. BMC Med Res Methodol 2004; 4: 21-21.
17. Walwyn REA, Roberts C,. Therapist variation within randomised trials of psychotherapy: Implications for precision, internal and external validity. Stat Meth Med Res 2010; 19: 291-315.
18. Roberts C, Roberts SA,. Design and analysis of clinical trials with clustering effects due to treatment. Clin Trials 2005; 2: 152-162.
19. Serlin RC, Wampold BE, Levin JR,. Should providers of treatment be regarded as a random factor? If it ain't broke, don't "fix" it: A comment on Siemer and Joormann (2003). Psychol Meth 2003; 8: 524-534.
20. House A, Ajjan R, Bryant L, et al. Managing with learning disability and diabetes. HTA protocol, http://www.nets.nihr.ac.uk/projects/hta/1010203 (2014, accessed 1 June 2015).
21. Mariani L, Marubini E,. Design and analysis of phase II cancer trials: A review of statistical methods and guidelines for medical researchers. Int Stat Rev 1996; 64: 61-88.
22. Fleming TR,. One-sample multiple testing procedure for phase II clinical trials. Biometrics 1982; 38: 143-151.
23. A'Hern RP,. Sample size tables for exact single-stage phase II designs. Stat Med 2001; 20: 859-866.
24. Brown SR, Gregory WM, Twelves CI, et al. Designing phase II trials in cancer: A systematic review and guidance. Br J Cancer 2011; 105: 194-199.
25. Simon R,. Optimal two-stage designs for phase II clinical trials. Control Clin Trials 1989; 10: 1-10.
26. Simon R, Wittes RE, Ellenberg SS,. Randomized phase II clinical trials. Cancer Treat Rep 1985; 69: 1375-1381.
27. Jung S-H,. Randomized phase II trials with a prospective control. Stat Med 2008; 27: 568-583.
28. Bryant J, Day R,. Incorporating toxicity considerations into the design of two-stage phase II clinical trials. Biometrics 1995; 51: 1372-1383.
29. Stallard N, Thall PF, Whitehead J,. Decision theoretic designs for phase II clinical trials with multiple outcomes. Biometrics 1999; 55: 971-977.
30. Sill MW, Rubinstein L, Litwin S, et al. A method for utilizing co-primary efficacy outcome measures to screen regimens for activity in two-stage Phase II clinical trials. Clin Trials 2012; 9: 385-395.
31. Tan S-B, Machin D,. Bayesian two-stage designs for phase II clinical trials. Stat Med 2002; 21: 1991-2012.
32. Brunier HC, Whitehead J,. Sample sizes for phase II clinical trials derived from Bayesian decision theory. Stat Med 1994; 13: 2493-2502.
33. Stallard N,. Optimal sample sizes for phase II clinical trials and pilot studies. Stat Med 2012; 31: 1031-1042.
34. Conaway MR, Petroni GR,. Bivariate sequential designs for phase II trials. Biometrics 1995; 51: 656-664.
35. Conaway MR, Petroni GR,. Designs for phase II trials allowing for a trade-off between response and toxicity. Biometrics 1996; 52: 1375-1386.
36. Robertson T, Wright FT, Dykstra R,. Order restricted statistical inference, New York: John Wiley and Sons, 1988.
37. Tournoux C, De Rycke Y, Médioni J, et al. Methods of joint evaluation of efficacy and toxicity in phase II clinical trials. Contemp Clin Trials 2007; 28: 514-524.
38. Sargent DJ, Chan V, Goldberg RM,. A three-outcome design for phase II clinical trials. Control Clin Trials 2001; 22: 117-125.
39. Lindley DV,. The choice of sample size. J Roy Stat Soc Ser D 1997; 46: 129-138.
40. Spiegelhalter DJ, Freedman LS, Parmar MKB,. Bayesian approaches to randomized trials. J Roy Stat Soc Ser A 1994; 157: 357-416.
41. Berger JO,. Statistical decision Theory and Bayesian analysis, 2nd ed. New York: Springer-Verlag, 1985.
42. Thall PF, Nguyen HQ, Braun TM, et al. Using joint utilities of the times to response and toxicity to adaptively optimize schedule-dose regimes. Biometrics 2013; 69: 673-682.
43. Adcock CJ,. Sample size determination: A review. J Roy Stat Soc Ser D 1997; 46: 261-283.
44. Wolpert DH, Macready WG,. No free lunch theorems for optimization. IEEE Trans Evolut Comput 1997; 1: 67-82.
45. Mebane J, Walter R, Sekhon JS,. Genetic optimization using derivatives: The rgenoud Package for R. J Stat Softw 2011; 42: 1-26.
46. Roberts C,. The implications of variation in outcome between health professionals for the design and analysis of randomized controlled trials. Stat Med 1999; 18: 2605-2615.
47. Schnurr PP, Friedman MJ, Engel CC, et al. Cognitive behavioral therapy for posttraumatic stress disorder in women: A randomized controlled trial. JAMA 2007; 297: 820-830.
48. Goldstein H,. Multilevel statistical models, 3rd ed. London: Arnold, 2003.
49. Cohen JA, Mannarino AP,. A treatment outcome study for sexually abused preschool children: Initial findings. J Am Acad Child Adolesc Psychiatry 1996; 35: 42-50.
50. Wampold BE, Serlin RC,. The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychol Meth 2000; 5: 425-433.
51. Siemer M, Joormann J,. Power and measures of effect size in analysis of variance with fixed versus random nested factors. Psychol Meth 2003; 8: 497-517.
52. Siemer M and Joormann J. Assumptions and consequences of treating providers in therapy studies as fixed versus random effects: Reply to Crits-Christoph, Tu, and Gallop (2003) and Serlin, Wampold, and Levin (2003). Psychol Meth 2003; 8: 535-544.
53. Lee KJ, Thompson SG,. The use of random effects models to allow for clustering in individually randomized trials. Clin Trials 2005; 2: 163-173.
54. McCulloch CE, Searle SR, Neuhaus JM,. Generalized, linear, and mixed models, 2nd ed. New York: Wiley, 2008.
55. Landau S, Stahl D,. Sample size and power calculations for medical studies by simulation when closed form expressions are not available. Stat Meth Med Res 2013; 22: 324-345.
56. Donner A, Klar N,. Design and analysis of cluster randomization trials in health research, London: Arnold Publishers, 2000.
57. Hu J, Su Z,. Efficient error determination in sequential clinical trial design. J Comput Graph Stat 2008; 17: 925-948.
58. Browne WJ, Lahi MG and Parker RM. A guide to sample size calculations for random effect models via simulation and the MLPowSim Software Package, 2009.
59. Hooper R,. Versatile sample-size calculation using simulation. STATA J 2013; 13: 21-38.
60. Smith MK, Marshall A,. Importance of protocols for simulation studies in clinical drug development. Stat Meth Med Res 2010; 20: 613-622.
61. Jones DR,. A taxonomy of global optimization methods based on response surfaces. J Global Optim 2001; 21: 345-383.
62. Roustant O, Ginsbourger D, Deville Y,. DiceKriging, DiceOptim: Two R Packages for the analysis of computer experiments by Kriging-based metamodeling and optimization. J Stat Softw 2012; 51: 1-55.
63. Martinez C. Ruben, BayesOpt: A Bayesian optimization library for nonlinear optimization. Experimental Design and Bandits. CoRR, 2014. abs/1405.7430.
64. Campbell MK, Elbourne DR, Altman DG,. CONSORT statement: Extension to cluster randomised trials. BMJ 2004; 328: 702-708.
65. Spiegelhalter DJ,. Bayesian methods for cluster randomized trials with continuous responses. Stat Med 2001; 20: 435-452.
66. Turner RM, Omar RZ, Thompson SG,. Bayesian methods of analysis for cluster randomized trials with binary outcome data. Stat Med 2001; 20: 453-472.
67. Turner RM, Toby Prevost A, Thompson SG,. Allowing for imprecision of the intracluster correlation coefficient in the design of cluster randomized trials. Stat Med 2004; 23: 1195-1214.
68. Turner RM, Thompson SG, Spiegelhalter DJ,. Prior distributions for the intracluster correlation coefficient, based on multiple previous estimates, and their application in cluster randomized trials. Clin Trials 2005; 2: 108-118.
69. Hobbs BP, Carlin BP, Sargent DJ,. Adaptive adjustment of the randomization ratio using historical control data. Clin Trials 2013; 10: 430-440.
70. Hobbs BP, Carlin BP, Mandrekar SJ, et al. Hierarchical commensurate and power prior models for adaptive incorporation of historical information in clinical trials. Biometrics 2011; 67: 1047-1056.
71. Hox J,. Multilevel analysis: Techniques and applications, Hillsdale, NJ: Lawrence Erlbaum Associates, Inc., 2002.
72. Miettinen KM,. Nonlinear multiobject multioptimization, Dordrecht: Kluwer Academic Publishers, 1998.
73. Khan I, Sarker S-J, Hackshaw A,. Smaller sample sizes for phase II trials based on exact tests with actual error rates by trading-off their nominal levels of significance and power. Br J Cancer 2012; 107: 1801-1809.
74. Jung S-H, Carey M, Kim KM,. Graphical search for two-stage designs for phase II clinical trials. Control Clin Trials 2001; 22: 367-372.