Experimental design and primary data analysis methods for comparing adaptive interventions

Psychological Methods

Volumn 17, Issue 4, 2012, Pages 457-477

  Nahum Shani, Inbal ^a   Qian, Min ^b   Almirall, Daniel ^a   Pelham, William E ^c   Gnagy, Beth ^c   Fabiano, Gregory A ^d   Waxmonsky, James G ^e   Yu, Jihnhee ^d   Murphy, Susan A ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

^b Columbia University ^*  (United States)

^c FLORIDA INTERNATIONAL UNIVERSITY   (United States)

^d UNIVERSITY AT BUFFALO   (United States)

^e FLORIDA INTERNATIONAL UNIVERSITY   (United States)

Author keywords

Adaptive interventions; Experimental design; Sequential multiple assignment randomized trial (SMART)

Indexed keywords

ARTICLE; BEHAVIORAL SCIENCE; HUMAN; METHODOLOGY; RANDOMIZED CONTROLLED TRIAL (TOPIC); SOCIOLOGY; STATISTICAL ANALYSIS;

BEHAVIORAL SCIENCES; DATA INTERPRETATION, STATISTICAL; HUMANS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN; SOCIAL SCIENCES;

EID: 84867783897     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0029372     Document Type: Article

References (76)

1. Abele, A. E., & Wiese, B. S. (2008). The nomological network of selfmanagement strategies and career success. Journal of Occupational and Organizational Psychology, 81, 733-749. doi:10.1348/096317907X256726
2. Baltes, P. B., & Baltes, M. M. (1990). Psychological perspectives on successful aging: The model of selective optimization with compensation. In P. B. Baltes & M. M. Baltes (Eds.), Successful aging: Perspectives from the behavioural sciences (pp. 1-34). Cambridge, MA: Cambridge University Press. doi:10.1017/CBO9780511665684.003
3. Barlas, Y., & Yasarcan, H. (2006). Goal setting, evaluation, learning and revision: A dynamic modeling approach. Evaluation and Program Planning, 29, 79-87. doi:10.1016/j.evalprogplan.2005.08.003
4. Beck, J. S., Liese, B. S., & Najavits, L. M. (2005). Cognitive therapy. In R. J. Frances, S. I. Miller, & A. Mack (Eds.), Clinical textbook of addictive disorders (3rd ed., pp. 474-501). New York, NY: Guilford Press.
5. Bierman, K. L., Nix, R. L., Maples, J. J., & Murphy, S. A. (2006). Examining clinical judgment in an adaptive intervention design: The Fast Track Program. Journal of Consulting and Clinical Psychology, 74, 468-481. doi:10.1037/0022-006X.74.3.468
6. Campion, M. A., & Lord, R. G. (1982). A control systems conceptualization of the goal-setting and changing process. Organizational Behavior and Human Performance, 30, 265-287. doi:10.1016/00305073(82)90221-5
7. Cole, M. B. (2005). Group dynamics in occupational therapy: The theoretical basis and practice application of group treatment (3rd ed.). Thorofare, NJ: Slack.
8. Cole, S. R., & Hernán, M. A. (2008). Constructing inverse probability weights for marginal structural models. American Journal of Epidemiology, 168, 656-664. doi:10.1093/aje/kwn164
9. Collins, L. M., Dziak, J. J., & Li, R. (2009). Design of experiments with multiple independent variables: A resource management perspective on complete and reduced factorial designs. Psychological Methods, 14, 202-224. doi:10.1037/a0015826
10. Collins, L. M., Murphy, S. A., & Bierman, K. A. (2004). A conceptual framework for adaptive preventive interventions. Prevention Science, 5, 185-196. doi:10.1023/B:PREV.0000037641.26017.00
11. Conduct Problems Prevention Research Group. (1992). A developmental and clinical model for the prevention of conduct disorders: The FAST Track Program. Development and Psychopathology, 4, 509-527. doi: 10.1017/S0954579400004855
12. Connell, A. M. (2009). Employing complier average causal effect analytic methods to examine effects of randomized encouragement trials. The American Journal of Drug and Alcohol Abuse, 35, 253-259. doi: 10.1080/00952990903005882
13. Connell, A. M., & Dishion, T. J. (2008). Reducing depression among at-risk early adolescents: Three-year effects of a family-centered intervention embedded within schools. Journal of Family Psychology, 22, 574-585. doi:10.1037/0893-3200.22.3.574
14. Connell, A. M., Dishion, T. J., Yasui, M., & Kavanagh, K. (2007). An adaptive approach to family intervention: Linking engagement in family-centered intervention to reductions in adolescent problem behavior. Journal of Consulting and Clinical Psychology, 75, 568-579. doi: 10.1037/0022-006X.75.4.568
15. Cook, T. D., & Campbell, D. T. (1979). Quasiexperimentation: Design and analysis issues for field settings. Chicago, IL: Rand McNally.
16. Dishion, T. J., & Kavanagh, K. (2003). Intervening in adolescent problem behavior: A family-centered approach. New York, NY: Guilford Press.
17. Donders, A. R. T., van der Heijden, G. J. M. G., Stijnen, T., & Moons, K. G. M. (2006). A gentle introduction to imputation of missing values. Journal of Clinical Epidemiology, 59, 1087-1091. doi:10.1016/j.jclinepi.2006.01.014
18. Dziak, J. J., Nahum-Shani, I., & Collins, L. M. (2012). Multilevel factorial experiments for developing behavioral interventions: Power, sample size, and resource considerations. Psychological Methods, 17, 153-175. doi:10.1037/a0026972
19. Efron, B. (1971). Forcing a sequential experiment to be balanced. Biometrika, 58, 403-417. doi:10.1093/biomet/58.3.403
20. Fabiano, G. A., Pelham, W. E., Waschbusch, D. A., Gnagy, E. M., Lahey, B. B., Chronis, A. M.,... Burrows-MacLean, L. (2006). A practical measure of impairment: Psychometric properties of the impairment rating scale in samples of children with attention-deficit/hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35, 369-385. doi:10.1207/s15374424jccp3503_3
21. Fava, M., Rush, A. J., Trivedi, M., Nierenberg, A. A., Thase, M. E., Sackeim, H.,... Kupfer, D. J. (2003). Background and rationale for the sequenced treatment alternatives to relieve depression (STAR*D) study. Psychiatric Clinics of North America, 26, 457-494. doi:10.1016/S0193953X(02)00107-7
22. Fried, Y., & Slowik, L. H. (2004). Enriching goal-setting theory with time: An integrated approach. The Academy of Management Review, 29, 404-422.
23. Gopher, D. (1993). The skill of attention control: Acquisition and execution of attention strategies. In D. Meyer & S. Kornblum (Eds.), Attention and Performance XIV: Synergies in experimental psychology, artificial intelligence, and cognitive neuroscience (pp. 299-322). Cambridge, MA: MIT Press.
24. Gunter, L., Zhu, J., & Murphy, S. A. (2011). Variable selection for qualitative interactions. Statistical Methodology, 8, 42-55. doi:10.1016/j.stamet.2009.05.003
25. Hedden, S. L., Woolson, R. F., & Malcolm, R. J. (2006). Randomization in substance abuse clinical trials. Substance Abuse Treatment, Prevention, and Policy, 1, 1-17. doi:10.1186/1747-597X-1-6
26. Hernán, M. A. (2004). A definition of causal effect for epidemiological research. Journal of Epidemiology and Community Health, 58, 265-271. doi:10.1136/jech.2002.006361
27. Hernán, M. A., Brumback, B., & Robins, J. M. (2000). Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology, 11, 561-570. doi:10.1097/00001648200009000-00012
28. Hernán, M. A., Brumback, B., & Robins, J. M. (2002). Estimating the causal effect of zidovudine on CD4 count with a marginal structural model for repeated measures. Statistics in Medicine, 21, 1689-1709. doi:10.1002/sim.1144
29. Hirschi, A., & Vondracek, F. W. (2009). Adaptation of career goals to self and opportunities in early adolescence. Journal of Vocational Behavior, 75, 120-128. doi:10.1016/j.jvb.2009.05.005
30. Hu, F. B., Goldberg, J., Hedeker, D., Flay, B. F., & Pentz, M. A. (1998). Comparison of population-averaged and subject-specific approaches for analyzing repeated binary outcomes. American Journal of Epidemiology, 147, 694-703. doi:10.1093/oxfordjournals.aje.a009511
31. Kernan, W. N., Viscoli, C. M., Makuch, R. W., Brass, L. M., & Horwitz, R. I. (1999). Stratified randomization for clinical trials. Journal of Clinical Epidemiology, 52, 19-26. doi:10.1016/S0895-4356(98)00138-3
32. Laurenceau, J.-P., Hayes, A. M., & Feldman, G. C. (2007). Statistical and methodological issues in the study of change in psychotherapy. Clinical Psychology Review, 27, 682-695. doi:10.1016/j.cpr.2007.01.007
33. Lavori, P. W., & Dawson, R. (2000). A design for testing clinical strategies: Biased individually tailored within-subject randomization. Journal of the Royal Statistical Society, Series A: Statistics in Society, 163, 29-38. doi:10.1111/1467-985X.00154
34. Lavori, P. W., & Dawson, R. (2004). Dynamic treatment regimes: Practical design considerations. Clinical Trials, 1, 9-20. doi:10.1191/1740774S04cn002oa
35. Lavori, P. W., Rush, A. J., Wisniewski, S. R., Alpert, J., Fava, M., Kupfer, D. J.,... Trivedi, M. (2001). Strengthening clinical effectiveness trials: Equipoise-stratified randomization. Biological Psychiatry, 50, 792-801. doi:10.1016/S0006-3223(01)01223-9
36. Liang, K. Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73, 13-22. doi:10.1093/biomet/73.1.13
37. Locke, E. A., & Latham, P. G. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice-Hall.
38. Lunceford, J. K., Davidian, M., & Tsiatis, A. A. (2002). Estimation of survival distributions of treatment policies in two-stage randomization designs in clinical trials. Biometrics, 58, 48-57. doi:10.1111/j.0006341X.2002.00048.x
39. Marlowe, D. B., Festinger, D. S., Arabia, P. L., Dugosh, K. L., Benasutti, K. M., Croft, J. R., & McKay, J. R. (2008). Adaptive interventions in drug court: A pilot experiment. Criminal Justice Review, 33, 343-360. doi:10.1177/0734016808320325
40. Martocchio, J. J., & Webster, J. (1992). Effects of feedback and cognitive playfulness on performance in microcomputer software training. Personnel Psychology, 45, 553-578. doi:10.1111/j.17446570.1992.tb00860.x
41. McKay, J. R. (2005). Is there a case for extended interventions for alcohol and drug use disorders? Addiction, 100, 1594-1610. doi:10.1111/j.13600443.2005.01208.x
42. Montgomery, A. A., Peters, T. J., & Little, P. (2003). Design, analysis and presentation of factorial randomized controlled trials. BMC Medical Research Methodology, 3, 26. doi:10.1186/1471-2288-3-26
43. Moore, R., & Blackburn, I. (1997). Cognitive therapy in the treatment of nonresponders to antidepressant medication: A controlled pilot study. Behavioural and Cognitive Psychotherapy, 25, 251-259. doi:10.1017/S1352465800018543
44. Murphy, S. A. (2005). An experimental design for the development of adaptive treatment strategies. Statistics in Medicine, 24, 1455-1481. doi:10.1002/sim.2022
45. Murphy, S. A., Collins, L. M., & Rush, A. J. (2007). Customizing treatment to the patient: Adaptive treatment strategies [Editorial]. Drug and Alcohol Dependence, 88(Suppl. 2), S1-S3. doi:10.1016/j.drugalcdep .2007.02.001
46. Murphy, S. A., van der Laan, M. J., Robins, J. M., & Conduct Problems Prevention Research Group. (2001). Marginal mean models for dynamic regimes. Journal of the American Statistical Association, 96, 1410- 1423. doi:10.1198/016214501753382327
47. Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W. E., Gnagy, B., Fabiano, G. A.,... Murphy, S. A. (2012). Q-learning: A data analysis method for constructing adaptive interventions. Psychological Methods,17, 478-494. doi:10.1037/a0029373
48. Oetting, A. I., Levy, J. A., Weiss, R. D., & Murphy, S. A. (2007). Statistical methodology for a SMART design in the development of adaptive treatment strategies. In P. E. Shrout (Ed.), Causality and psychopathology: Finding the determinants of disorders and their cures (pp. 179-205). Arlington, VA: American Psychiatric Publishing.
49. Orellana, L., Rotnitzky, A., & Robins, J. M. (2010). Dynamic regime marginal structural mean models for estimation of optimal dynamic treatment regimes, Part II: Proofs of results. The International Journal of Biostatistics, 6(2). doi:10.2202/1557-4679.1242
50. Pelham, W. E., Evans, S. W., Gnagy, E. M., & Greenslade, K. E. (1992). Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders: Prevalence, factor analyses, and conditional probabilities in a special education sample. School Psychology Review, 21, 285-299.
51. Pelham, W. E., Hoza, B., Pillow, D. R., Gnagy, E. M., Kipp, H. L., Greiner, A. R.,... Fitzpatrick, E. (2002). Effects of methylphenidate and expectancy on children with ADHD: Behavior, academic performance, and attributions in a summer treatment program and regular classroom setting. Journal of Consulting and Clinical Psychology, 70, 320-335. doi:10.1037/0022-006X.70.2.320
52. Robins, J. (1986). A new approach to causal inference in mortality studies with sustained exposure periods: Application to control of the healthy worker survivor effect. Computers and Mathematics with Applications, 14, 1393-1512.
53. Robins, J., Orellana, L., & Rotnitzky, A. (2008). Estimation and extrapolation of optimal treatment and testing strategies. Statistics in Medicine, 27, 4678-4721. doi:10.1002/sim.3301
54. Robins, J. M., & Wasserman, L. (1997). Estimation of effects of sequential treatments by reparameterizing directed acyclic graphs. In D. Geiger & P. Shenoy (Eds.), Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (pp. 409-420). San Francisco, CA: Kaufmann.
55. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688-701. doi:10.1037/h0037350
56. Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6, 34-58. doi:10.1214/aos/1176344064
57. Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence-based medicine: What it is and what it isn't. British Medical Journal, 312, 71-72. doi:10.1136/bmj.312.7023.71
58. SAS Institute. (2008). SAS/STAT 9.2 user's guide. Cary, NC: Author.
59. Schneider, L. S., Tariot, P. N., Lyketsos, C. G., Dagerman, K. S., Davis, K. L., Davis, S.,... Lieberman, J. A. (2001). National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE): Alzheimer disease trial methodology. The American Journal of Geriatric Psychiatry, 9, 346-360.
60. Shadish, W. R. (2002). Revisiting field experimentation: Field notes for the future. Psychological Methods, 7, 3-18. doi:10.1037/1082-989X.7.1.3
61. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
62. Shani, G. (2007). A training approach to bridge the gap between wizards and macro users when working with computers (Unpublished master's thesis). Department of Industrial Engineering and Management, Technion: Israel Institute of Technology, Haifa, Israel.
63. Shortreed, S. M., Laber, E., Pineau, J., & Murphy, S. A. (2010). Imputation methods for the Clinical Antipsychotic Trials of Intervention and Effectiveness study (Technical Report No. SOCS-TR-2010.8). School of Computer Science, McGill University, Montreal, Quebec, Canada.
64. Simon, R. (1979). Restricted randomization designs in clinical trials. Biometrics, 35, 503-512. doi:10.2307/2530354
65. Splawa-Neyman, J. (1990). On the application of probability theory to agricultural experiments: Essay on principles. Section 9. Statistical Science, 5, 465-480. (Reprinted from Roczniki Nauk Rolniczych Tom X, pp. 1-51, by D. M. Dabrowska & T. P. Speed, Eds., 1923)
66. Stone, R. M., Berg, D. T., George, S. L., Dodge, R. K., Paciucci, P. A., Schulman, P.,... Schiffer, C. A. (1995). Granulocyte-macrophage colony-stimulating factor after initial chemotherapy for elderly patients with primary acute myelogenous leukemia. The New England Journal of Medicine, 332, 1671-1677. doi:10.1056/NEJM199506223322503
67. Stroup, T. S., McEvoy, J. P., Swartz, M. S., Byerly, M. J., Glick, I. D., Canive, J. M.,... Lieberman, J. A. (2003). National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE): Schizophrenia trial design and protocol development. Schizophrenia Bulletin, 29, 15-31. doi:10.1093/oxfordjournals.schbul.a006986
68. Thall, P. F., Logothetis, C., Pagliaro, L. C., Wen, S., Brown, M. A., Williams, D., & Millikan, R. E. (2007). Adaptive therapy for androgenindependent prostate cancer: A randomized selection trial of four regimens. Journal of the National Cancer Institute, 99, 1613-1622. doi: 10.1093/jnci/djm189
69. Thall, P. F., Millikan, R. E., & Sung, H. G. (2000). Evaluating multiple treatment courses in clinical trials. Statistics and Medicine, 19, 1011- 1028. doi:10.1002/(SICI)1097-0258(20000430)19:8<011::AIDSIM414>3.0.CO;2-M
70. Thall, P. F., Sung, H. G., & Estey, E. H. (2002). Selecting therapeutic strategies based on efficacy and death in multicourse clinical trials. Journal of the American Statistical Association, 97, 29-39. doi:10.1198/016214502753479202
71. Thall, P. F., & Wathen, J. K. (2005). Covariate-adjusted adaptive randomization in a sarcoma trial with multi-stage treatments. Statistics in Medicine, 24, 1947-1964. doi:10.1002/sim.2077
72. Wahed, A. S., & Tsiatis, A. A. (2004). Optimal estimator for the survival distribution and related quantities for treatment policies in two-stage randomization designs in clinical trials. Biometrics, 60, 124-133. doi: 10.1111/j.0006-341X.2004.00160.x
73. Wahed, A. S., & Tsiatis, A. A. (2006). Semiparametric efficient estimation of survival distribution for treatment policies in two-stage randomization designs in clinical trials with censored data. Biometrika, 93, 163-177. doi:10.1093/biomet/93.1.163
74. Walsh, B. T., Agras, W. S., Devlin, M. J., Fairburn, C. G., Wilson, G. T., Kahn, C., & Chally, M. K. (2000). Fluoxetine for bulimia nervosa following poor response to psychotherapy. The American Journal of Psychiatry, 157, 1332-1334. doi:10.1176/appi.ajp.157.8.1332
75. Weisz, J. R., Chu, B. C., & Polo, A. J. (2004). Treatment dissemination and evidence-based practice: Strengthening interventions through clinician- researcher collaboration. Clinical Psychology: Science and Practice, 11, 300-307. doi:10.1093/clipsy.bph085
76. Yalom, I. D. (1995). The theory and practice of group psychotherapy (4th ed.). New York, NY: Basic Books