Inference for Optimal Dynamic Treatment Regimes Using an Adaptive m-Out-of-n Bootstrap Scheme

Biometrics

Volumn 69, Issue 3, 2013, Pages 714-723

  Chakraborty, Bibhas ^a   Laber, Eric B ^b   Zhao, Yingqi ^c  

^a Columbia University ^*  (United States)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

^c UNIVERSITY OF WISCONSIN SCHOOL OF MEDICINE AND PUBLIC HEALTH   (United States)

Author keywords

Dynamic Treatment Regime; m Out of n Bootstrap; Nonregularity; Q learning

Indexed keywords

LEARNING ALGORITHMS; STATISTICAL METHODS;

ASYMPTOTICS; COVARIATES; DECISION RULES; DYNAMIC TREATMENT REGIME; DYNAMIC TREATMENTS; M-OUT-OF-N BOOTSTRAP; NON-SMOOTHNESS; NONREGULARITY; OPTIMAL DYNAMICS; Q-LEARNING;

PATIENT TREATMENT;

BIOMETRY; BOOTSTRAPPING; CONFIDENCE INTERVAL; DATA SET; DISEASE TREATMENT; NUMERICAL METHOD; SIMULATION;

ANTIDEPRESSANT AGENT;

ARTICLE; ARTIFICIAL INTELLIGENCE; BIOMETRY; COMPUTER SIMULATION; CONFIDENCE INTERVAL; DECISION THEORY; DYNAMIC TREATMENT REGIME; HUMAN; M-OUT-OF-N BOOTSTRAP; MAJOR DEPRESSION; METHODOLOGY; MONTE CARLO METHOD; NONREGULARITY; Q-LEARNING; RANDOMIZED CONTROLLED TRIAL (TOPIC); STATISTICAL MODEL; STATISTICS;

DYNAMIC TREATMENT REGIME; M-OUT-OF-N BOOTSTRAP; NONREGULARITY; Q-LEARNING;

ANTIDEPRESSIVE AGENTS; ARTIFICIAL INTELLIGENCE; BIOMETRY; COMPUTER SIMULATION; CONFIDENCE INTERVALS; DECISION THEORY; DEPRESSIVE DISORDER, MAJOR; HUMANS; LINEAR MODELS; LOGISTIC MODELS; MODELS, STATISTICAL; MONTE CARLO METHOD; RANDOMIZED CONTROLLED TRIALS AS TOPIC;

EID: 84901199092     PISSN: 0006341X     EISSN: 15410420     Source Type: Journal    
DOI: 10.1111/biom.12052     Document Type: Article

References (42)

1. Bellman, R. (1957). Dynamic programming. Princeton University Press, Princeton.
2. Bickel, P. J. and Freedman, D. A. (1981). Some asymptotic theory for the bootstrap. The Annals of Statistics 9, 1196-1217.
3. Bickel P., Gotze F., and van Zwet W. (1997). Resampling fewer than n observations: gains, losses and remedies for losses. Statistica Sinica 7, 1-31.
4. Bickel, P. and Sakov, A. (2008). On the choice of m in the m out of n bootstrap and confidence bounds for extrema. Statistica Sinica 18, 967-985.
5. Bretagnolle, J. (1983). Lois limites du bootstrap de certaines fonctionnelles. In Annales de l'IHP Probabilités et statistiques, volume 19, pages 281-296. Elsevier.
6. Chakraborty, B. and Murphy, S.A. and Strecher, V. (2010). Inference for non-regular parameters in optimal dynamic treatment regimes. Statistical Methods in Medical Research 19, 317-343.
7. Cheung, K. Y., Lee, S. M. S., and Young, G. A. (2005). Iterating the m out of n bootstrap in nonregular smooth function models. Statistica Sinica 15, 945-967.
8. Davison, A. C. and Hinkley, D. V. (1997). Bootstrap methods and their application. Cambridge University Press, Cambridge, UK.
9. Dawson, R. and Lavori, P. (2004). Placebo-free designs for evaluating new mental health treatments: the use of adaptive treatment strategies. Statistics in Medicinepages 3249-3262.
10. Dümbgen, L. (1993). On nondifferentiable functions and the bootstrap. Probability Theory and Related Fieldspages 125-140.
11. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics 7, 1-26.
12. Efron, B. and Tibshirani, R. (1993). An Introduction to the Bootstrap, volume 57. Chapman & Hall/CRC, Boca Raton, Florida.
13. Fava, M., Rush, A., Trivedi, M., Nierenberg, A., Thase, M., Sackeim, H., Quitkin, F., Wisniewski, S., Lavori, P., Rosenbaum, J., and Kupfer, D. (2003). Background and rationale for the sequenced treatment alternatives to relieve depression (star d) study. Psychiatric Clinics of North America 26, 457-494.
14. Friedman, J. H. and Hall, P. (2007). On bagging and nonlinear estimation. Journal of statistical planning and inference 137, 669-683.
15. Hall, P., Horowitz, J., and Jing, B. (1995). On blocking rules for the bootstrap with dependent data. Biometrika 82, 561-574.
16. Hirano, K. and Porter, J. (2012). Impossibility results for nondifferentiable functionals. Econometrica forthcoming.
17. Huang, J., Sen, P., and Shao, J. (1996). Bootstrapping a sample quantile when the density has a jump. Statistica Sinica 6, 299-309.
18. Laber, E. and Murphy, S. (2011). Adaptive confidence intervals for the test error in classification. Journal of the American Statistical Association 106, 904-913.
19. Laber, E., Qian, M., Lizotte, D., and Murphy, S. (2012). Statistical inference in dynamic treatment regimes. arXiv:1006.5831v2 [stat.ME].
20. Lavori, P. and Dawson, R. (2000). A design for testing clinical strategies: biased adaptive within-subject randomization. Journal of the Royal Statistical Society, Series A 163, 29-38.
21. Lavori, P. and Dawson, R. (2004). Dynamic treatment regimes: practical design considerations. Clinical Trials 1, 9-20.
22. Lee, S. (1999). On a class of m out of n bootstrap confidence intervals. Journal of the Royal Statistical Society, Series B 61, 901-911.
23. Moodie, E. and Richardson, T. (2010). Estimating optimal dynamic regimes: correcting bias under the null. Scandinavian Journal of Statistics 37, 126-146.
24. Murphy, S. (2003). Optimal dynamic treatment regimes (with discussions). Journal of the Royal Statistical Society, Series B 65, 331-366.
25. Murphy, S. (2005a). An experimental design for the development of adaptive treatment strategies. Statistics in Medicine 24, 1455-1481.
26. Murphy, S. A. (2005b). A generalization error for q-learning. Journal of Machine Learning Research 6, 1073-1097.
27. Pineau, J., Bellernare, M., Rush, A., Ghizaru, A., and Murphy, S. (2007). Constructing evidence-based treatment strategies using methods from computer science. Drug and Alcohol Dependence 88, S52-S60.
28. Politis, D., Romano, J., and Wolf, M. (1999). Subsampling. Springer Verlag, New York.
29. Pötscher, B. (2007). Confidence sets based on sparse estimators are necessarily large. Arxiv preprint arXiv:0711.1036.
30. Pötscher, B. M. and Schneider, U. (2008). Confidence sets based on penalized maximum likelihood estimators. Mpra paper, University Library of Munich, Germany.
31. Robins J. M. (2004). Optimal structural nested models for optimal sequential decisions. In Lin, D. Y. and Heagerty, P., editors, Proceedings of the Second Seattle Symposium on Biostatistics, pp. 189-326, New York, Springer.
32. Rush A.J., Trivedi M.H., Ibrahim H.M., Carmody T.J., Arnow B., Klein D.N., Markowitz J.C., Ninan P.T., Kornstein S., Manber R., Thase M.E., Kocsis J.H., and Keller M.B. (2003). The 16-Item quick inventory of depressive symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biological Psychiatry 54, 573-583.
33. Rush, A., Fava, M., Wisniewski, S., Lavori, P., Trivedi, M., Sackeim, H., Thase, M., Nierenberg, A., Quitkin, F., Kashner, T., Kupfer, D., Rosenbaum, J., Alpert, J., Stewart, J., McGrath, P., Biggs, M., Shores-Wilson, K., Lebowitz, B., Ritz, L., and Niederehe, G. (2004). Sequenced treatment alternatives to relieve depression (stard): rationale and design. Controlled Clinical Trials 25, 119-142.
34. Schulte, P.J., Tsiatis, A.A., Laber, E.B., and Davidian, M. (2012). Q- and a-learning methods for estimating optimal dynamic treatment regimes. Arxiv preprint arXiv:1202.4177.
35. Shao, J. (1994). Bootstrap sample size in nonregular cases. Proceedings of the American Mathematical Society 122, 1251-1262.
36. Shao, J. and Wu, C. (1989). A general theory for jackknife variance estimation. The Annals of Statisticspages 1176-1197.
37. Shortreed, S., Laber, E., Lizotte, D., Stroup, T., Pineau, J., and Murphy, S. (2011). Informing sequential clinical decision-making through reinforcement learning: an empirical study. Machine Learning 84, 109-136.
38. Song, R., Wang, W., Zeng, D., and Kosorok, M. (2012). Penalized q-learning for dynamic treatment regimes. arXiv:1108.5338v1 [stat.ME].
39. Sutton, R. and Barto, A. (1998). Reinforcement learning: An introduction. MIT Press, Cambridge.
40. Swanepoel, J. (1986). A note on proving that the (modified) bootstrap works. Communications in Statistics-Theory and Methods 15, 3193-3203.
41. Watkins, C. (1989). Learning from delayed rewards. Ph.D. Thesis, Cambridge University.
42. Zhao, Y., Kosorok, M., and Zeng, D. (2009) Reinforcement learning design for cancer clinical trials. Statistics in Medicine 28, 3294-3315.