Penalized Q-learning for dynamic treatment regimens

Statistica Sinica

Volumn 25, Issue 3, 2015, Pages 901-920

  Song, Rui ^a   Wang, Weiwei ^b   Zeng, Donglin ^c   Kosorok, Michael R ^c  

^a NORTH CAROLINA STATE UNIVERSITY   (United States)

^b Two Sigma Investments   (United States)

^c University of North Carolina at Chapel Hill   (United States)

Author keywords

Dynamic treatment regimen; Individual selection; Multistage; Penalized Q learning; Q learning; Shrinkage; Two stage procedure

Indexed keywords

EID: 84990041926     PISSN: 10170405     EISSN: None     Source Type: Journal    
DOI: 10.5705/ss.2012.364     Document Type: Article

References (25)

1. Candes, E. and Tao, T. (2007). The dantzig selector: Statistical estimation when p is much larger than n (with discussion). Ann. Statist. 35, 2313-2404.
2. Chakraborty, B., Murphy, S. and Strecher, V. (2010). Inference for non-regular parameters in optimal dynamic treatment regimes. Statist. Meth. Medical Res. 19, 317-343.
3. Chakraborty, B., Laber, E. and Zhao, Y. (2013). Inference for optimal dynamic treatment regimes using an adaptive m-out-of-n bootstrap scheme. Biometrics 69, 714-723.
4. Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96, 1348-1360.
5. Fava, M., Rush, A., Trivedi, M., Nierenberg, A., Thase, M., Sackeim, H., Quitkin, F., Wisniewski, S., Lavori, P., Rosenbaum, J., Kupfer, D. and STAR D Invest Grp (2003). Background and rationale for the Sequenced Treatment Alternatives to Relieve Depression (STAR∗D) study. Psychiatric Clinics of North America 26, 457-494.
6. Frank, I. E. and Friedman, J. H. (1993). A statistical view of some chemometrics regression tools (with discussion). Technometrics 35, 109-148.
7. Hirano, K. and Porter, J. R. (To appear). Impossibility results for nondifferentiable functionals. Econometrica.
8. Kaelbling, P. L., M., L. and Moore, A. (1996). Reinforcement learning: A survey. J. Artificial Intelligence Research 4, 237-285.
9. Lavori, P.W. and Dawson, A. (2000). A design for testing clinical strategies: Biased adaptive withinsubject randomization. J. Roy. Statist. Soc. A 163, 29-38.
10. Lunceford, J., Davidian, M. and Tsiatis, A. (2002). Estimation of survival distributions of treatment policies in two-stage randomization designs in clinical trials. Biometrics 58, 48-57.
11. Moodie, E. E. M., Platt, R. W. and Kramer, M. S. (2009). Estimating response-maximized decision rules with applications to breastfeeding. J. Amer. Statist. Assoc. 104, 155-165.
12. Moodie, E. E. M. and Richardson, T. S. (2010). Estimating optimal dynamic regimes: Correcting bias under the null. Scand. J. Statist. 37, 126-146.
13. Murphy, S. (2003). Optimal dynamic treatment regimes. J. Roy. Statist. Soc. Ser. B 65, 331-355.
14. Murphy, S. (2005). An experimental design for the development of adaptive treatment strategies. Statist. Medicine 24, 1455-1481.
15. Robins, J. M. (2004). Optimal structural nested models for optimal sequential decisions. In Proceedings of the Second Seattle Symposium on Biostatistics. Springer.
16. Thall, P., Millikan, R. and Sung, H. (2000). Evaluating multiple treatment courses in clinical trials. Statist. Medicine 19, 1011-1028.
17. Thall, P., Sung, H. and Estey, E. (2002). Selecting therapeutic strategies based on efficacy and death in multicourse clinical trials. J. Amer. Statist. Assoc. 97, 29-39.
18. Thall, P. F.,Wooten, L. H., Logothetis, C. J., Millikan, R. E. and Tannir, N. M. (2007). Bayesian and frequentist two-stage treatment strategies based on sequential failure times subject to interval censoring. Statist. Medicine 26, 4687-4702.
19. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58, 267-288.
20. Wahed, A. and Tsiatis, A. (2006). Semiparametric efficient estimation of survival distributions in two-stage randomisation designs in clinical trials with censored data. Biometrika 93, 163-177.
21. Wahed, A. S. and Tsiatis, A. A. (2004). Optimal estimator for the survival distribution and related quantities for treatment policies in two-stage randomised designs in clinical trials. Biometrics 60, 124-133.
22. Zhao, Y., Kosorok, M. R. and Zeng, D. (2009). Reinforcement learning design for cancer clinical trials. Statist. Medicine 28, 3294-3315.
23. Zhao, Y., Zeng, D., Socinski, M. and Kosorok, M. (2011). Reinforcement learning strategies for clinical trials in non-small cell lung cancer. Biometrics 67, 1422-1433.
24. Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101, 1418-1429.
25. Zou, H. and Li, R. (2008). One-step sparse estimates in nonconcave penalized likelihood models. Ann. Statist. 36, 1509-1533.