Reinforcement learning design for cancer clinical trials

Statistics in Medicine

Volumn 28, Issue 26, 2009, Pages 3294-3315

  Zhao, Yufan ^a   Kosorok, Michael R ^a   Zeng, Donglin ^a  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

Adaptive design; Clinical trials; Dynamic treatment regime; Extremely randomized trees; Multistage decision problems; Non small cell lung cancer; Optimal policy; Reinforcement learning; Support vector regression

Indexed keywords

ALGORITHM; ARTICLE; CANCER CHEMOTHERAPY; CANCER RESEARCH; CLINICAL TRIAL; CYSTIC FIBROSIS; INTERMETHOD COMPARISON; LUNG NON SMALL CELL CANCER; MACHINE LEARNING; MATHEMATICAL MODEL; MONTE CARLO METHOD; PROBABILITY; REINFORCEMENT LEARNING; SIMULATION; SUPPORT VECTOR MACHINE; SYSTEM ANALYSIS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIOSTATISTICS; CARCINOMA, NON-SMALL-CELL LUNG; CLINICAL TRIALS AS TOPIC; HUMANS; LUNG NEOPLASMS; MODELS, STATISTICAL; NEOPLASMS; REGRESSION ANALYSIS; REINFORCEMENT (PSYCHOLOGY); STATISTICS, NONPARAMETRIC;

EID: 70449449564     PISSN: 02776715     EISSN: 10970258     Source Type: Journal    
DOI: 10.1002/sim.3720     Document Type: Article

References (42)

1. Hogberg T. Widening bottlenecks in drug discovery Glimpses from Drug Discovery Technology Europe. Drug Discovery Today 2005; 10(12):820-822. (Pubitemid 40824736)
2. Food and Drug Administration. Innovation or stagnation: challenge and opportunity on the critical path to new medical products. White Paper, 2004.
3. Stinchcombe TE, Socinski MA. Considerations for second-line therapy of non-small cell lung cancer. The Oncologist 2008; 13(suppl. 1):28-36.
4. Thall PF, Millikan RE, Sung HG. Evaluating multiple treatment courses in clinical trials. Statistics in Medicine 2000; 19:1011-1028. (Pubitemid 30238912)
5. Thall PF, Wooten LH, Logothetis CJ, Millikan RE, Tannir NM. Bayesian and frequentist two-stage treatment strategies based on sequential failure times subject to interval censoring. Statistics in Medicine 2007; 26: 4687-4702.
6. Murphy SA. An experimental design for the development of adaptive treatment strategies. Statistics in Medicine 2005; 24:1455-1481. (Pubitemid 40716347)
7. Murphy SA, Lynch KG, Oslin D, McKay JR, TenHave T. Developing adaptive treatment strategies in substance abuse research. Drug and Alcohol Dependence 2007; 88S:S24-S30. (Pubitemid 46541624)
8. Pineau J, Bellemare MG, Rush AJ, Ghizaru A, Murphy SA. Constructing evidence-based treatment strategies using methods from computer science. Drug and Alcohol Dependence 2007; 88S:S52-S60. (Pubitemid 46546455)
9. Murphy SA, Oslin DW, Rush AJ, Zhu J. Methodological challenges in constructing effective treatment sequences for chronic psychiatric disorders. Neuropsychoarmacology 2007; 32:257-262. (Pubitemid 46106932)
10. Sahba F, Tizhoosh HR, Salama MM. Application of reinforcement learning for segmentation of transrectal ultrasound images. BMC Medical Imaging 2008; 8(8). (Pubitemid 351772686)
11. Sutton RS, Barto AG. Reinforcement Learning: An Introduction. MIT Press: Cambridge, MA, 1998.
12. Bellman RE. Dynamic Programming. Princeton University Press: Princeton, 1957.
13. Minsky ML. Steps toward artificial intelligence. Proceedings of the Institute of Radio Engineers 1961; 49:8-30.
14. Howard R. Dynamic Programming and Markov Processes. MIT Press: Cambridge, MA, 1960.
15. Puterman ML, Shin MC. Modified policy iteration algorithms for discounted Markov decision problems. Management Science 1978; 24:1127-1137.
16. Bertsekas DP. Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall: Englewood Cliffs, 1987.
17. Sutton RS. Learning to predict by the method of temporal differences. Machine Learning 1988; 3(1):9-44.
18. Kaelbling LP, Littman ML, Moore AW. Reinforcement learning: a survey. Journal of Artificial Intelligence Research 1996; 4:237-285.
19. Watkins CJCH. Learning from delayed rewards. Ph.D. Thesis, King's College, Cambridge, U.K., 1989.
20. Watkins CJCH, Dayan P. Q-learning. Machine Learning 1992; 8:279-292.
21. Murphy SA. A generalization error for Q-learning. Journal of Machine Learning Research 2005; 6:1073-1097.
22. Tsitsiklis JN, Van Roy B. Feature-based methods for large scale dynamic programming. Machine Learning 1996; 22:59-94. (Pubitemid 126724363)
23. Vapnik V, Golowich S, Smola A. Support vector method for function approximation, regression estimation, and signal processing. Advances in Neural Information Processing Systems 1997; 9:281-287.
24. Vapnik V. The Nature of Statistical Learning Theory. Springer: New York, 1995.
25. Cover TM. Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Transactions on Electronic Computers 1965; 14:326-334.
26. Cortes C, Vapnik V. Support-vector networks. Machine Learning 1995; 20:273-297.
27. Chen BJ, Chang MW, Lin CJ. Load forecasting using support vector machines: a study on EUNITE competition 2001. IEEE Transactions on Power Systems 2004; 19:1821-1830.
28. Smola A, Scholkopf B. A tutorial on support vector regression. Statistics and Computing 2004; 14:199-222.
29. LIBSVM: a library for support vector machines (version 2.31). Available from: http://www.csie.ntu.edu.tw/~cjlin/libsvm [7 September 2001].
30. Ernst D, Geurts P, Wehenkel L. Tree-based batch model reinforcement learning. Journal of Machine Learning Research 2005; 6:503-556. (Pubitemid 40958851)
31. Geurts P, Ernst D, Wehenkel L. Extremely randomized trees. Machine Learning 2006; 11:3-42.
32. Ernst D, Stan GB, Goncalves J, Wehenkel L. Clinical data based optimal STI strategies for HIV: a reinforcement learning approach. Proceedings of the 45th IEEE Conference on Decision and Control, San Diego, U.S.A., 2008; 1671-1678. (Pubitemid 351283311)
33. Guez A, Vincent R, Avoli M, Pineau J. Adaptive treatment of Epilepsy via batch-mode reinforcement learning. Innovative Applications of Artificial Intelligence 2008.
34. Adams B, Banks H, Kwon HD, Tran H. Dynamic multidrug therapies for HIV: optimal and STI control approaches. Mathematical Biosciences and Engineering 2004; 1:223-241.
35. Shepherd FA, Pereira JR, Ciuleanu T, Tan EH, Hirsh V, Thongprasert S, Campos D, Maoleekoonpiroj S, Smylie M, Martins R, van Kooten M, Dediu M, Findlay B, Tu D, Johnston D, Bezjak A, Clark G, Santabarbara P, Seymo L. Erlotinib in previously treated non-small-cell lung cancer. The New England Journal of Medicine 2005; 353:123-132. (Pubitemid 41058344)
36. Scagliotti GV, Parikh P, Von Pawel J, Biesma B, Vansteenkiste J, Manegold C, Serwatowski P, Gatzemeier U, Digumarti R, Zukin M, Lee JS, Mellemgaard A, Park K, Patil S, Rolski J, Goksel T, de Marinis F, Simms L, Sugarman KP, Gandara D. Phase III study comparing cisplatin plus gemcitabine with cisplatin plus pemetrexed in chemotherapy-naive patients with advanced-stage non-small-cell lung cancer. Journal of Clinical Oncology 2008; 26:3543-3551.
37. Sandler A, Gray R, Perry MC, Brahmer J, Schiller JH, Dowlati A, Lilenbaum R, Johnson DH. Paclitaxel-Carboplatin alone or with bevacizumab for non-small-cell lung cancer. The New England Journal of Medicine 2006; 355:2542-2550. (Pubitemid 44917502)
38. Pirker R, Szczesna A, Von Pawel J, Krzakowski M, Ramlau R, Park K, Gatzemeier U, Bajeta E, Emig M, Pereira JR. FLEX: a randomized, multicenter, phase III study of cetuximab in combination with cisplatin/vinorelbine (CV) versus CV alone in the first-line treatment of patients with advanced non-small cell lung cancer (NSCLC). Journal of Clinical Oncology 2008; abstr 3.
39. Shepherd FA, Dancey J, Ramlau R, Mattson K, Gralla R, O'Rourke M, Levitan N, Gressot L, Vincent M, Burkes R, Coughlin S, Kim Y, Berille J. Prospective randomized trial of docetaxel versus best supportive care in patients with non-small-cell lung cancer previously treated with platinum-based chemotherapy. Journal of Clinical Oncology 2000; 18:2095-2103. (Pubitemid 30324363)
40. Hanna N, Shepherd FA, Fossella FV, Pereira JR, de Marinis F, Von Pawel J, Gatzemeier U, Chang T, Tsao Y, Pless M, Muller T, Lim HL, Desch C, Szondy K, Gervais R, Shaharyar, Manegold C, Paul S, Paoletti P, Einhorn L, Bunn PA. Randomized phase III trial of pemetrexed versus docetaxel in patients with non-small-cell lung cancer previously treated with chemotherapy. Journal of Clinical Oncology 2004; 22:1589-1597. (Pubitemid 41079796)
41. Fidias P, Dakhil S, Lyss A, Loesch D, Waterhouse D, Cunneen J, Chen R, Treat J, Obasaju C, Schiller J. Phase III study of immediate versus delayed docetaxel after induction therapy with gemcitabine plus carboplatin in advanced non-small-cell lung cancer: updated report with survival. ASCO 2007, vol.25, 20 June suppl, LBA7516.
42. Ciuleanu TE, Brodowicz T, Belani CP, Kim J, Krzakowski M, Laack E, Wu Y, Peterson P, Adachi S, Zielinski CC. Maintenance pemetrexed plus best supportive care (BSC) versus placebo plus BSC: a phase III study. ASCO 2008; abstr 8011.