Adaptive importance sampling with automatic model selection in value function approximation

Proceedings of the National Conference on Artificial Intelligence

Volumn 3, Issue , 2008, Pages 1351-1356

  Hachiya, Hirotaka ^a   Akiyama, Takayuki ^a   Sugiyama, Masashi ^a   Peters, Jan ^b  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BIONICS; COMMERCE; REINFORCEMENT;

ADAPTIVE IMPORTANCE SAMPLINGS; AUTOMATIC MODEL SELECTIONS; BIAS AND VARIANCES; CROSS VALIDATIONS; DATA SAMPLES; ESSENTIAL PROBLEMS; IMPORTANCE SAMPLINGS; VALUE FUNCTIONS;

ARTIFICIAL INTELLIGENCE;

EID: 57749096203     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. Bagnell, J. A.; Kakade, S.; Ng, A. Y.; and Schneider, J. 2003. Policy search by dynamic programming. In NIPS 16.
2. Fishman, G. S. 1996. Monte Carlo: Concepts, Algorithms, and Applications. Berlin: Springer- Verlag.
3. Kakade, S. 2002. A natural policy gradient. In NIPS 14.
4. Lagoudakis, M. G., and Parr, R. 2003. Least-squares policy iteration. JMLR 4:1107-1149.
5. Peshkin, L. Shelton, C. 2002. Learning from scarce experience. In Proc. of ICML.
6. Precup, D.; Sutton, R. S.; and Dasgupta, S. 2001. Off-policy temporal-difference learning with function approximation. In Proc. of ICML.
7. Precup, D.; Sutton, R. S.; and Singh, S. 2000. Eligibility traces for off-policy policy evaluation. In Proc. of ICML.
8. Schoknecht, R. 2003. Optimality of reinforcement learning algorithms with linear function approximation. In NIPS 15.
9. Shelton, C. R. 2001. Policy improvement for pomdps using normalized importance sampling. In Proc. of UAI.
10. Shimodaira, H. 2000. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference 90(2):227-244.
11. Sugiyama, M.; Krauledat, M.; and Müller, K.-R. 2007. Covariate shift adaptation by importance weighted cross validation. JMLR 8:985-1005.
12. Sutton, R. S., and Barto, A. G. 1998. Reinforcement Learning: An Introduction. The MIT Press.
13. Watkins, C. 1989. Learning from Delayed Rewards. Ph.D. Dissertation, King's College, University of Oxford.