Adaptive Importance Sampling with Automatic Model Selection in Value Function Approximation

Proceedings of the 23rd AAAI Conference on Artificial Intelligence, AAAI 2008

Volumn , Issue , 2008, Pages 1351-1356

  Hachiya, Hirotaka ^a   Akiyama, Takayuki ^a   Sugiyama, Masashi ^a   Peters, Jan ^b  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ECONOMIC AND SOCIAL EFFECTS; REINFORCEMENT LEARNING;

ADAPTIVE IMPORTANCE SAMPLING; AUTOMATIC MODEL SELECTION; DATA SAMPLE; DATA SAMPLING; ESSENTIAL PROBLEMS; REINFORCEMENT LEARNINGS; SAMPLING TECHNIQUE; TRADE OFF; VALUE FUNCTION APPROXIMATION; VALUE FUNCTIONS;

IMPORTANCE SAMPLING;

EID: 85167416290     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. Bagnell, J. A.; Kakade, S.; Ng, A. Y.; and Schneider, J. 2003. Policy search by dynamic programming. In NIPS 16.
2. Fishman, G. S. 1996. Monte Carlo: Concepts, Algorithms, and Applications. Berlin: Springer-Verlag.
3. Kakade, S. 2002. A natural policy gradient. In NIPS 14.
4. Lagoudakis, M. G., and Parr, R. 2003. Least-squares policy iteration. JMLR 4:1107–1149.
5. Peshkin, L. Shelton, C. 2002. Learning from scarce experience. In Proc. of ICML.
6. Precup, D.; Sutton, R. S.; and Dasgupta, S. 2001. Off-policy temporal-difference learning with function approximation. In Proc. of ICML.
7. Precup, D.; Sutton, R. S.; and Singh, S. 2000. Eligibility traces for off-policy policy evaluation. In Proc. of ICML.
8. Schoknecht, R. 2003. Optimality of reinforcement learning algorithms with linear function approximation. In NIPS 15.
9. Shelton, C. R. 2001. Policy improvement for pomdps using normalized importance sampling. In Proc. of UAI.
10. Shimodaira, H. 2000. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference 90(2):227–244.
11. Sugiyama, M.; Krauledat, M.; and Müller, K.-R. 2007. Co-variate shift adaptation by importance weighted cross validation. JMLR 8:985–1005.
12. Sutton, R. S., and Barto, A. G. 1998. Reinforcement Learning: An Introduction. The MIT Press.
13. Watkins, C. 1989. Learning from Delayed Rewards. Ph.D. Dissertation, King’s College, University of Oxford.