Approximate policy iteration with a policy language bias: Solving relational markov decision processes

Journal of Artificial Intelligence Research

Volumn 25, Issue , 2006, Pages 75-118

  Fern, Alan ^a   Yoon, Sungwook ^b   Givan, Robert ^b  

^a OREGON STATE UNIVERSITY   (United States)

^b PURDUE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; FUNCTIONS; LEARNING SYSTEMS; MARKOV PROCESSES; PLANNING; STOCHASTIC CONTROL SYSTEMS;

APPROXIMATE POLICY ITERATION (API); BOOTSTRAPPING; MARKOV DECISION PROCESSES (MDP); RANDOM WALKS;

ARTIFICIAL INTELLIGENCE;

EID: 33744466799     PISSN: 10769757     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.1700     Document Type: Article

References (51)

1. Aler, R., Borrajo, D., & Isasi, P. (2002). Using genetic programming to learn and improve control knowledge. Artificial Intelligence, 141(1-2), 29-56.
2. Ambite, J. L., Knoblock, C. A., & Minton, S. (2000). Learning plan rewriting rules. In Artificial Intelligence Planning Systems, pp. 3-12.
3. Bacchus, F. (2001). The AIPS '00 planning competition. AI Magazine, 22(3)(3), 57-62.
4. Bacchus, F., & Kabanza, F. (2000). Using temporal logics to express search control knowledge for planning. Artificial Intelligence, 16, 123-191.
5. Bagnell, J., Kakade, S., Ng, A., & Schneider, J. (2003). Policy search by dynamic programming. In Proceedings of the 16th Conference on Advances in Neural Information Processing.
6. Bellman, R. (1957). Dynamic Programming. Princeton University Press.
7. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-Dynamic Programming. Athena Scientific.
8. Boutilier, C., & Dearden, R. (1996). Approximating value trees in structured dynamic programming. In Saitta, L. (Ed.), International Conference on Machine Learning.
9. Boutilier, C., Dearden, R., & Goldszmidt, M. (2000). Stochastic dynamic programming with factored representations. Artificial Intelligence, 121(1-2), 49-107.
10. Boutilier, C., Reiter, R., & Price, B. (2001). Symbolic dynamic programming for first-order MDPs. In International Joint Conference on Artificial Intelligence.
11. Dean, T., & Givan, R. (1997). Model minimization in markov decision processes. In National Conference on Artificial Intelligence, pp. 106-111.
12. Dean, T., Givan, R., & Leach, S. (1997). Model reduction techniques for computing approximately optimal solutions for Markov decision processes. In Conference on Uncertainty in Artificial Intelligence, pp. 124-131.
13. Dietterich, T., & Wang, X. (2001). Batch value function approximation via support vectors. In Proceedings of the Conference on Advances in Neural Information Processing.
14. Dzeroski, S., DeRaedt, L., & Driessens, K. (2001). Relational reinforcement learning. Machine Learning, 43, 7-52.
15. Estlin, T. A., & Mooney, R. J. (1996). Multi-strategy learning of search control for partialorder planning. In National Conference on Artificial Intelligence.
16. Fern, A., Yoon, S., & Givan, R. (2003). Approximate policy iteration with a policy language bias. In Proceedings of the 16th Conference on Advances in Neural Information Processing.
17. Finkelstein, L., & Markovitch, S. (1998). A selective macro-learning algorithm and its application to the N×N sliding-tile puzzle. Journal of Artificial Intelligence Research, 8, 223-263.
18. Givan, R., Dean, T., & Greig, M. (2003). Equivalence notions and model minimization in Markov decision processes. Artificial Intelligence, 147(1-2), 163-223.
19. Gretton, C., & Thiebaux, S. (2004). Exploiting first-order regression in inductive policy selection. In Conference on Uncertainty in Artificial Intelligence.
20. Guestrin, C., Koller, D., Gearhart, C., & Kanodia, N. (2003a). Generalizing plans to new environments in relational mdps. In International Joint Conference on Artificial Intelligence.
21. Guestrin, C., Koller, D., Parr, R., & Venkataraman, S. (2003b). Efficient solution algorithms for factored mdps. Journal of Artificial Intelligence Research, 19, 399-468.
22. Hoffman, J., Porteous, J., & Sebastia, L. (2004). Ordered landmarks in planning. Journal of Artificial Intelligence Research, 22, 215-278.
23. Hoffmann, J., & Nebel, B. (2001). The FF planning system: Fast plan generation through heuristic search. Journal of Artificial Intelligence Research, 14, 263-302.
24. Howard, R. (1960). Dynamic Programming and Markov Decision Processes. MIT Press.
25. Huang, Y.-C., Selman, B., & Kautz, H. (2000). Learning declarative control rules for constraint-based planning. In International Conference on Machine Learning, pp. 415-422.
26. Kearns, M. J., Mansour, Y., & Ng, A. Y. (2002). A sparse sampling algorithm for near-optimal planning in large markov decision processes. Machine Learning, 45(2-3), 193-208.
27. Kersting, K., Van Otterlo, M., & DeRaedt, L. (2004). Bellman goes relational. In Proceedings of the Twenty-First International Conference on Machine Learning.
28. Khardon, R. (1999a). Learning action strategies for planning domains. Artificial Intelligence, 113(1-2), 125-148.
29. Khardon, R. (1999b). Learning to take actions. Machine Learning, 35(1), 57-90.
30. Lagoudakis, M., & Parr, R. (2003). Reinforcement learning as classification: Leveraging modern classifiers. In International Conference on Machine Learning.
31. Langford, J., & Zadrozny, B. (2004). Reducing t-step reinforcement learning to classification. http://hunch.net/~jl/projects/reductions/ RL_to_class/colt_submission.ps.
32. Martin, M., & Geffner, H. (2000). Learning generalized policies in planning domains using concept languages. In International Conference on Principles of Knowledge Representation and Reasoning.
33. Mataric, M. (1994). Reward functions for accelarated learning. In Proceedings of the International Conference on Machine Learning.
34. McAllester, D., & Givan, R. (1993). Taxonomic syntax for first order inference. Journal of the ACM, 40(2), 246-283.
35. McAllester, D. (1991). Observations on cognitive judgements. In National Conference on Artificial Intelligence.
36. McGovern, A., Moss, E., & Barto, A. (2002). Building a basic block instruction scheduler using reinforcement learning and rollouts. Machine Learning, 49(2/3), 141-160.
37. Minton, S. (1988). Quantitative results concerning the utility of explanation-based learning. In National Conference on Artificial Intelligence.
38. Minton, S. (Ed.). (1993). Machine Learning Methods for Planning. Morgan Kaufmann.
39. Minton, S., Carbonell, J., Knoblock, C. A., Kuokka, D. R., Etzioni, O., & Gil, Y. (1989). Explanation-based learning: A problem solving perspective. Artificial Intelligence, 40, 63-118.
40. Natarajan, B. K. (1989). On learning from exercises. In Annual Workshop on Computational Learning Theory.
41. Reddy, C., & Tadepalli, P. (1997). Learning goal-decomposition rules using exercises. In International Conference on Machine Learning, pp. 278-286. Morgan Kaufmann.
42. Rivest, R. (1987). Learning decision lists. Machine Learning, 2(3), 229-246.
43. Tesauro, G. (1992). Practical issues in temporal difference learning. Machine Learning, 8, 257-277.
44. Tesauro, G., & Galperin, G. (1996). On-line policy improvement using monte-carlo search. In Conference on Advances in Neural Information Processing.
45. Tsitsiklis, J., & Van Roy, B. (1996). Feature-based methods for large scale DP. Machine Learning, 22, 59-94.
46. Veloso, M., Carbonell, J., Perez, A., Borrajo, D., Fink, E., & Blythe, J. (1995). Integrating planning and learning: The PRODIGY architecture. Journal of Experimental and Theoretical AI, 7(1).
47. Wu, G., Chong, E., & Givan, R. (2001). Congestion control via online sampling. In Infocom.
48. Yan, X., Diaconis, P., Rusmevichientong, P., & Van Roy, B. (2004). Solitaire: Man versus machine. In Conference on Advances in Neural Information Processing.
49. Yoon, S., Fern, A., & Givan, R. (2002). Inductive policy selection for first-order MDPs. In Conference on Uncertainty in Artificial Intelligence.
50. Younes, H. (2003). Extending pddl to model stochastic decision processes. In Proceedings of the International Conference on Automated Planning and Scheduling Workshop on PDDL.
51. Zimmerman, T., & Kambhampati, S. (2003). Learning-assisted automated planning: Looking back, taking stock, going forward. AI Magazine, 24(2)(2), 73-96.