Abstraction and approximate decision-theoretic planning

Artificial Intelligence

Volumn 89, Issue 1-2, 1997, Pages 219-283

  Dearden, Richard ^a   Boutilier, Craig ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Abstraction; Approximation; Decision theory; Execution; Heuristics; Markov decision processes; Planning; Search

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; DECISION THEORY; HEURISTIC METHODS; MARKOV PROCESSES; OPTIMIZATION;

DECISION-THEORETIC PLANNING;

PLANNING;

EID: 0030697013     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/s0004-3702(96)00023-9     Document Type: Article

References (56)

1. B.W. Ballard, The *-minimax search procedure for trees containing chance nodes, Artif. Intell. 21 (1983) 327-350.
2. A.G. Barto, S.J. Bradtke and S.P. Singh, Learning to act using real-time dynamic programming, Artif. Intell. 72 (1995) 81-138.
3. R.E. Bellman, Dynamic Programming (Princeton University Press, Princeton, NJ, 1957).
4. D.P. Bertsekas and D.A. Castanon, Adaptive aggregation for infinite horizon dynamic programming, IEEE Trans. Automat. Control 34 (1989) 589-598.
5. D.P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models (Prentice-Hall, Englewood Cliffs, NJ, 1987).
6. M. Boddy and T.L. Dean, Solving time-dependent planning problems, in: Proceedings IJCAI-89, Detroit, MI (1989) 979-984.
7. M. Boddy and T.L. Dean, Deliberation scheduling for problem solving in time-constrained environments, Artif. Intell. 67 (1994) 245-285.
8. C. Boutilier, T.L. Dean and S. Hanks, Planning under uncertainty: structural assumptions and computational leverage, in: Proceedings Third European Workshop on Planning, Assisi (1995).
9. C. Boutilier and R. Dearden, Approximating value trees in structured dynamic programming, in: Proceedings Thirteenth International Conference on Machine Learning, Bari (1996) 54-62.
10. C. Boutilier, R. Dearden and M. Goldszmidt, Exploiting structure in policy construction, in: Proceedings IJCAI-95, Montreal, Que. (1995) 1104-1111.
11. C. Boutilier and D. Poole, Computing optimal policies for partially observable decision processes using compact representations, in: Proceedings AAAI-96, Portland, OR (1996) 1168-1175.
12. C. Boutilier and M.L. Puterman, Process-oriented planning and average-reward optimality, in: Proceedings IJCAI-95, Montreal, Que. (1995) 1096-1103.
13. A.R. Cassandra, L.P. Kaelbling and M.L. Littman, Acting optimally in partially observable stochastic domains, in: Proceedings AAAI-94, Seattle, WA (1994) 1023-1028.
14. D. Chapman and L.P. Kaelbling, Input generalization in delayed reinforcement learning: an algorithm and performance comparisons, in: Proceedings IJCAI-91, Sydney (1991) 726-731.
15. T.L. Dean, L.P. Kaelbling, J. Kirman and A. Nicholson, Planning with deadlines in stochastic domains, in: Proceedings AAAI-93, Washington, DC (1993) 574-579.
16. T.L. Dean and K. Kanazawa, A model for reasoning about persistence and causation, Comput. Intell. 5 (1989) 142-150.
17. D. Draper, S. Hanks and D.S. Weld, A probabilistic model of action for least-commitment planning with information gathering, in: Proceedings Tenth Conference on Uncertainty in Artificial Intelligence, Seattle, WA (1994) 178-186.
18. M. Drummond, K. Swanson, J. Bresina and R. Levinson, Reaction-first search, in: Proceedings IJCAI-93, Chambery (1993) 1408-1414.
19. R.E. Fikes and N.J. Nilsson, Strips: a new approach to the application of theorem proving to problem solving, Artif. Intell. 2 (1971) 189-208.
20. S. French, Decision Theory (Halsted Press, New York, 1986).
21. D. Geiger and D. Heckerman, Advances in probabilistic reasoning, in: Proceedings Seventh Conference on Uncertainty in Artificial Intelligence, Los Angeles, CA (1991) 118-126.
22. P. Haddawy and A. Doan, Abstracting probabilistic actions, in: Proceedings Tenth Conference on Uncertainty in Artificial Intelligence, Seattle, WA (1994) 270-271.
23. E.J. Horvitz, Computation and action under bounded resources, Tech. Rept. KSL-90-76, Stanford University, Stanford, CA (1990).
24. E.J. Horvitz and A.C. Klein, Utility-based abstraction and categorization, in: Proceedings Ninth Conference on Uncertainty in Artificial Intelligence, Washington, DC (1993) 128-135.
25. R.A. Howard, Dynamic Programming and Markov Processes (MIT Press, Cambridge, MA, 1960).
26. R.A. Howard, Dynamic Probabilistic Systems (Wiley, New York, 1971).
27. R.A. Howard and J.E. Matheson, eds., Readings on the Principles and Applications of Decision Analysis (Strategic Decision Group, Menlo Park, CA, 1984).
28. R.L. Keeney and H. Raiffa, Decisions with Multiple Objectives: Preferences and Value Trade-offs (Wiley, New York, 1978).
29. C.A. Knoblock, Automatically generating abstractions for planning, Artif. Intell. 68 (1994) 243-302.
30. C.A. Knoblock, J.D. Tenenberg and Q. Yang, Characterizing abstraction hierarchies for planning, in: Proceedings AAAI-91, Anaheim, CA (1991) 692-697.
31. R.E. Korf, Real-time heuristic search, Artif. Intell. 42 (1990) 189-211.
32. N. Kushmerick, S. Hanks and D.S. Weld, An algorithm for probabilistic least-commitment planning, in: Proceedings AAAI-94, Seattle, WA (1994) 1073-1078.
33. M.L. Littman, A.R. Cassandra and L.P. Kaelbling, Learning policies for partially observable environments: scaling up, in: Proceedings Twelfth International Conference on Machine Learning, Lake Tahoe (1995) 362-370.
34. M.L. Littman, T.L. Dean and L.P. Kaelbling, On the complexity of solving Markov decision problems, in: Proceedings Eleventh Conference on Uncertainty in Artificial Intelligence, Montreal, Que. (1995) 394-402.
35. W.S. Lovejoy, A survey of algorithmic methods for partially observed Markov decision processes, Ann. Oper. Res. 28 (1991) 47-66.
36. D. McAllester and D. Rosenblitt, Systematic nonlinear planning, in: Proceedings AAAI-91, Anaheim, CA (1991) 634-639.
37. A.W. Moore and C.G. Atkeson, The parti-game algorithm for variable resolution reinforcement learning in multidimensional state spaces, Mach. Learn. (to appear).
38. A.E. Nicholson and L.P. Kaelbling, Toward approximate planning in very large stochastic domains, in: AAAI Spring Symposium on Decision Theoretic Planning, Stanford, CA (1994) 190-196.
39. R. Parr and S.J. Russell, Approximating optimal policies for partially observable stochastic domains, in: Proceedings IJCAI-95, Montreal, Que. (1995) 1088-1094.
40. J. Pearl, Heuristics: Intelligent Search Strategies for Computer Problem Solving (Addison-Wesley, Reading, MA, 1984).
41. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann, San Mateo, CA, 1988).
42. E. Pednault, ADL: exploring the middle ground between STRIPS and the situation calculus, in: Proceedings First International Conference on Principles of Knowledge Representation and Reasoning, Toronto, Ont. (1989) 324-332.
43. M.A. Peot and D.E. Smith, Conditional nonlinear planning, in: Proceedings First International Conference on AI Planning Systems, College Park, MD (1992) 189-197.
44. D. Poole, Exploiting the rule structure for decision making within the independent choice logic, in: Proceedings Eleventh Conference on Uncertainty in Artificial Intelligence, Montreal, Que. (1995) 454-463.
45. M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming (Wiley, New York, 1994).
46. M.L. Puterman and M.C. Shin, Modified policy iteration algorithms for discounted Markov decision problems, Manage. Sci. 24 (1978) 1127-1137.
47. S.J. Russell and E. Wefald, Do the Right Thing: Studies in Limited Rationality (MIT Press, Cambridge, MA, 1991).
48. S.J. Russell and S. Zilberstein, Composing real-time systems, in: Proceedings IJCAI-91, Sydney (1991) 212-217.
49. E.D. Sacerdoti, Planning in a hierarchy of abstraction spaces, Artif. Intell. 5 (1974) 115-135.
50. E.D. Sacerdoti, The nonlinear nature of plans, in: Proceedings IJCAI-75, Tblisi (1975) 206-214.
51. M.J. Schoppers, Universal plans for reactive robots in unpredictable environments, in: Proceedings IJCAI-87, Milan (1987) 1039-1046.
52. P.L. Schweitzer, M.L. Puterman and K.W. Kindle, Iterative aggregation-disaggregation procedures for discounted semi-Markov reward processes, Oper. Res. 33 (1985) 589-605.
53. S.P. Singh, T. Jaakkola and M.I. Jordan, Reinforcement learning with soft state aggregation, in: S.J. Hanson, J.D. Cowan and C.L. Giles, eds., Advances in Neural Information Processing Systems 7 (Morgan Kaufmann, San Mateo, CA, 1994).
54. R.D. Smallwood and E.J. Sondik, The optimal control of partially observable Markov processes over a finite horizon, Oper. Res. 21 (1973) 1071-1088.
55. D.E. Smith and M.A. Peot, Postponing threats in partial-order planning, in: Proceedings AAAI-93, Washington, DC (1993) 500-506.
56. J. Tash and S.J. Russell, Control strategies for a stochastic planner, in: Proceedings AAAI-94, Seattle, WA (1994) 1079-1085.