The linear programming approach to approximate dynamic programming

Operations Research

Volumn 51, Issue 6, 2003, Pages 850-865

  De Farias, D P ^a   Van Roy, B ^b  

^a Massachusetts Institute of Technology Cambridge   (United States)

^b Stanford University   (United States)

Author keywords

Dynamic programming optimal control: approximations large scale problems; Queues, algorithms: control of queueing networks

Indexed keywords

LARGE-SCALE PROBLEMS; QUEUES; STOCHASTIC MODELS;

ALGORITHMS; APPROXIMATION THEORY; COSTS; DYNAMIC PROGRAMMING; OPTIMAL CONTROL SYSTEMS; PARAMETER ESTIMATION; QUEUEING NETWORKS; STATE SPACE METHODS; STOCHASTIC CONTROL SYSTEMS; VECTORS;

LINEAR PROGRAMMING;

EID: 0348090400     PISSN: 0030364X     EISSN: None     Source Type: Journal    
DOI: 10.1287/opre.51.6.850.24925     Document Type: Article

References (37)

1. Bertsekas, D. 1995. Dynamic Programming and Optimal Control. Athena Scientific, Belmont, MA.
2. _, J. N. Tsitsiklis. 1996. Neuro-Dynamic Programming. Athena Scientific, Belmont, MA.
3. _, D. Gamarnik, J. Tsitsiklis. 2001. Performance of multiclass Markovian queueing networks via piecewise linear Lyapunov functions. Ann. Appl. Probab. 11(4) 1384-1428.
4. Bishop, C. M. 1995. Neural Networks for Pattern Recognition. Oxford University Press, New York.
5. Borkar, V. 1988. A convex analytic approach to Markov decision processes. Probab. Theory Related Fields 78 583-602.
6. Chen, R.-R., S. Meyn. 1999. Value iteration and optimization of multiclass queueing networks. Queueing Systems 32 65-97.
7. Chen, V. C. P., D. Ruppert, C. A. Shoemaker. 1999. Applying experimental design and regression splines to high-dimensional continuous-state stochastic dynamic programming. Oper. Res. 47(1) 38-53.
8. Crites, R. H., A. G. Barto. 1996. Improving elevator performance using reinforcement learning. Advances in Neural Information Processing Systems, Vol. 8. MIT Press, Cambridge, MA, 1017-1023.
9. Dayan, P. 1992. The convergence of TD(λ) for general λ. Machine Learning 8 341-362.
10. de Farias, D. P., B. Van Roy. 2000. On the existence of fixed points for appproximate value iteration and temporal-difference learning. J. Optim. Theory Appl. 105(3) 589-608.
11. _, _. 2001. On constraint sampling in the linear programming approach to approximate dynamic programming. Conditionally accepted to Math. Oper. Res.
12. De Ghellinck, G. 1960. Les problèmes de décisions séquentielles. Cahiers du Centre d'Etudes de Recherche Opérationnelle 2 161-179.
13. Denardo, E. V. 1970. On linear programming in a Markov decision problem. Management Sci. 16(5) 282-288.
14. D'Epenoux, F. 1963. A probabilistic production and inventory problem. Management Sci. 10(1) 98-108.
15. Gordon, G. 1999. Approximate solutions to Markov decision processess, Ph.D. thesis, Carnegie Mellon University, Pittsburgh, PA.
16. Grötschel, M., O. Holland. 1991. Solution of large-scale symmetric travelling salesman problems. Math. Programming 51 141-202.
17. Guestrin, C., D. Koller, R. Parr. 2002. Efficient solution algorithms for factored MDPs. J. Artificial Intelligence Res. Forthcoming.
18. Haykin, S. 1994. Neural Networks: A Comprehensive Formulation. Macmillan, New York.
19. Hordijk, A., L. C. M. Kallenberg. 1979. Linear programming and Markov decision chains. Management Sci. 25 352-362.
20. Kumar, P. R., T. I. Seidman. 1990. Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems. IEEE Trans. Automatic Control 35(3) 289-298.
21. Longstaff, F., E. S. Schwartz. 2001. Valuing American options by simulation: A simple least squares approach. Rev. Financial Stud. 14 113-147.
22. Manne, A. S. 1960. Linear programming and sequential decisions. Management Sci. 6(3) 259-267.
23. Morrison, J. R., P. R. Kumar. 1999. New linear program performance bounds for queueing networks. J. Optim. Theory Appl. 100(3) 575-597.
24. Paschalidis, I. C., J. N. Tsitsiklis. 2000. Congestion-dependent pricing of network services. IEEE/ACM Trans. Networking 8(2) 171-184.
25. Rybko, A. N., A. L. Stolyar. 1992. On the ergodicity of stochastic processes describing the operation of open queueing networks. Problemy Peredachi Informatsii 28 3-26.
26. Schuurmans, D., R. Patrascu. 2001. Direct value-approximation for factored MDPs. Advances in Neural Information Processing Systems, Vol. 14. MIT Press, Cambridge, MA, 1579-1586.
27. Schweitzer, P., A. Seidmann. 1985. Generalized polynomial approximations in Markovian decision processes. J. Math. Anal. Appl. 110 568-582.
28. Sutton, R. S. 1988. Learning to predict by the methods of temporal differences. Machine Learning 3 9-44.
29. _, A. G. Barto. 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA.
30. Tesauro, C. J. 1995. Temporal difference learning and TD-gammon. Comm. ACM 38 58-68.
31. Trick, M., S. Zin. 1993. A linear programming approach to solving dynamic programs. Unpublished manuscript.
32. _, _. 1997. Spline approximations to value functions: A linear programming approach. Macroeconomic Dynamics 1 255-277.
33. Tsitsiklis, J. N., B. Van Roy. 1997. An analysis of temporal-difference learning with function approximation. IEEE Trans. Auto. Control 42(5) 674-690.
34. _, _. 2001. Regression methods for pricing complex American-style options. IEEE Trans. Neural Networks 12(4) 694-703.
35. Van Roy, B. 1998. Learning and value function approximation in complex decision processes. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA.
36. _. 2000. Neuro-dynamic programming: Overview and recent trends. E. Feinberg, A. Schwartz, eds. Markov Decision Processes: Models, Methods, Directions, and Open Problems. Kluwer, Norwell, MA.
37. Zhang, W., T. G. Dietterich. 1996. High-performance job-shop scheduling with a time-delay TD(λ) network. Advances in Neural Information Processing Systems, Vol. 8. MIT Press, Cambridge, MA, 1024-1030.