Finding the K best policies in a finite-horizon Markov decision process

European Journal of Operational Research

Volumn 175, Issue 2, 2006, Pages 1164-1179

  Nielsen, Lars Relund ^a   Kristensen, Anders Ringgaard ^a  

^a UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

Directed hypergraphs; Finite horizon Markov decision processes; Hyperpaths; K best policies; Stochastic dynamic programming

Indexed keywords

ALGORITHMS; DECISION MAKING; FINITE AUTOMATA; FUZZY SETS; OPTIMAL CONTROL SYSTEMS; STOCHASTIC PROGRAMMING;

DIRECTED HYPERGRAPHS; FINITE HORIZON MARKOV DECISION PROCESSES; HYPERPATHS; K BEST POLICIES; STOCHASTIC DYNAMIC PROGRAMMING;

MARKOV PROCESSES;

EID: 33748421291     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2005.06.011     Document Type: Article

References (28)

1. Altman E. Constrained Markov Decision Processes (1999), Chapman & Hall/CRC
2. Altman E., and Shwartz A. Adaptive control of constrained Markov chains: Criteria and policies. Annals of Operations Research 28 (1991) 101-134
3. Ausiello G., Datri A., and Sacca D. Minimal representation of directed hypergraphs. SIAM Journal on Computing 15 2 (1986) 418-431
4. Ausiello G., Franciosa P.G., and Frigioni D. Directed hypergraphs: Problems, algorithmic results, and a novel decremental approach. Lecture Notes in Computer Science 2202 (2001) 312-328
5. Giorgio Ausiello, Giuseppe F. Italiano, Umberto Nanni, Hypergraph traversal revisited: Cost measures and dynamic algorithms, in: Mathematical Foundations of Computer Science: 23rd International Symposium, Lecture Notes in Computer Science, vol. 1450, August 1998, p. 116. Available from: .
6. Bellman R. Dynamic Programming (1957), Princeton University Press
7. Berge C. Graphs and Hypergraphs (1973), North-Holland
8. Derman C., and Veinott Jr. A.F. Constrained Markov decision chains. Management Science 19 (1972) 389-390
9. Derman C., and Klein M. Some remarks on finite horizon Markovian decision models. Operations Research 13 (1965) 272-278
10. Gallo G., Longo G., Pallottino S., and Nguyen S. Directed hypergraphs and applications. Discrete Applied Mathematics 42 (1993) 177-201
11. Hordijk A., and Kallenberg L.C.M. Constrained undiscounted stochastic dynamic programming. Mathematics of Operations Research 9 (1984) 276-289
12. Howard R.A. Dynamic Programming and Markov Processes (1960), The M.I.T. Press, Cambridge, Massachusetts
13. Jeroslow R.G., Martin K., Rardin R.L., and Wang J. Gainfree Leontief substitution flow problems. Mathematical Programming 57 (1992) 375-414
14. L.C.M. Kallenberg, Linear programming and finite Markovian control problems, Mathematical Centre Tracts, Amsterdam, 1983, p. 148.
15. L.C.M. Kallenberg, Survey of linear programming for standard and nonstandard Markovian control problems, Part I: Theory. ZOR-Methods and Models in Operations Research, 40 (1994) 1-42.
16. Kristensen A.R. Hierarchic Markov processes and their applications in replacement models. European Journal of Operational Research 35 (1988) 207-215
17. Kristensen A.R. A general software system for Markov decision processes in herd management applications. Computers and Electronics in Agriculture 38 3 (2003) 199-215
18. Kristensen A.R., and Jørgensen E. Multi-level hierarchic Markov processes as a framework for herd management support. Annals of Operations Research 94 1 (2000) 69-90
19. S. Nguyen, S. Pallottino, Hyperpaths and shortest hyperpaths, in: Combinatorial optimization (Como, 1986), Lecture Notes in Math, vol. 1403, Springer, 1989, pp. 258-271.
20. L.R. Nielsen, Route choice in stochastic time-dependent networks, PhD thesis, Department of Operations Research, University of Aarhus, 2004.
21. Nielsen L.R., Andersen K.A., and Pretolani D. Bicriterion shortest hyperpaths in random time-dependent networks. IMA Journal of Management Mathematics 14 3 (2003) 271-303
22. Nielsen L.R., Andersen K.A., and Pretolani D. Finding the K shortest hyperpaths. Computers & Operations Research 32 6 (2005) 1477-1497
23. L.R. Nielsen, D. Pretolani, K.A. Andersen, Finding the K shortest hyperpaths using reoptimization, OR letters, in press, doi:10.1016/j.orl.2005.04.008.
24. L.R. Nielsen, D. Pretolani, K.A. Andersen, K shortest paths in stochastic time-dependent networks. Technical Report WP-L-2004-05, Department of Accounting, Finance and Logistics, Aarhus School of Business, 2004. Available from: .
25. D. Nilsson, Finding sets of most probable configurations in Bayesian networks and best strategies in infludence diagrams. PhD thesis, Department of Mathematics, Aalborg University, 1998.
26. Pretolani D. A directed hypergraph model for random time-dependent shortest paths. European Journal of Operational Research 123 (2000) 315-324
27. Puterman M.L. Markov Decision Processes. Wiley Series in Probability and Mathematical Statistics (1994), Wiley-Interscience
28. Ramalingam G., and Reps T. An incremental algorithm for a generalization of the shortest-path problem. Journal of Algorithms 21 2 (1996) 267-305