Combinatorial design of a stochastic Markov decision process

Operations Research/ Computer Science Interfaces Series

Volumn 47, Issue , 2009, Pages 167-193

  Dimitrov, Nedialko B ^a   Morton, David P ^a  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Action space design; Markov decision process; Stochastic optimization

Indexed keywords

EID: 77955453073     PISSN: 1387666X     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-0-387-88843-9_9     Document Type: Article

References (31)

1. Adelman D (2007) Dynamic bid-prices in revenue management. Operations Research 55:647-661
2. Altman E, Shwartz A (1991) Sensitivity of constrained Markov decision problems. Annals of Operations Research 32:1-22
3. Beier JC, Killeen GF, Githure JI (1999) Entomological inoculation rates and plasmodium falciparum malaria prevalence in Africa. American Journal of Tropical Medicine and Hygiene 61:109-113
4. Bellman RE (1957) Dynamic Programming. Princeton University Press
5. Benders JF (1962) Partitioning procedures for solving mixed-variables programming problems. Numerische Mathematik 4:238-252
6. Bertsekas DP, Tsitsiklis JN (1991) An analysis of stochastic shortest path problems. Mathematics of Operations Research 16:580-595
7. Beutler FJ, Ross KW (1985) Optimal policies for controlled Markov chains with a constraint. Journal of Mathematical Analysis and Application 112:236-252
8. Blackwell D (1962) Discrete dynamic programming. Annals of Mathematical Statistics 33:719-726
9. Derman C (1962) On sequential decisions and Markov chains. Management Science 9:16-24
10. Dimitrov N, Moffett A, Morton DP, Sarkar S (2008) Markov decision processes for optimal malaria control, working paper, Graduate Program in Operations Research, The University of Texas at Austin
11. Eaton JH, Zadeh LA (1962) Optimal pursuit strategies in discrete state probabilistic systems. Transactions of the ASME, Series D, Journal of Basic Engineering 84:23-29
12. van Eijk AM, Ayisi JG, ter Kuile FO, Otieno JA, Misore AO, et al (2004) Effectiveness of intermittent preventive treatment with sulphadoxine-pyrimethamine for control of malaria in pregnancy in western Kenya: a hospital based study. Tropical Medicine and International Health 9:351-360
13. d'Epenoux F (1963) A probabilistic production and inventory problem. Management Science 10:98-108, english translation of an article published in Revue Francaise de Recherche Opérationnelle 14, 1960.
14. de Farias DP, Roy BV (2003) The linear programming approach to approximate dynamic programming. Operations Research 51:850-856
15. de Farias DP, Roy BV (2004) On constraint sampling for the linear programming approach to approximate dynamic programming. Mathematics of Operations Research 29:462-478
16. Feinberg EA, Shwartz A (1995) Constrained markov decision models with weighted discounted rewards. Mathematics of Operations Research 20:302-320
17. Guyatt HL, Corlett SK, Robinson TP, Ochola SA, Snow RW (2002) Malaria prevention in highland Kenya: indoor residual house-spraying vs. insecticide-treated bednets. Tropical Medicine and International Health 7:298-303
18. Howard RA (1960) Dynamic Programming and Markov Processes. MIT Press, Cambridge, MA
19. Kallenberg LCM (1983) Linear programming and finite Markovian control problems. Tech. Rep. 148, Mathematical Centre Tracts, Amsterdam
20. Kallenberg LCM (1994) Survey of linear programming for standard and nonstandard Markovian control problems. part i: Theory. Mathematical Methods of Operations Research 40:1-42
21. Kayentao K, Kodio M, Newman RD, Maiga H, Doumtabe D, et al (2004) Comparison of intermittent preventive treatment with chemoprophylaxis for the prevention of malaria during pregnancy in Mali. The Journal of Infectious Diseases 191:109-116
22. Kiszewski A, Johns B, Schapira A, Delacollette C, Crowell V, et al (2007) Estimated global resources needed to attain international malaria control goals. Bulletin of the World Health Organization 85:623-630
23. Manne AS (1960) Linear programming and sequential decisions. Management Science 6:259-267
24. Moffett A, Shackelford N, Sarkar S (2007) Malaria in Africa: Vector species niche models and relative risk maps. PLoS ONE 2:e824
25. Prékopa A (1995) Stochastic Programming. Kluwer Academic Publishers, Dordrecht
26. Puterman ML (2005) Markov Decision Processes: Discrete Dynamic Programming. John Wiley & Sons, Inc., Hoboken, New Jersey
27. Ross KW (1989) Randomized and past dependent policies for markov decision processes with finite action set. Operations Research 37:474-477
28. Schweitzer P, Seidmann A (1985) Generalized polynomial approximations in Markovian decision processes. Journal of Mathematical Analysis and Applications 110:568-582
29. Van Slyke RM, Wets R (1969) L-shaped linear programs with applications to optimal control and stochastic programming. SIAM Journal on Applied Mathematics 17:638-663
30. Veinott, Jr AF (1969) Discrete dynamic programming with sensitive discount optimality criteria. The Annals of Mathematical Statistics 40:1635-1660
31. Wagner HM (1960) On the optimality of pure strategies. Management Science 6:268-269