Lagrange dual decomposition for finite horizon Markov decision processes

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6911 LNAI, Issue PART 1, 2011, Pages 487-502

  Furmston, Thomas ^a   Barber, David ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Lagrange Duality; Markov Decision Processes; Planning

Indexed keywords

CONVERGENT ALGORITHMS; DUAL DECOMPOSITION; EMPIRICAL PERFORMANCE; EXPECTATION-MAXIMISATION; FINITE-HORIZON MARKOV DECISION PROCESS; HARD PROBLEMS; LAGRANGE DUAL; LAGRANGE DUALITY; MARKOV DECISION PROCESSES; NONSTATIONARY; PLANNING ALGORITHMS; POLICY GRADIENT; STATIONARY POLICY; SUB-PROBLEMS; LAGRANGE DUAL DECOMPOSITIONS;

LAGRANGE MULTIPLIERS; LEARNING SYSTEMS; MARKOV PROCESSES; DATA MINING; MACHINE LEARNING; PLANNING;

LEARNING ALGORITHMS; BEHAVIORAL RESEARCH;

EID: 80052418186     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-23780-5_41     Document Type: Conference Paper

References (20)

1. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
2. Vlassis, N.: A Concise Introduction to Multiagent Systems and Distributed Artificial Intelligence. Synthesis Lectures on Artificial Intelligence and Machine Learning 1(1), 1-71 (2007)
3. Bertsekas, D.P.: Dynamic Programming and Optimal Control, 2nd edn. Athena Scientific, Belmont (2000)
4. Shachter, R.D.: Probabilistic Inference and Influence Diagrams. Operations Research 36, 589-604 (1988)
5. Williams, R.: Simple Statistical Gradient Following Algorithms for Connectionist Reinforcement Learning. Machine Learning 8, 229-256 (1992)
6. Toussaint, M., Storkey, A., Harmeling, S.: Bayesian Time Series Models. In: Expectation-Maximization Methods for Solving (PO)MDPs and Optimal Control Problems, Cambridge University, Cambridge (in press 2011), userpage. fu-berlin.de/~mtoussai
7. Furmston, T., Barber, D.: Efficient Inference in Markov Control Problems. In: Uncertainty in Artificial Intelligence. North-Holland, Amsterdam (2011)
8. Furmston, T., Barber, D.: An analysis of the Expectation Maximisation algorithm for Markov Decision Processes. Research Report RN/11/13, Centre for Computational Statistics and Machine Learning, University College London (2011)
9. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)
10. Sontag, D., Globerson, A., Jaakkola, T.: Introduction to Dual Decomposition for Inference. In: Sra, S., Nowozin, S., Wright, S. (eds.) Optimisation for Machine Learning, MIT Press, Cambridge (2011)
11. Furmston, T., Barber, D.: Variational Methods for Reinforcement Learning. AISTATS 9(13), 241-248 (2010)
12. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)
13. Komodakis, N., Paragios, N., Tziritas, G.: MRF Optimization via Dual Decomposition: Message-Passing Revisited. In: IEEE 11th International Conference on Computer Vision, ICCV, pp. 1-8 (2007)
14. Dearden, R., Friedman, N., Russell, S.: Bayesian Q learning. AAAI 15, 761-768 (1998)
15. Sutton, R.: Generalization in Reinforcment Learning: Successful Examples Using Sparse Coarse Coding. NIPS (8), 1038-1044 (1996)
16. Hoffman, M., Doucet, A., De Freitas, N., Jasra, A.: Bayesian Policy Learning with Trans-Dimensional MCMC. NIPS (20), 665-672 (2008)
17. Hoffman, M., De Freitas, N., Doucet, A., Peters, J.: An Expectation Maximization Algorithm for Continuous Markov Decision Processes with Arbitrary Rewards. AISTATS 5(12), 232-239 (2009)
18. Salakhutdinov, R., Roweis, S., Ghahramani, Z.: Optimization with EM and Expectation-Conjugate-Gradient. ICML (20), 672-679 (2003)
19. Fraley, C.: On Computing the Largest Fraction of Missing Information for the EM Algorithm and the Worst Linear Function for Data Augmentation. Research Report EDI-INF-RR-0934, University OF Washington (1999)
20. Barber, D.: Bayesian Reasoning and Machine Learning. Cambridge University Press, Cambridge (2011)