Estimation of non-stationary Markov Chain transition models

Proceedings of the IEEE Conference on Decision and Control

Volumn , Issue , 2008, Pages 55-60

  Bertuccelli, L F ^a   How, J P ^a  

^a Massachusetts Institute of Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HIDDEN MARKOV MODELS; ONLINE SYSTEMS;

CHAIN TRANSITIONS; DIRICHLET DISTRIBUTIONS; ESTIMATION TECHNIQUES; MARKOV CHAIN MODELS; MAXIMUM A POSTERIORI; ON-LINE ESTIMATION; OPTIMAL PERFORMANCE; STATE OBSERVATION;

MARKOV CHAINS;

EID: 62949122724     PISSN: 07431546     EISSN: 25762370     Source Type: Conference Proceeding    
DOI: 10.1109/CDC.2008.4738904     Document Type: Conference Paper

References (19)

1. A. Barto, S. Bradtke, and S. Singh. Learning to Act using Real-Time Dynamic Programming. Artificial Intelligence, 72:81-138, 1993.
2. D. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, 2005.
3. L. F. Bertuccelli. Robust Decision-Making with Model Uncertainty in Aerospace Systems. PhD thesis, MIT, 2008.
4. L. F. Bertuccelli and J. P. How. Robust Decision-Making for Uncertain Markov Decision Processes Using Sigma Point Sampling. IEEE American Controls Conference, 2008.
5. P. Borkar and P. Varaiya. Adaptive Control of Markov Chains, I: Finite Parameter Set. IEEE Trans. on Automatic Control, AC-24(6), 1979.
6. G. Iyengar. Robust Dynamic Programming. Math. Oper. Res., 30(2):257-280, 2005.
7. R. Jaulmes, J. Pineau, and D. Precup. Active Learning in Partially Observable Markov Decision Processes. European Conference on Machine Learning (ECML), 2005.
8. R. Jaulmes, J. Pineau, and D. Precup. Learning in Non-Stationary Partially Observable Markov Decision Processes. ECML Workshop on Reinforcement Learning in Non-Stationary Environments, 2005.
9. V. Jilkov and X. Li. Online Bayesian Estimation of Transition Probabilities for Markovian Jump Systems. IEEE Trans. on Signal Processing, 52(6), 2004.
10. V. Konda and J. Tsitsiklis. Linear stochastic approximation driven by slowly varying Markov chains. Systems and Control Letters, 50, 2003.
11. V. Krishnamurthy and J. B. Moore. On-Line Estimation of Hidden Markov Model Parameters Based on the Kullback-Leibler Information Measure. IEEE Trans on Signal Processing, 41(8), 1993.
12. P. R. Kumar and W. Lin. Simultaneous Identification and Adaptive Control of Unknown Systems over Finite Parameters Sets. IEEE Trans. on Automatic Control, AC-28(1), 1983.
13. S. Mannor, D. Simester, P. Sun, and J. Tsitsiklis. Bias and Variance Approximation in Value Function Estimates. Management Science, 52(2):308-322, 2007.
14. P. Marbach. Simulation-based methods for Markov Decision Processes. PhD thesis, MIT, 1998.
15. R. W. Miller. Asymptotic behavior of the Kalman filter with exponential aging. AIAA Journal, 9, 1971.
16. A. Nilim and L. El Ghaoui. Robust Solutions to Markov Decision Problems with Uncertain Transition Matrices. Operations Research, 53(5), 2005.
17. L. R. Rabiner. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. IEEE Trans., 77(2), 1990.
18. M. Sato, K. Abe, and H. Takeda. Learning Control of Finite Markov Chains with Unknown Transition Probabilities. IEEE Trans. on Automatic Control, AC-27(2), 1982.
19. Y. Bar Shalom, X. Rong Li, and T. Kirubarajan. Estimation with Applications to Tracking and Navigation. Wiley Interscience, 2001.