Robust adaptive Markov decision processes in multi-vehicle applications

Proceedings of the American Control Conference

Volumn , Issue , 2009, Pages 1304-1309

  Bertuccelli, Luca F ^a   Bethke, Brett ^a   How, Jonathan P ^a  

^a Massachusetts Institute of Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTUAL FLIGHT; ADAPTATION PROCESS; ADAPTIVE FRAMEWORK; ADAPTIVE POLICY; INDIVIDUAL STRENGTH; MARKOV DECISION PROCESSES; MISSION PERFORMANCE; MODEL UPDATING; MULTI-VEHICLES; NUMBER OF STATE; OPTIMAL VALUE FUNCTIONS; ROBUST ADAPTIVE; TRANSIENT BEHAVIOR; TRANSIENT PERFORMANCE; TRANSITION PROBABILITIES; WORST-CASE PERFORMANCE;

MARKOV PROCESSES;

SIMULATORS;

EID: 70449640592     PISSN: 07431619     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ACC.2009.5160511     Document Type: Conference Paper

References (26)

1. A. Nilim and L. E. Ghaoui, "Robust Solutions to Markov Decision Problems with Uncertain Transition Matrices," Operations Research, vol. 53, no. 5, 2005.
2. G. Iyengar, "Robust Dynamic Programming," Math. Oper. Res., vol. 30, no. 2, pp. 257-280, 2005.
3. S. Mannor, D. Simester, P. Sun, and J. Tsitsiklis, "Bias and Variance Approximation in Value Function Estimates," Management Science, vol. 52, no. 2, pp. 308-322, 2007.
4. L. F. Bertuccelli and J. P. How, "Robust Decision-Making for Uncertain Markov Decision Processes Using Sigma Point Sampling," IEEE American Controls Conference, 2008.
5. D. E. Brown and C. C. White., "Methods for reasoning with imprecise probabilities in intelligent decision systems," IEEE Conference on Systems, Man and Cybernetics, pp. 161-163, 1990.
6. J. K. Satia and R. E. Lave., "Markovian Decision Processes with Uncertain Transition Probabilities," Operations Research, vol. 21, no. 3, 1973.
7. C. C. White and H. K. Eldeib., "Markov Decision Processes with Imprecise Transition Probabilities," Operations Research, vol. 42, no. 4, 1994.
8. A. Bagnell, A. Ng, and J. Schneider, "Solving Uncertain Markov Decision Processes," NIPS, 2001.
9. K. J. Astrom and B. Wittenmark, Adaptive Control. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1994.
10. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). The MIT Press, 1998.
11. R. Jaulmes, J. Pineau, and D. Precup., "Active Learning in Partially Observable Markov Decision Processes," European Conference on Machine Learning (ECML), 2005.
12. R. Jaulmes, J. Pineau, and D. Precup., "Learning in Non-Stationary Partially Observable Markov Decision Processes," ECML Workshop on Reinforcement Learning in Non-Stationary Environments, 2005.
13. P. Marbach, Simulation-based methods for Markov Decision Processes. PhD thesis, MIT, 1998.
14. V. Konda and J. Tsitsiklis, "Linear stochastic approximation driven by slowly varying Markov chains," Systems and Control Letters, vol. 50, 2003.
15. M. Sato, K. Abe, and H. Takeda., "Learning Control of Finite Markov Chains with Unknown Transition Probabilities," IEEE Trans. on Automatic Control, vol. AC-27, no. 2, 1982.
16. P. R. Kumar and W. Lin., "Simultaneous Identification and Adaptive Control of Unknown Systems over Finite Parameters Sets.," IEEE Trans. on Automatic Control, vol. AC-28, no. 1, 1983.
17. J. Ford and J. Moore, "Adaptive Estimation of HMM Transition Probabilities," IEEE Transactions on Signal Processing, vol. 46, no. 5, 1998.
18. P. A. Ioannou and J. Sun, Robust Adaptive Control. Prentice-Hall, 1996.
19. M. Alighanbari and J. P. How, "A Robust Approach to the UAV Task Assignment Problem," International Journal of Robust and Nonlinear Control, vol. 18, no. 2, 2008.
20. M. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, 2005.
21. L. F. Bertuccelli, Robust Decision-Making with Model Uncertainty in Aerospace Systems. PhD thesis, MIT, 2008.
22. B. Bethke, J. How, and J. Vian., " Group Health Management of UAV Teams With Applications to Persistent Surveillance," IEEE American Controls Conference, 2008.
23. L. F. Bertuccelli and J. P. How, "Estimation of Non-Stationary Markov Chain Transition Models," IEEE Conference on Decision and Control, 2008.
24. A. Barto, S. Bradtke, and S. Singh., " Learning to Act using Real-Time Dynamic Programming," Artificial Intelligence, vol. 72, pp. 81-138, 1993.
25. V. Gullapalli and A. Barto., "Convergence of Indirect Adaptive Asynchronous Value Iteration Algorithms," Advances in NIPS, 1994.
26. B. Bethke, L. Bertuccelli, and J. P. How, "Experimental Demonstration of MDP- Based Planning with Model Uncertainty," in AIAA Guidance Navigation and Control Conference, Aug 2008. AIAA-2008-6322.