Approximate dynamic programming with a fuzzy parameterization

Automatica

Volumn 46, Issue 5, 2010, Pages 804-814

  Buşoniu, Lucian ^a   Ernst, Damien ^c   De Schutter, Bart ^a,b   Babuška, Robert ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^c UNIVERSITY OF LIÈGE   (Belgium)

Author keywords

Approximate dynamic programming; Convergence analysis; Fuzzy approximation; Value iteration

Indexed keywords

ACTION SPACES; APPROXIMATE DYNAMIC PROGRAMMING; APPROXIMATION ACCURACY; APPROXIMATORS; ASYNCHRONOUS ALGORITHMS; CONTROL ACTIONS; CONVERGENCE ANALYSIS; DETERMINISTIC PROCESS; DISCRETE SETS; DISCRETIZATIONS; FINITE NUMBER; FUZZY APPROXIMATION; FUZZY PARTITION; ITERATION ALGORITHMS; NON-LINEAR OPTIMAL CONTROL; OPTIMAL SOLUTIONS; PROCESS STATE; REWARD FUNCTION; STATE SPACE; SUBOPTIMALITY; TWO-LINK MANIPULATOR; VALUE ITERATION;

ALGORITHMS; ASYMPTOTIC ANALYSIS; DYNAMIC POSITIONING; MANIPULATORS; OPTIMAL SYSTEMS; OPTIMIZATION; PERCOLATION (SOLID STATE); PROCESS CONTROL;

DYNAMIC PROGRAMMING;

EID: 77950867376     PISSN: 00051098     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.automatica.2010.02.006     Document Type: Article

References (26)

1. Antos A., Munos R., and Szepesvári Cs. Fitted Q-iteration in continuous action-space MDPs. In: Platt J.C., Koller D., Singer Y., and Roweis S.T. (Eds). Advances in neural information processing systems: Vol. 20 (2008), MIT Press 9-16
2. Berenji H.R., and Vengerov D. A convergent actor-critic-based FRL algorithm with application to power management of wireless transmitters. IEEE Transactions on Fuzzy Systems 11 4 (2003) 478-485
3. Bertsekas D.P. Dynamic programming and optimal control, Vol. 2. 3rd ed. (2007), Athena Scientific
4. Bertsekas D.P., and Tsitsiklis J.N. Neuro-dynamic programming (1996), Athena Scientific
5. Brown M., and Harris C. Neurofuzzy adaptive modeling and control (1994), Prentice Hall
6. Buşoniu, L., De Schutter, B., & Babuška, R. (2006). Decentralized reinforcement learning control of a robotic manipulator. In Proceedings 9th international conference of control, automation, robotics, and vision, ICARCV-06 (pp. 1347-1352). Singapore, 5-8 December.
7. Buşoniu, L., Ernst, D., De Schutter, B., & Babuška, R. (2007). Fuzzy approximation for convergent model-based reinforcement learning. In Proceedings 2007 IEEE international conference on fuzzy systems, FUZZ-IEEE-07 (pp. 968-973). London, UK, 23-26 July.
8. Buşoniu, L., Ernst, D., De Schutter, B., & Babuška, R. (2008a). Consistency of fuzzy model-based reinforcement learning. In Proceedings 2008 IEEE international conference on fuzzy systems, FUZZ-IEEE-08 (pp. 518-524). Hong Kong, 1-6 June.
9. Buşoniu L., Ernst D., De Schutter B., and Babuška R. Continuous-state reinforcement learning with fuzzy approximation. In: Tuyls I.K., Nowé A., Guessoum Z., and Kudenko D. (Eds). Adaptive agents and multi-agent systems III. Lecture notes in computer science Vol. 4865 (2008), Springer 27-43
10. Chow C.-S., and Tsitsiklis J.N. An optimal one-way multigrid algorithm for discrete-time stochastic control. IEEE Transactions on Automatic Control 36 8 (1991) 898-914
11. Ernst D., Geurts P., and Wehenkel L. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research 6 (2005) 503-556
12. Farahmand, A. M., Ghavamzadeh, M., Szepesvári, Cs., & Mannor, S. (2009). Regularized fitted Q-iteration for planning in continuous-space Markovian decision problems. In Proceedings 2009 American control conference, ACC-09(pp. 725-730). St. Louis, US, 10-12 June.
13. Glorennec, P. Y. (2000). Reinforcement learning: An overview. In Proceedings European symposium on intelligent techniques, ESIT-00 (pp. 17-35). Aachen, Germany, 14-15 September.
14. Gordon, G. (1995). Stable function approximation in dynamic programming. In Proceedings 12th international conference on machine learning, ICML-95(pp. 261-268). Tahoe City, US, 9-12 July.
15. Horiuchi, T., Fujino, A., Katai, O., & Sawaragi, T. (1996). Fuzzy interpolation-based Q-learning with continuous states and actions. In Proceedings 5th IEEE international conference on fuzzy systems, FUZZ-IEEE-96 (pp. 594-600). New Orleans, US, 8-11 September.
16. Istratescu V.I. Fixed point theory: An introduction (2002), Springer
17. Jouffe L. Fuzzy inference system learning by reinforcement methods. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews 28 3 (1998) 338-355
18. Kruse R., Gebhardt J.E., and Klowon F. Foundations of fuzzy systems (1994), Wiley
19. Lagoudakis M.G., and Parr R. Least-squares policy iteration. Journal of Machine Learning Research 4 (2003) 1107-1149
20. Lin C.-K. A reinforcement learning adaptive fuzzy controller for robots. Fuzzy Sets and Systems 137 3 (2003) 339-352
21. Munos R., and Moore A. Variable-resolution discretization in optimal control. Machine Learning 49 2-3 (2002) 291-323
22. Munos R., and Szepesvári Cs. Finite time bounds for fitted value iteration. Journal of Machine Learning Research 9 (2008) 815-857
23. Santos M.S., and Vigo-Aguiar J. Analysis of a numerical dynamic programming algorithm applied to economic models. Econometrica 66 2 (1998) 409-426
24. Sutton R.S., and Barto A.G. Reinforcement learning: An introduction (1998), MIT Press
25. Szepesvári, Cs., & Smart, W. D. (2004). Interpolation-based Q-learning. In Proceedings 21st international conference on machine learning, ICML-04(pp. 791-798). Bannf, Canada, 4-8 July.
26. Tsitsiklis J.N., and Van Roy B. Feature-based methods for large scale dynamic programming. Machine Learning 22 1-3 (1996) 59-94