Continuous-action reinforcement learning with fast policy search and adaptive basis function selection

Soft Computing

Volumn 15, Issue 6, 2011, Pages 1055-1070

  Xu, Xin ^a   Liu, Chunming ^a   Hu, Dewen ^a  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

Author keywords

Approximate policy iteration; Generalization; Learning control; Markov decision processes; Reinforcement learning

Indexed keywords

ACTION SPACES; ADAPTIVE BASIS FUNCTION; COMPUTATIONALLY EFFICIENT; CONTINUOUS SPACES; CONTINUOUS STATE; DECISION PROBLEMS; GENERALIZATION; GENERALIZATION ABILITY; KERNEL MACHINE; LEARNING CONTROL; LEARNING EFFICIENCY; LINEAR FUNCTIONS; MACHINE-LEARNING; MARKOV DECISION PROBLEM; MARKOV DECISION PROCESSES; NEAR-OPTIMAL POLICIES; OPEN PROBLEMS; OPTIMAL ACTIONS; POLICY ITERATION; POLICY SEARCH; SELECTION METHODS; SIMULATION RESULT; SPARSE APPROXIMATIONS; TEMPORAL DIFFERENCE LEARNING; VALUE FUNCTIONS;

ARTIFICIAL INTELLIGENCE; LEARNING ALGORITHMS; MARKOV PROCESSES; OPTIMIZATION;

REINFORCEMENT LEARNING;

EID: 79956192776     PISSN: 14327643     EISSN: 14337479     Source Type: Journal    
DOI: 10.1007/s00500-010-0581-3     Document Type: Article

References (30)

1. Bach FR, Jordan MI (2002) Kernel independent component analysis. J Mach Learn Res 3: 1-48.
2. Barto AG, Sutton RS, Anderson CW (1983) Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans Syst Man Cybern 13(5): 835-846.
3. Baxter J, Bartlett PL (2001) Infinite-horizon policy-gradient estimation. J Artif Intell Res 15: 319-350.
4. Bertsekas DP, Tsitsiklis JN (1996) Neurodynamic programming. Athena Scientific, Belmont.
5. Boyan J (2002) Technical update: least-squares temporal difference learning. Mach Learn 49(2-3): 233-246.
6. Crites RH, Barto AG (1998) Elevator group control using multiple reinforcement learning agents. Mach Learn 33(2-3): 235-262.
7. Dayan P (1992) The convergence of TD(λ) for general λ. Mach Learn 8: 341-362.
8. Dayan P, Sejnowski TJ (1994) TD(λ) converges with probability 1. Mach Learn 14: 295-301.
9. Engel Y, Mannor S, Meir R (2004) The kernel recursive least-squares algorithm. IEEE Trans Signal Process 52(8): 2275-2285.
10. Hasselt HV, Wiering M (2007) Reinforcement learning in continuous action spaces. In: 2007 IEEE symposium on approximate dynamic programming and reinforcement learning, pp 272-279.
11. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4: 237-285.
12. Lagoudakis MG, Parr R (2003) Least-squares policy iteration. J Mach Learn Res 4: 1107-1149.
13. Lazaric A, Restelli M, Bonarini A (2008) Reinforcement learning in continuous action spaces through sequential Monte Carlo methods. In: Advances in neural information processing systems. MIT Press, Cambridge.
14. Mahadevan S, Maggioni M (2007) Proto-value functions: a Laplacian framework for learning representation and control in Markov decision processes. J Mach Learn Res 8: 2169-2231.
15. Millan JDR, Posenato D, Dedieu E (2002) Continuous-action q-learning. Mach Learn 49(2/3): 247-265.
16. Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8(5): 997-1007.
17. Rasmussen CE, Kuss M (2004) Gaussian processes in reinforcement learning. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in neural information processing systems, vol 16. MIT Press, Cambridge, pp 751-759.
18. Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, Cambridge.
19. Singh SP, Jaakkola T, Littman ML, Szepesvari C (2000) Convergence results for single-step on-policy reinforcement-learning algorithms. Mach Learn 38: 287-308.
20. Sutton R (1988) Learning to predict by the method of temporal differences. Mach Learn 3(1): 9-44.
21. Sutton R (1996) Generalization in reinforcement learning: successful examples using sparse coarse coding. In: Advances in neural information processing systems, vol 8. MIT Press, Cambridge, pp 1038-1044.
22. Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press, Cambridge.
23. Tesauro G (1994) TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Comput 6: 215-219.
24. Tsitsiklis JN (1994) Asynchronous stochastic approximation and Q-learning. Mach Learn 16: 185-202.
25. Tsitsiklis JN, Roy BV (1997) An analysis of temporal difference learning with function approximation. IEEE Trans Autom Control 42(5): 674-690.
26. Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8: 279-292.
27. Whiteson S, Stone P (2006) Evolutionary function approximation for reinforcement learning. J Mach Learn Res 7: 877-917.
28. Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8: 229-256.
29. Xu X, Hu DW, Lu XC (2007) Kernel-based least-squares policy iteration for reinforcement learning. IEEE Trans Neural Netw 18(4): 973-997.
30. Zhang W, Dietterich T (1995) A reinforcement learning approach to job-shop scheduling. In: Proceedings of the fourteenth international joint conference on artificial intelligence. Morgan Kaufmann, pp 1114-1120.