Reinforcement learning algorithms with function approximation: Recent advances and applications

Information Sciences

Volumn 261, Issue , 2014, Pages 1-31

  Xu, Xin ^a   Zuo, Lei ^a   Huang, Zhenhua ^a  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

Author keywords

Approximate dynamic programming; Function approximation; Generalization; Learning control; Reinforcement learning

Indexed keywords

APPROXIMATE DYNAMIC PROGRAMMING; FEATURE REPRESENTATION; FUNCTION APPROXIMATION; FUNCTION APPROXIMATION TECHNIQUES; GENERALIZATION; LEARNING CONTROL; MARKOV DECISION PROCESSES; PREDICTION AND CONTROL;

BENCHMARKING; LEARNING ALGORITHMS; MARKOV PROCESSES; REINFORCEMENT LEARNING;

APPROXIMATION ALGORITHMS;

EID: 84891828192     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2013.08.037     Document Type: Article

References (155)

1. B. Abdulhai, and R. Pringle et al. Reinforcement learning for true adaptive traffic signal control Journal of Transportation Engineering 129 3 2003 278 285
2. A. Al-Tamimi, F.L. Lewis, and M. Abu-Khalaf Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control Automatica 43 2007 473 481
3. A. Al-Tamimi, M. Abu-Khalaf, and F.L. Lewis Adaptive critic designs for discrete-time zero-sum games with application to H-Infinity control IEEE Transactions on Systems Man Cybernetics-Part B 2006
4. S. Amari Natural gradient works efficiently in learning Neural Computation 10 2 1998 251 276
5. A. Antos, R. Munos, C. Szepesvari, Regularized fitted Q-iteration for planning in continuous-space Markovian decision problems, in: 2009 American Control Conference, Hyatt Regency Riverfront, St. Louis, MO, USA, June 10-12, pp. 725-730.
6. D. Andre, S.J. Russell, State abstraction for programmable reinforcement learning agents, in: Proceedings of the Eighteenth National Conference on Artificial Intelligence, 2002, pp. 119-125.
7. I. Arel, and C. Liu et al. Reinforcement learning-based multi-agent system for network traffic signal control IET Intelligent Transport Systems 4 2 2010 128 135
8. F.R. Bach, and M.I. Jordan Kernel independent component analysis Journal of Machine Learning Research 3 2002 1 48
9. J.A. Bagnell, J.G. Schneider, Autonomous helicopter control using reinforcement learning policy search methods, in: Proceedings of the 2001 IEEE International Conference on Robotics & Automation, Seoul, Korea, 2001, pp. 1615-1620.
10. J.A. Bagnell, and J.G. Schneider Covariant policy search G. Gottlob, T. Walsh, Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03) 2003 Morgan Kaufmann San Francisco, CA, USA 1019 1024
11. L.C. Baird Residual algorithms: reinforcement learning with function approximation Proceedings of the 12th International Conference on Machine Learning (ICML 1995) 1995 Morgan Kaufman San Francisco, CA, USA 30 37
12. P.G. Balaji, and X. German et al. Urban traffic signal control using reinforcement learning agents IET Intelligent Transport Systems 4 3 2010 177 188
13. S.N. Balakrishnan, and V. Biega Adaptive-critic-based neural networks for aircraft optimal control Journal of Guidance, Control, Dynamics 19 4 1996 893 898
14. A.G. Barto, and T.G. Dietterich Reinforcement learning and its relationship to supervised learning J. Si, A. Barto, W. Powell, D. Wunsch, Handbook of Learning and Approximate Dynamic Programming 2004 Wiley-IEEE Press New York
15. A.G. Barto, and S. Mahadevan Recent advances in hierarchical reinforcement learning Discrete Event Dynamic Systems-Theory and Applications 13 1-2 2003 41 77
16. A.G. Barto, R. Sutton, and C.W. Anderson Neuron-like adaptive elements that can solve difficult learning control problems IEEE Transactions on Systems, Man, and Cybernetics 13 5 1983 834 846
17. J. Baxter, and P.L. Bartlett Infinite-horizon policy-gradient estimation Journal of Artificial Intelligence Research 15 2001 319 350
18. D.P. Bertsekas, and J. Tsitsiklis Neuro-Dynamic Programming 1996 Athena Scientific
19. S. Bhatnagar, R.S. Sutton, M. Ghavamzadeh, and M. Lee Natural actor-critic algorithms Automatica 45 11 2009 2471 2482
20. V.S. Borkar Stochastic approximation with two time scales Systems & Control Letters 29 5 1997 291 294
21. V.S. Borkar Stochastic Approximation: A Dynamical Systems Viewpoint 2008 Cambridge University Press
22. J. Boyan, A.W. Moore, Generalization in reinforcement learning: safely approximating the value function, in: Advances in Neural Information Processing Systems, 1995, pp. 369-376.
23. J. Boyan Technical update: least-squares temporal difference learning Machine Learning 49 2-3 2002 233 246
24. J. Boyan, and M. Littman Packet routing in dynamically changing networks: a reinforcement learning approach Advances in neural information processing systems 6 NIPS 1994 1994
25. S. Bradtke, B. Ydstie, A. Barto, Adaptive linear quadratic control using policy iteration, Univ. Massachusetts, Amherst, MA, Tech. Rep. CMPSCI-94-49, June 1994.
26. S. Bradtke, Incremental Dynamic Programming for On-Line Adaptive Optimal Control, Ph.D. thesis, University of Massachusetts, Computer Science Dept. Tech. Rep., 1994, pp. 94-62.
27. S.J. Brartke, and A. Barto Linear least-squares algorithms for temporal difference learning Machine Learning 22 1996 33 57
28. L. Busoniu, R. Babuska, B. De Schutter, and D. Ernst Reinforcement Learning and Dynamic Programming Using Function Approximators 2010 CRC Press NY
29. X. Cao Stochastic Learning and Optimization 2009 Springer-Verlag Berlin
30. M. Carreras, and J. Yuh et al. A behavior-based scheme using reinforcement learning for autonomous underwater vehicles IEEE Journal of Oceanic Engineering 30 2 2005 416 427
31. R.H. Crites, and A.G. Barto Elevator group control using multiple reinforcement learning agents Machine Learning 33 2-3 1998 235 262
32. R.H. Crites, and A.G. Barto Improving elevator performance using reinforcement learning Advances in Neural Information Processing Systems 8 NIPS 1995 1996
33. C. Darken, J. Moody, Note on learning rate schedules for stochastic optimization, in: Lippman, et al. (Eds.), Advances in Neural Information Processing Systems, vol. 3, 1991, pp. 1009-1016.
34. P. Dayan The convergence of TD(λ) for general λ Machine Learning 8 1992 341 362
35. P. Dayan, and T.J. Sejnowski TD(λ) converges with probability 1 Machine Learning 14 1994 295 301
36. T.G. Dietterich, and X. Wang Batch value function approximation via support vectors Advances in Neural Information Processing Systems vol. 14 2002 MIT Press Cambridge, MA 1491 1498
37. T.G. Dietterich Hierarchical reinforcement learning with the Max-Q value function decomposition Journal of Artificial Intelligence Research 13 2000 227 303
38. T.G. Dietterich, State abstraction in MAXQ hierarchical reinforcement learning, in: S.A. Solla, T.K. Leen, K.R. Muller (Eds.), Advances in Neural Information Processing Systems, NIPS, 2000, pp. 994-1000.
39. K. Driessens, and S. Dzeroski Integrating guidance into relational reinforcement learning Machine Learning 57 2004 271 304
40. K. Driessens, J. Ramon, and H. Blockeel Speeding up relational reinforcement learning through the use of an incremental first order decision tree learner L. De Raedt, P. Flach, Proceedings of the 13th European Conference on Machine Learning Lecture Notes in Artificial Intelligence vol. 2167 2001 Springer-Verlag 97 108
41. K. Driessens, and J. Ramon Relational instance based regression for relational reinforcement learning Proceedings of the Twentieth International Conference on Machine Learning 2003 AAAI Press 123 130
42. Y. Engel, S. Mannor, R. Meir, "Bayes meets bellman: the Gaussian Process approach to temporal difference learning, in: Proceedings of the Twentieth International Conference of Machine Learning, Washington, DC, 2003, pp. 154-161.
43. R. Enns, and J. Si Helicopter trimming and tracking control using direct neural dynamic programming IEEE Transactions on Neural Networks 14 4 2003 929 939
44. D. Ernst, P. Geurts, and L. Wehenkel Tree-based batch mode reinforcement learning Journal of Machine Learning Research 6 2005 503 556
45. D. Ernst, and M. Glavic et al. Power systems stability control: reinforcement learning framework IEEE Transactions on Power Systems 19 1 2004 427 435
46. A. Farahmand, and Cs. Szepesvári Model selection in reinforcement learning Machine Learning 85 3 2011 299 332
47. A.m. Farahmand, M. Ghavamzadeh, Cs. Szepesvári, and S. Mannor Regularized policy iteration NIPS 2008 441 448
48. A. Galindo-Serrano, and L. Giupponi Distributed Q-Learning for aggregated interference control in cognitive radio networks IEEE Transactions on Vehicular Technology 59 4 2010 1823 1834
49. K. Driessens, J. Ramon, and T. Gärtner Graph kernels and Gaussian Processes for relational reinforcement learning Machine Learning 64 1-3 2006 91 119
50. T. Gärtner, P. Flach, S. Wrobel, On graph kernels: hardness results and efficient alternatives, in: M.W.B. Scholkopf (Ed.), Proceedings of the 16th Annual Conference on Computational Learning Theory and the 7th Kernel Workshop, 2003, pp. 129-143.
51. A. Gosavi Reinforcement learning for long-run average cost European Journal of Operational Research 155 2004 654 674
52. A.P. George, and W.B. Powell Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming Machine Learning 65 2006 167 198
53. A. Geramifard, M. Bowling, M. Zinkevich, and R.S. Sutton iLSTD: eligibility traces and convergence analysis B. Scḧolkopf, J. Platt, T. Hoffman, Advances in Neural Information Processing Systems vol. 19 2007 MIT Press Cambridge, MA 441 448
54. M. Ghavamzadeh, Y. Engel, Bayesian policy gradient algorithms, in: Advances in Neural Information Processing Systems, 2006, pp. 457-464.
55. M. Ghavamzadeh, and S. Mahadevan Hierarchical average reward reinforcement learning Journal of Machine Learning Research 8 2007 2629 2669
56. D. Haussler, Convolution Kernels on Discrete Structures, Technical Report, Department of Computer Science, University of California at Santa Cruz, 1999.
57. B. Hengst, Safe state abstraction and reusable continuing subtasks in hierarchical reinforcement learning, in: AI 2007: Advances in Artificial Intelligence, Lecture Notes in Computer Science, vol. 4830, 2007, pp. 58-67.
58. Q.Y. Hu, and W.Y. Yue Markov Decision Processes with Their Applications 2008 Springer
59. K.M. Iftekharuddin Transformation invariant on-line target recognition IEEE Transactions on Neural Networks 22 6 2011 906 918
60. T. Jaakkola, M. Jordan, and S. Singh On the convergence of stochastic iterative dynamic programming algorithms Neural Computation 6 6 1994 185 1201
61. M.A.K. Jaradat, and M. AI-Rousan et al. Reinforcement based mobile robot navigation in dynamic environment Robotics and Computer-Integrated Manufacturing 27 2011 135 149
62. T. Jiang, and D. Grace et al. Efficient exploration in reinforcement learning-based cognitive radio spectrum sharing IET Communication 5 10 2011 1309 1317
63. J. Johns, M. Petrik, and S. Mahadevan Hybrid least-squares algorithms for approximate policy evaluation Machine Learning 76 2009 243 256
64. S. Kakade A natural policy gradient Advances in Neural Information Processing Systems 2002 1531 1538
65. J. Kober, and J. Peters Policy search for motor primitives in robotics Advances in Neural Information Processing Systems NIPS 2008 2008
66. N. Kohl, P. Stone, Machine learning for fast quadrupedal locomotion, in: D.L. McGuinness, G. Ferguson (Eds.), Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI 2004), AAAI Press, Menlo Park, pp. 611-616.
67. V.R. Konda, and J.N. Tsitsiklis Actor-critic algorithms Advances in Neural Information Processing Systems vol. 12 2000 MIT Press Cambridge, MA
68. M.G. Lagoudakis, and R. Parr Least-squares policy iteration Journal of Machine Learning Research 4 2003 1107 1149
69. F.L. Lewis, G. Lendaris, and D. Liu Special issue on approximate dynamic programming and reinforcement learning for feedback control IEEE Transactions on Systems, Man, and Cybernetics B 38 4 2008
70. F.L. Lewis, and D. Vrabie Reinforcement learning and adaptive dynamic programming for feedback control IEEE Circuits and Systems Magazine 9 3 2009 32 50
71. D. Liu, Y. Zhang, and H. Zhang A self-learning call admission control scheme for CDMA cellular networks IEEE Transactions on Neural Networks 16 5 2005 1219 1228
72. H.R. Maei, C. Szepesvári, S. Bhatnagar, D. Precup, and R.S. Sutton Convergent temporal-difference learning with arbitrary smooth function approximation J. Laferty, C. Williams, Advances in Neural Information Processing Systems vol. 22 2010 MIT Press Cambridge, MA, USA
73. H.R. Maei, C. Szepesvári, S. Bhatnagar, and R. Sutton Toward off-policy learning control with function approximation J. Furnkranz, T. Joachims, ICML 2010 Omnipress 719 726
74. S. Mahadevan, Proto-value functions: developmental reinforcement learning, in: Proceedings of the 22nd International Conference on Machine Learning, 2005, pp. 553-560.
75. S. Mahadevan, and M. Maggioni Proto-value functions: a laplacian framework for learning representation and control in markov decision processes Journal of Machine Learning Research 8 2007 2169 2231
76. A.R. Mahmood, R. Sutton, T.Degris, P.M. Pilarski, Tuning-free step-size adaptation, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Kyoto, Japan, 2012.
77. M. McPartl, and M. Gallagher Reinforcement learning in first person shooter games IEEE Transactions on Computational Intelligence and AI in Games 3 1 2011 43 56
78. M.L. Minsky, Theory of Neural-Analog Reinforcement Systems and its Application to the Brain-Model Problem, Ph.D. Thesis, Princeton University, 1954.
79. J. Mitola III, and G.Q. Maguire Cognitive radio: making software radios more personal IEEE Personal Communications 6 4 1999 13 18
80. S. Mohagheghi, and G.K. Venayagamoorthy et al. Adaptive critic design based neuro-fuzzy controller for a static compensator in a multimachine power system IEEE Transactions on Power Systems 21 4 2006 1744 1754
81. V. Nanduri, and T.K. Das A reinforcement learning model to assess market power under auction-based energy pricingm IEEE Transactions on Power Systems 22 1 2007 85 95
82. A. Nedic, and D.P. Bertsekas Least squares policy evaluation algorithms with linear function approximation Discrete Event Dynamic Systems 13 1 2003 79 110
83. A.Y. Ng, and H.J. Kim et al. Autonomous helicopter flight via reinforcement learning Advances in Neural Information Processing Systems 16 NIPS 2003 2004
84. J.O.J. Lee, J.W. Lee, and B.-T. Zhang Adaptive stock trading with dynamic asset allocation using reinforcement learning Information Sciences 176 15 2006 2121 2147
85. D. Ormoneit, and S. Sen Kernel-based reinforcement learning Machine Learning 49 2-3 2002 161 178
86. R. Parr, and S. Russell Reinforcement learning with hierarchies of machines Advances in Neural Information Processing Systems 1998 MIT Press Cambridge, MA 1043 1049
87. J. Peng, and B. Bhanu Delayed reinforcement learning for adaptive image segmentation and feature extraction IEEE Transactions on System Man and Cybernetics-Part C 28 3 1998 482 488
88. J. Peng, and B. Bhaun Closed-loop object recognition using reinforcement learning IEEE Transactions on Pattern Analysis and Machine Intelligence 20 2 1998 139 154
89. J. Peters, and S. Schaal Natural actor-critic Neurocomputing 71 2008 1180 1190
90. J. Peters, S. Schaal, Policy gradient methods for robotics, in: Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots ans Systems, Beijing, China, 2006, pp. 2219-2225.
91. J. Peters, S. Vijayakumar, S. Schaal, Reinforcement learning for humanoid robotics, in: IEEE/RSJ International Conference on Humanoid Robotics, 2003.
92. W.B. Powell Approximate Dynamic Programming: Solving the Curses of Dimensionality 2007 Wiley NY
93. L.A. Prashanth, and S. Bhatnagar Reinforcement learning with function approximation for traffic signal control IEEE Transactions on Intelligence Transportation Systems 12 2 2011 412 421
94. D.V. Prokhorov, and D.C. Wunsch Adaptive critic designs IEEE Transactions Neural Networks 8 5 1997 997 1007
95. C.E. Rasmussen, and M. Kuss Gaussian processes in reinforcement learning S. Thrun, L.K. Saul, B. Schölkopf, Advances in Neural Information Processing Systems vol. 16 2004 MIT Press 751 759
96. M. Riedmiller, and T. Gabel et al. Reinforcement learning for robot soccer Autonomous Robots 27 1 2009 55 74
97. M. Riedmiller, M. Montemerlo, et al., Learning to drive in 20 min, in: Proceedings of the FBIT 2007 Conference, Jeju, Korea, 2007.
98. S. Richter, D. Aberdeen, J. Yu, Natural actor-critic for road traffic optimisation, in: Advances in Neural Information Processing Systems, 2006, pp. 3522-3529.
99. A.L. Samuel Some studies in machine learning using game of checkers IBM Jounal on Research and Development 3 1959 211 229
100. B. Schölkopf, and A. Smola Learning with Kernels 2002 MIT Press Cambridge, MA
101. B. Schölkopf, S. Mika, C.J.C. Burges, P. Knirsch, K.-R. Müller, G. Rätsch, and A.J. Smola Input space vs feature space in kernel-based algorithms IEEE Transactions on Neural Networks 10 3 1999 1000 1017
102. A. Schwartz A reinforcement learning method for maximizing undiscounted rewards Proceedings of the Tenth Annual Conference on Machine Learning 1993 Morgan Kaufmann 298 305
103. T. Shimokawa, and K. Suzuki et al. Predicting investment behavior: an augmented reinforcement learning model Neurocomputing 72 2009 3447 3461
104. D. Silver, and R.S. Sutton et al. Reinforcement learning of local shape in the game of Go Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (IJCAI 2007) 2007 1053 1058
105. S.P. Singh, and D. Bertsekas Reinforcement learning for dynamic channel allocation in cellular telephone systems Advances in Neural Information Processsing Systems 9 NIPS 1996 1997 974 980
106. S.P. Singh, T. Jaakkola, M.L. Littman, and Cs. Szepesvari Convergence results for single-step on-policy reinforcement-learning algorithms Machine Learning 38 2000 287 308
107. S.P. Singh, and R.C. Yee An upper bound on the loss from approximate optimal value functions Machine Learning 16 3 1994 227 233
108. T. Söderström, and P. Stoica Instrumental Variable Methods in System Identification 1983 Springer-Verlag Berlin
109. P. Stone, and R.S. Sutton et al. Reinforcement learning for RoboCup-soccer keepaway Adaptive Behavior 13 3 2005 165 188
110. R. Sutton, and A.G. Barto Reinforcement Learning. An Introduction 1998 MIT Press Cambridge MA
111. R. Sutton, A.G. Barto, and R.J. Williams Reinforcement learning is direct adaptive control IEEE Control Systems 12 2 1992 19 22
112. R. Sutton, A.G. Barto, A temporal-difference model of classical conditioning, in: Proceedings of the 9th Annual Conference Cognitive Science Society, 1987, pp. 355-378.
113. R. Sutton Learning to predict by the method of temporal differences Machine Learning 3 1988 9 44
114. R. Sutton, H.R. Maei, D. Precup, S. Bhatnagar, D. Silver, C. Szepesvari, E. Wiewiora, Fast gradient-descent methods for temporal-difference learning with linear function approximation, in: Proceedings of the 26th Annual International Conference on Machine Learning (ICML-09), 2009, pp. 993-1000.
115. R. Sutton, C. Szepesvari, and H.R. Maei A convergent O(n) temporal-difference algorithm for off-policy learning with linear function approximation Advances in Neural Information Processing Systems vol. 21 2009 MIT Press Cambridge, MA, USA 1609 1616
116. R. Sutton, Adapting bias by gradient descent: an incremental version of delta-bar-delta, in: Proceedings of the 10th National Conference on Artificial Intelligence, 1992, pp. 171-176.
117. R. Sutton, D. Precup, and S. Singh Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning Artificial Intelligence 112 1999 181 211
118. R. Sutton, Cs. Szepesvári, A. Geramifard, M. Bowling, Dyna-style planning with linear function approximation and prioritized sweeping, UAI, 2008, pp. 528-536.
119. Cs. Szepesvári Algorithms for Reinforcement Learning 2010 Morgan and Claypool
120. G. Tesauro TD-Gammon, a self-teaching backgammon program, achieves master-level play Neural Computation 6 1994 215 219
121. S.Thrun, A. Schwartz, Issues in using function approximation for reinforcement learning, in: Proceedings of the Fourth Connectionist Models Summer School, 1993, pp. 255-263.
122. J.N. Tsitsiklis, Asynchronous Stochastic Approximation and Q-learning, Technical Report LIDS-P-2172, Laboratory for Information and Decision Systems, MIT, Cambridge, MA, 1993.
123. J.N. Tsitsiklis, and B.V. Roy An analysis of temporal difference learning with function approximation IEEE Transactions on Automatic Control 42 5 1997 674 690
124. W.T.B. Uther, M.M. Veloso, Tree based discretization for continuous state space reinforcement learning, in: Proceedings of AAAI-98, 1998, pp. 769-774.
125. K.G. Vamvoudakis, and Frank L. Lewis Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem Automatica 46 5 2010 878 888
126. K.G. Vamvoudakis, and Frank L. Lewis Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations Automatica 47 8 2011 1556 1569
127. V. Vapnik Statistical Learning Theory 1998 Wiley Interscience NewYork
128. G.K. Venayagamoorthy, R.G. Harley, and D.C. Wunsch Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator IEEE Transactions on Neural Networks 13 3 2002 764 773
129. J.G. Vlachogiannis, and N.D. Hatziargyriou Reinforcement learning for reactive power control IEEE Transactions on Power Systems 19 3 2004 1225 1317
130. D. Vrabie, F. Lewis, and M. Abu-Khalaf Adaptive optimal control for continuous-time linear systems based on policy iteration Automatica 45 2 2009 477 484
131. D. Vrabie, and F. Lewis Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems Neural Networks 22 3 2009 237 246
132. X. Wang, Y. Cheng, and J.-Q. Yi A fuzzy actor-critic reinforcement learning network Information Sciences 177 18 2007 3764 3781
133. F.Y. Wang, H. Zhang, and D. Liu Adaptive dynamic programming: an introduction IEEE Computational Intelligence Magazine 2009 39 47
134. C. Watkins, Learning from Delayed Rewards, Ph.D. thesis, Cambridge Univ., Cambridge, England, 1989.
135. C. Watkins, and P. Dayan Q-Learning Machine Learning 8 1992 279 292
136. P.J. Werbos Intelligence in the brain: a theory of how it works and how to build it Neural Networks 2009 200 212
137. P.J. Werbos, Using ADP to understand and replicate brain intelligence: the next level design, in: IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, 2007, pp. 209-216.
138. P.J. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavior Sciences, Ph.D. thesis, Committee Appl. Math. Harvard Univ., 1974.
139. B. Widrow, N. Gupta, and S. Maitra Punish/reward: Learning with a critic in adaptive threshold systems IEEE Transactions on Systems, Man, and Cybernetics SMC-3 5 1973 455 465
140. X. Xu Reinforcement Learning and Approximate Dynamic Programming 2010 Science Press Beijing
141. X. Xu, H.G. He, and D.W. Hu Efficient reinforcement learning using recursive least-squares methods Journal of Artificial Intelligence Research 16 2002 259 292
142. X. Xu, T. Xie, D.W. Hu, and X.C. Lu Kernel least-squares temporal difference learning International Journal of Information Technology 11 9 2005 54 63
143. X. Xu, D.W. Hu, and X.C. Lu Kernel based least-squares policy iteration for reinforcement learning IEEE Transactions on Neural Networks 18 4 2007 973 992
144. X. Xu Sequential anomaly detection based on temporal-difference learning: principles, models and case studies Applied Soft Computing 10 3 2010 859 867
145. X. Xu, C. Liu, S. Yang, and D. Hu Hierarchical approximate policy iteration with binary-tree state space decomposition IEEE Transactions on Neural Networks 22 12 2011 1863 1877
146. X. Xu, C. Liu, and D. Hu Continuous-action reinforcement learning with fast policy search and adaptive basis function selection Soft Computing - A Fusion of Foundations, Methodologies and Applications 15 6 2011 1055 1070
147. X. Xu, Z. Hou, C. Lian, and H. He Online learning control using adaptive critic designs with sparse kernel machines IEEE Transactions on Neural Networks and Learning Systems 24 5 2013 762 775
148. K.-L.A. Yau, P. Komisarczuk, et al., Applications of reinforcement learning to cognitive radio networks, 2010 IEEE International Conference on Communication Workshops (ICC), 2010, pp. 1-6.
149. P. Yin, and B. Bhanu et al. Integrating relevance feedback techniques for image retrieval using reinforcement learning IEEE Transactions on Pattern Analysis and Machine Intelligence 27 10 2005 1536 1551
150. T. Yu, and B. Zhou et al. Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q(λ) learning IEEE Transactions on Power Systems 26 3 2011 1272 1282
151. H. Zhang, L. Cui, X. Zhang, and Y. Luo Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method IEEE Transactions on Neural Networks 22 12 2011 2226 2236
152. W. Zhang, T.G. Dietterich. A reinforcement learning approach to job-shop scheduling, in: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI 1995), 1995, pp. 1114-1120.
153. P. Zhou, and Y. Chang et al. Reinforcement learning for repeated power control game in cognitive radio networks IEEE Journal on Selected Areas in Communications 30 1 2012 54 69
154. C. Zhou Robot learning with GA-based fuzzy reinforcement learning agents Information Sciences 145 2002 45 68
155. S. Dzeroski, L. De Raedt, and H. Blockeel Relational reinforcement Learning Proceedings of the 15th International Conference on Machine Learning 1998 Morgan Kaufmann 136 143