Reinforcement learning via kernel temporal difference

Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS

Volumn , Issue , 2011, Pages 5662-5665

  Bae, Jihye ^a   Chhatbar, Pratik ^b   Francis, Joseph T ^b   Sanchez, Justin C ^c   Principe, Jose C ^a  

^a J Crayton Pruitt Family Department of Biomedical Engineering   (United States)

^b Downstate Health Sciences University   (United States)

^c University of Miami College of Engineering   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FASTER CONVERGENCE; MOTOR STATE; REACHING TASK; STOCHASTIC GRADIENT; TEMPORAL DIFFERENCE ERRORS; TEMPORAL DIFFERENCES; TIME DELAY NEURAL NETWORKS; VALUE FUNCTIONS;

ADAPTIVE FILTERS; NEURAL NETWORKS; REINFORCEMENT;

REINFORCEMENT LEARNING;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BIOMIMETICS; BRAIN; COMPUTER INTERFACE; ELECTROENCEPHALOGRAPHY; HUMAN; METHODOLOGY; PHYSIOLOGY; REINFORCEMENT;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIOMIMETICS; BRAIN; ELECTROENCEPHALOGRAPHY; HUMANS; PATTERN RECOGNITION, AUTOMATED; REINFORCEMENT (PSYCHOLOGY); USER-COMPUTER INTERFACE;

EID: 84055199049     PISSN: 1557170X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IEMBS.2011.6091370     Document Type: Conference Paper

References (22)

1. R. S. Sutton, "Learning to Predict by the Methods of Temporal Differences," Kluwer Academic Publishers, Boston, pp. 9-44, 1988.
2. J. Wessberg, C. R. Stambaugh, J. D. Kralik, P. D. Beck, M. Laubach, J. K. Chapin, J. Kim, S. J. Biggs, M. A. Srinivasan, and M. A. L. Nicolelis, "Real-time prediction of hand trajectory by ensembles of cortical neurons inprimates," Nature, vol. 408, pp. 361-365, Nov. 2000.
3. M. D. Serruya, N. G. Hatsopoulos, L. Paninski, M. R. Fellows, and J. P. Donoghue, "Brain-machine interface: Instant neural control of a movement signal," Nature, vol. 416, pp. 141-142, 2002.
4. D. M. Taylor, S. I. H. Tilley, and A. B. Schwartz, "Information conveyed through brain-control: Cursor versus robot," IEEE Trans. Neural Systems and Rehabilitation Eng., vol. 11, pp. 107-119, 2003.
5. J. M. Carmena, M. A. Lebedev, R. E. Crist, J. E. O'Doherty, D. M. Santucci, D. F. Dimitrov, P. G. Patil, C. S. Henriquez, and M. A. Nicolelis, "Learning to control a brain-machine interface for reaching and grasping by primates," PLoS Bio., vol. 1, no. 2, Nov. 2003.
6. J. C. Sanchez, "From Cortical Neural Spike Trains to Behavior: Modeling and Analysis," Ph.D. dissertation, Dept. Biomedical Eng., Univ. of Florida, Gainesville, FL, 2004.
7. S. Musallam, B. D. Corneil, B. Greger, H. Scherberger, and R. A. Andersen, "Cognitive Control Signals for Neural Prosthetics," Science9, vol. 305, no. 5681, pp. 258-262 Jul. 2004.
8. M. A. Lebedev, J. M. Carmena, J. E. O'Doherty, M. Zacksenhouse, C. S. Henriquez, J. C. Principe, and M. A. L. Nicolelis, "Cortical Ensemble Adaptation to Represent Velocity of an Artificial Actuator Controlled by a Brain-Machine Interface," The Journal of Neuroscience, vol. 25, no. 19, pp. 4681-4693, May 2005.
9. L. R. Hochberg, M. D. Serruya, G. M. Friehs, J. A. Mukand, M. Saleh, A. H. Caplan, A. Branner, D. Chen, R. D. Penn, and J. P. Donoghue, "Neuronal ensemble control of prosthetic devices by a human with tetraplegia," Nature, vol. 442, pp. 164-171, Jul. 2006.
10. M. Velliste, S. Perel, M. C. Spalding, A. S. Whitford, and A. B. Schwartz, "Cortical control of a prosthetic arm for self-feeding," Nature, vol. 453, pp. 1098-1101, Jun. 2008.
11. J. DiGiovanna, B. Mahmoudi, J. Fortes, J. C. Principe, and J. C. Sanchez, "Co-adaptive Brain-Machine Interface via Reinforcement Learning," IEEE Trans. Biomedical Engineering, vol. 56, no. 1, Jan, 2009.
12. B. Mahmoudi, "Integrating Robotic Action with Biologic Perception: A Brain Machine Symbiosis Theory," Ph.D. dissertation, Dept. Biomedical Eng., Univ. of Florida, Gainesville, FL, 2010.
13. S. Haykin, Neural Networks: a comprehensive foundation. Maxwell Macmillan Canada, 1994.
14. J. C. Sanchez, A. Tarigoppula, J. S. Choi, B. T. Marsh, P. Y. Chhatbar, B. Mahmoudi, and J. T. Francis, "Control of a Center-Out Reaching Task using a Reinforcement Learning Brain-Machine Interface," submitted for publication.
15. S. Haykin, Neural Networks and Learning Machines. Upper Saddle River, NJ: Pearson Education, Inc., 2009.
16. th Annual ACM Workshop on COLT, PA: ACM Press, pp. 144-152, 1992.
17. B. Scholkopf and A.J. Smola, Learning with Kernels. The MIT Press, 2002.
18. W. Liu, J. C. Principe, and S. Haykin, Kernel Adaptive Filtering. Wiley, 2010.
19. W. Liu, P. P. Pokharel, and J. C. Principe, "The Kernel Least-Mean Square Algorithm," IEEE Trans. Signal Processing, vol. 56, no. 3, Feb. 2008.
20. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Inroduction. Cambridge, MA: The MIT Press, 1998.
21. C. J. C. H. Watkins and P. Dayan, "Q-learning," in Machine Learning, vol. 8, num. 3-4, Springer, 1992, pp.279-292.
22. Y. Engel, "Algorithms and Representations for Reinforcement Learning," Ph.D. dissertation, School of Eng. and Computer Science, Hebrew Univ., Jerusalem, 2005