Adaptive optimal control without weight transport

Neural Computation

Volumn 24, Issue 6, 2012, Pages 1487-1518

  Chinta, Lakshminarayan V ^a   Tweed, Douglas B ^b,c  

^a UNIVERSITY OF TORONTO   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

^c YORK UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; BRAIN; COMPUTER SIMULATION; LEARNING; NERVE CELL; PHYSIOLOGY; SYNAPSE;

ALGORITHMS; BRAIN; COMPUTER SIMULATION; LEARNING; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONS; SYNAPSES;

EID: 84865286931     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00277     Document Type: Letter

References (68)

1. Abdelghani, M. N., Lillicrap, T. P., & Tweed, D. B. (2008). Sensitivity derivatives for flexible sensorimotor learning. Neural Comput 20(8), 2085-2111.
2. Abdelghani, M. N., & Tweed, D. B. (2010). Learning course adjustments during arm movements with reversed sensitivity derivatives. BMC Neuroscience, 11, 150.
3. Abu-Khalaf, M., & Lewis, F. L. (2004). Nearly optimal state feedback control of constrained nonlinear systems using a neural networks HJB approach. Annual Reviews in Control, 28, 239-251.
4. Abu-Khalaf,M., & Lewis, F. L. (2005). Nearly optimal control laws for nonlinear systemswith saturating actuators using a neural networkHJB approach. Automatica, 41, 779-791.
5. Beard, R., Saridis, G., &Wean, J. (1997). Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica, 33(12), 2159-2177
6. Bishop, R., & Goldberg, S. (1980). Tensor analysis on manifolds. New York: Dover.
7. Bryson, A. E., & Ho, Y.-C. (1975). Applied optimal control: Optimization, estimation, and control. London: Taylor & Francis.
8. Chen-Harris, H., Joiner,W. M., Ethier, V., Zee, D. S.,&Shadmehr, R. (2008).Adaptive control of saccades via internal feedback. J. Neurosci., 28, 2804-2813.
9. Chistiakova, M., & Volgushev, M. (2009). Heterosynaptic plasticity in the neocortex. Experimental Brain Research, 199, 377-390.
10. Dragoi, G., & Tonegawa, S. (2011). Preplay of future place cell sequences by hippocampal cellular assemblies. Nature, 469, 397-403.
11. Engel, Y.,Mannor, S., & Meir, R. (2004). The kernel recursive least-square algorithm. IEEE Transactions on Signal Processing, 52, 2275-2285.
12. Farrell, J. A., & Polycarpou, M. M. (2006). Adaptive approximation based control: Unifying neural, fuzzy, and traditional adaptive approximation approaches. Hoboken, NJ: Wiley-Interscience.
13. Fitzsimonds, R. M., Song, H.-J., & Poo, M. M. (1997). Propagation of acitivitydependent synaptic depression in simple neural networks. Nature, 388, 439-448.
14. Fusi, S., Drew, P. J., & Abbott, L. F. (2005). Cascade models of synaptically stored memories. Neuron, 45, 599-611.
15. Gockenbach, M. S. (2003). Introduction to sequential quadratic programming. Available online at http://www.math.mtu.edu/~msgocken/ma5630spring2003/lectures/sqp1/sqp1.pdf.
16. Grossberg, S. (1987). Competitive learning: From interactive activation to adaptive resonance. Cognitive Science, 11, 23-63.
17. Guitton, D., & Volle, M. (1987). Gaze control in humans: Eye-head coordination during orienting movements to targets within and beyond the oculomotor range. Journal of Neurophysiology, 58, 427-459.
18. Gupta, A., van der Meer, M., Touretzky, D. S., & Redish, A. D. (2010). Hippocampal replay is not a simple function of experience. Neuron, 65, 695-705.
19. Harris, C. M., & Wolpert, D. M. (1998). Signal-dependent noise determines motor planning. Nature, 394, 780-784.
20. Haykin, S. (2002). Adaptive filter theory. Upper Saddle River, NJ: Prentice Hall.
21. Humeau, Y., Shaban, H., Bissi'ere, S., & Lothi, A. (2003). Presynaptic induction of heterosynaptic associative plasticity in the mammalian brain. Nature, 426, 843- 845.
22. Kawato, M., & Gomi, H. (1992). The cerebellum and VOR/OKR learning models. TINS, 15, 445-453.
23. Kirk, D. E. (2004). Optimal control theory: An introduction. Mineola, NY: Dover.
24. Koch, C. (1999). Biophysics of computation. New York: Oxford University Press.
25. Kolen, J. F., & Pollack, J. B. (1994). Back-propagation without weight transport. In Proceedings of Neural Networks 1994: IEEE World Conference on Computational Intelligence. Piscataway, NJ: IEEE.
26. Krauzlis, R. J., Basso, M. A., & Wurtz, R. H. (1997). Shared motor error for multiple eye movements. Science, 276, 1693-1695.
27. Laurutis, V. P., & Robinson, D. A. (1986). The vestibulo-ocular reflex during human saccadic eye movements. Journal of Physiology, 373, 209-233.
28. Levine, D. S. (2000). Introduction to neural and cognitive modeling, Mahwah, NJ: Erlbaum.
29. Lewis, F. L.,&Abu-Khalaf,M. (2003).AHamilton-Jacobi setup for constrained neural network control. In Proceedings of the IEEE International Symposium on Intelligent Control. Piscataway, NJ: IEEE.
30. Lewis, F. L., Jagannathan, S., & Yesildirek, A. (1999). Neural network control of robot manipulators and nonlinear systems. London: Taylor and Francis.
31. Li,W., & Todorov, E. (2004). Iterative linear quadratic regulator design for nonlinear biological movement systems. In Proceedings of the First International Conference on Informatics in Control, Automation and Robotics. New York: Kluwer.
32. Lisberger, S. G., & Sejnowski, T. J. (1992). Motor learning in a recurrent network model based on the vestibulo-ocular reflex. Nature, 360, 159-161.
33. Liu,W., Príncipe, J., &Haykin, S. (2010). Kernel adaptive filtering.Hoboken, NJ:Wiley.
34. Lyshevski, S. E. (1998).Optimal control of nonlinear continuous-time systems: design of bounded controllers via generalized nonquadratic functionals. In Proceedings of American Control Conference. Piscataway, NJ: IEEE.
35. Mayer, A., Hanlon, F., Franco, A. R., Teshiba, T. M., Thoma, R. J., Clark, V. P., et al. (2009). The neural networks underlying auditory sensory gating. NeuroImage, 44, 182-189.
36. Mazzoni, P., & Andersen, R. A., & Jordan, M. I. (1991). A more biologically plausible learning rule for neural networks. Proc. Natl. Acad. Sci. USA, 88, 4433- 4437.
37. Menzies, J., Porrill, J., Dutia, M., & Dean, P. (2010). Synaptic plasticity in medial vestibular nucleus neurons: Comparison with computational requirements of VOR adaptation. PLoS One, 5, 1-17.
38. Misslisch, H., Tweed, D., Fetter, M., Sievering, D., & Koenig, E. (1994). Rotational kinematics of the human vestibuloocular reflex. III. Listing's law. J. Neurophysiol., 72(5), 2490-2502.
39. Nakao, S., Curthoys, I. S., & Markham, C. H. (1980). Direct inhibitory projection of pause neurons to nystagmus-related pontomedullary reticular burst neurons in the cat. Experimental Brain Research, 40, 283-293.
40. O'Donnell, P. (2003). Dopamine gating of forebrain neural ensembles. European Journal of Neuroscience, 17, 429-435.
41. Oztas, E. (2003). Neuronal tracing. Neuroanatomy, 2, 2-5.
42. Plackett, R. (1950). Some theorems in least squares. Biometrika, 37, 149-157.
43. Poirazi, P., Brannon, T., & Mel, B. W. (2003). Pyramidal neuron as 2-layer neural network. Neuron, 37, 989-999.
44. Porrill, J., Dean, P., & Stone, J. V. (2004). Recurrent cerebellar architecture solves the motor-error problem. Proc. R. Soc. B, 271, 789-796.
45. Rivest, F., Bengio, Y., & Kalaska, J. (2005). Brain inspired reinforcement learning. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems, 17 (pp. 1129-1136). Cambridge, MA: MIT Press.
46. Rolls, E. T., & Deco, G. (2002). Computational neuroscience of vision. New York: Oxford University Press.
47. Saridis, G., & Lee, C. S. (1979). An approximation theory of optimal control for trainable manipulators. IEEE Transactions on Systems, Man, Cybernetics, 9(3), 152- 159.
48. Scḧolkopf, B., & Smola, A. J. (2002). Learning with kernels: Support vector machines, regularization, optimization, and beyond. Cambridge, MA: MIT Press.
49. Schreiber, K. M., Crawford, J. D., Fetter, M., & Tweed, D. B. (2001). The motor side of depth vision. Nature, 410, 819-822.
50. Seidemann, E., Zohary, E., & Newsome, W. T. (1998). Temporal gating of neural signals during performance of a visual discrimination task. Nature, 394, 72-75.
51. Shibata, T., Tabata, H., Schaal, S., & Kawato, M. (2005). A model of smooth pursuit in primates based on learning the target dynamics. Neural Networks, 18(3), 213-224.
52. Shimansky, Y. P.,Kang, T.,&He, J. (2004).Anovelmodel of motor learning capable of developing an optimal movement control law online from scratch. Biol. Cybern., 90, 133-145.
53. Singh, K., & Scott, S. H. (2003). A motor learning strategy reflects neural circuitry for limb control. Nature Neuroscience, 6, 399-403.
54. Slotine, J. J. E., & Li, W. (1991). Applied nonlinear control. Upper Saddle River, NJ: Prentice Hall.
55. Sontag, E. D. (1998). Mathematical control theory: Deterministic finite dimensional systems. New York: Springer.
56. Spivak, M. (1965). Calculus on manifolds. New York: Benjamin.
57. Spruston, N., & Kath,W. L. (2004). Dendritic arithmetic. Nat. Neurosci., 7, 567-569.
58. Stork, D. G. (1989). Is backpropagation biologically plausible? In Proceeding of Neural Networks 1989: International Joint Conference on Neural Networks (pp. 241-246). Piscataway, NJ: IEEE.
59. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge, MA: MIT Press.
60. Sutton, R. S., McAllester,D., Singh, S.,&Mansour, Y. (2000). Policy gradientmethods for reinforcement learning with function approximation. In S. A. Solla, T. K. Leen, & K.-K. Muller (Eds.), Advances in neural information processing systems, 12 (pp. 1057-1063). Cambridge, MA: MIT Press.
61. Tao, H.-Z.W., Zhang, L. I., Bim, G.-Q., & Poo, M.-M. (2000). Selective presynaptic propagation of long-term potentiation in defined neural nertworks. Journal of Neuroscience, 20, 3233-3243.
62. Todorov, E. (2004).Optimality principles in sensorimotor control. Nat. Neurosci., 7(9), 907-915.
63. Todorov, E., & Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nat. Neurosci., 5, 1226-1235.
64. Tweed, D., Haslwanter, T., & Fetter, M. (1998). Optimizing gaze control in three dimensions. Science, 281, 1363-1366.
65. Werfel, J., Xie, X., & Seung, A. S. (2005). Learning curves for stochastic gradient descent in linear feedforward networks. Neural Computation, 17, 2699-2718.
66. Whittle, P. (1996). Optimal control: Basics and beyond. Hoboken, NJ: Wiley.
67. Widrow, B.,&Hoff,M. (1960).Adaptive switching circuits. IREWESCON Convention Record, 4, 96-104.
68. Wolpert, D. M., & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11, 1317-1329.