Fast convergence of learning requires plasticity between inferior olive and deep cerebellar nuclei in a manipulation task: A closed-loop robotic simulation

Frontiers in Computational Neuroscience

Volumn 8, Issue AUG, 2014, Pages

  Luque, Niceto R ^a   Garrido, Jesús A ^b,c   Carrillo, Richard R ^a   D'Angelo, Egidio ^c,d   Ros, Eduardo ^a  

^a UNIVERSITY OF GRANADA   (Spain)

^b Consorzio Interuniversitario per le Scienze Fisiche della Materia ^*  (Italy)

^c UNIVERSITY OF PAVIA   (Italy)

^d C MONDINO NATIONAL NEUROLOGICAL INSTITUTE   (Italy)

Author keywords

Cerebellar nuclei; Inferior olive; Learning consolidation; Long term synaptic plasticity; Modeling

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; COMPLEX NETWORKS; MODELS; NEURONS; ROBOTICS;

ADAPTIVE MOTOR CONTROLS; CEREBELLAR NUCLEI; DISTRIBUTED LEARNING; GENERALIZATION PROPERTIES; INFERIOR OLIVE; PROCEDURAL LEARNING; ROBOTIC MANIPULATION; SYNAPTIC PLASTICITY;

BRAIN;

ARTICLE; BRAIN FUNCTION; CEREBELLUM NUCLEUS; DEEP CEREBELLAR NUCLEUS; INFERIOR OLIVE CEREBELLAR NUCLEUS; INFORMATION PROCESSING; LEARNING; MOTOR CONTROL; MOTOR PERFORMANCE; NERVE CELL PLASTICITY; PROCESS MODEL; PURKINJE CELL; ROBOTICS; TASK PERFORMANCE;

EID: 84924201566     PISSN: None     EISSN: 16625188     Source Type: Journal    
DOI: 10.3389/fncom.2014.00097     Document Type: Article

References (89)

1. Albu-Schäffer, A., Haddadin, S., Ott, C., Stemmer, A., Wimböck, T., and Hirzinger, G. (2007). The DLR lightweight robot: design and control concepts for robots in human environments. Int. J. Ind. Rob. 34, 376-385. doi: 10.1108/01439910710774386
2. Albus, J. S. (1971). A theory of cerebellar function. Math. Biosci.10, 25-61. doi: 10.1016/0025-5564(71)90051-4
3. Anastasio, T. J. (2001). Input minimization: a model of cerebellar learning without climbing fiber error signals. Neuroreport12:3825. doi: 10.1097/00001756-200112040-00045
4. Arimoto, S. (1984). "Stability and robustness of PID feedback control for robot manipulators of sensory capability, " inRobotics Research, 1st International Symposium(MIT Press), 783-799.
5. Bastian, A. J. (2006). Learning to predict the future: the cerebellum adapts feedforward movement control. Curr. Opin. Neurobiol.16, 645-649. doi: 10.1016/j.conb.2006.08.016
6. Baumel, Y., Jacobson, G. A., and Cohen, D. (2009). Implications of functional anatomy on information processing in the deep cerebellar nuclei. Front. Cell. Neurosci.3, 1-8. doi: 10.3389/neuro.03.014.2009
7. Bazzigaluppi, P. R. D. G. J., Van Der Giessen, R. S., Khosrovani, S., De Zeeuw, C. I., and De Jeu, M. T. G. (2012). Olivary subthreshold oscillations and burst activity revisited. Front. Neural Circuits6:91. doi: 10.3389/fncir.2012.00091
8. Bellebaum, C., Koch, B., Schwarz, M., and Daum, I. (2008). Focal basal ganglia lesions are associated with impairments in reward-based reversal learning. Brain 131, 829-841. doi: 10.1093/brain/awn011
9. Bengtsson, F., and Hesslow, G. (2006). Cerebellar control of the inferior olive. Cerebellum5, 7-14. doi: 10.1080/14734220500462757
10. Bostan, A. C., Dum, R. P., and Strick, P. L. (2013). Cerebellar networks with thecerebralcortexandbasalganglia. Trends Cogn. Sc i.17, 241-254. doi: 10.1016/j.tics.2013.03.003
11. Chai, K. M., Klanke, S., Williams, C., and Vijayakumar, S. (2008). "Multi-task gaussian process learning of robot inverse dynamics, " in Proceedings Advances in Neural Information Processing Systems (NIPS '08)(Vancouver, BC), 1-8.
12. D'Angelo, E. (2010). Editorial to the special issue of frontiers in cellular neuroscience: rebuilding cerebellar network computations from cellular neurophysiology. Front. Cell. Neurosci.4:131. doi: 10.3389/fncel.2010.00131
13. D'Angelo, E. (2011). Neural circuits of the cerebellum: hypothesis for function. J. Int. Neurosci.10, 317-352. doi: 10.1142/S0219635211002762
14. D'Angelo, E. (2014). The organization of plasticity in the cerebellar cortex: from synapses to control. Prog. Brain Res.210, 31-58. doi: 10.1016/B978-0-444-63356-9.00002-9
15. D'Angelo, E., and De Zeeuw, C. I. (2009). Timing and plasticity in the cerebellum: focus on the granular layer. Trends Neuros c i. 32:10. doi: 10.1016/j.tins.2008.09.007
16. De Gruijl, J. R. P. B., De Jeu, M. T. G., and De Zeeuw, C. I. (2012). Climbing fiber burst size and olivary sub-threshold oscillations in a network setting. PLoS Comput. Biol.8:e1002814. doi: 10.1371/journal.pcbi.1002814
17. Desmurget, M., and Grafton, S. (2000). Forward modeling allows feedback control for fast reaching movements. Trends Cogn. Sc i.4, 423-431. doi: 10.1016/S1364-6613(00)01537-0
18. De Zeeuw, C. I., Hoebeek, F. E., Bosman, L. W., Schonewille, M., Witter, L., and Koekkoek, S. K. (2011). Spatiotemporal firing patterns in the cerebellum. Nat. Rev. Neurosci.12, 327-344. doi: 10.1038/nrn3011
19. De Zeeuw, C. I., Simpson, J. I., Hoogenraad, C. C., Galjart, N., Koekkoek, S. K., and Ruigrok, T. J. (1998). Microcircuitry and function of the inferior olive. Trends Neurosci.21, 391-400. doi: 10.1016/S0166-2236(98)01310-1
20. Evans, G. J. O. (2007). Synaptic signalling in cerebellar plasticity. Biol. Cell99, 363-378. doi: 10.1042/BC20070010
21. Gao, Z., Vanbeugen, B. J., and De Zeeuw, C. I. (2012). Distributed synergistic plasticity and cerebellar learning. Nat. Rev. Neurosci.13, 1-17. doi: 10.1038/nrn3391
22. Garrido, J. A., Luque, N. R., D'Angelo, E., and Ros, E. (2013a). Distributed cerebellar plasticity implements adaptable gain control in a manipulation task: a closed-loop robotic simulation. Front. Neural Circuits 7:159. doi: 10.3389/fncir.2013.00159
23. Garrido, J. A., Ros, E., and D'Angelo, E. (2013b). Spike timing regulation on the millisecond scale by distributed synaptic plasticity at the cerebellum input stage: a simulation study. Front. Comput. Neurosci. 7:64. doi: 10.3389/fncom.2013.00064
24. Goodwin, G. C., and Sin, K. C. (1984). Adaptive Filtering Prediction and Control. Englewood Cliffs, NJ: Prentice Hall.
25. Hansel, C., Linden, D. J., and D'Angelo, E. (2001). Beyond parallel fiber LTD: the diversity of synaptic and non-synaptic plasticity in the cerebellum. Nat. Neurosci.4, 467-475.
26. Haruno, M., Wolpert, D., and Kawato, M. (1999). Multiple paired forward-inverse models for human motor learning and control. Adv. Neural Inf. Process. Syst.11 31-37.
27. Haruno, M., Wolpert, D., and Kawato, M. (2001). MOSAIC Model for sensorimotor learning and control. Neural Comput.13, 2201-2220. doi: 10.1162/089976601750541778
28. Hirzinger, G., Butterfab, J., Fischer, M., Grebenstein, M., Hähnle, M., Liu, H., et al. (2000). A Mechatronics Approach to the Design of Light-Weight Arms and Multifingered Hands. San Fransisco, CA: ICRA.
29. Hoellinger, T., Petieau, M., Duvinage, M., Castermans, T., Seetharaman, K., Cebolla, A. M., et al. (2013). Biological oscillations for learning walking coordination: dynamic recurrent neural network functionally models physiological central pattern generator. Front. Comput. Neurosci.7:70. doi: 10.3389/fncom.2013.00070
30. Hoffmann, H., and Petkos, G. (2007). "Sensor-assisted adaptive motor control under continuously varying context, " inInternational Conference on Informatics in Control, Automation and Robotics (ICINCO'07)(Angers).
31. Honda, T., Yamazaki, T., Tanaka, S., Nagao, S., and Nishino, T. (2011). Stimulusdependent state transition between synchronized oscillation and randomly repetitive burst in a model cerebellar granular layer. PLoS Comput. Biol. 7:e1002087. doi: 10.1371/journal.pcbi.1002087
32. Houk, J. C., and Barto, A. G. (1992). "Distributed sensorimotor learning, " in Tutorials in Motor Behavior II, eds G. E. Stelmach, J. Requin, et al. (Amsterdam: Elsevier Science Publishers B.V.), 71-100.
33. Houk, J. C., Buckingham, J. T., and Barto, A. G. (1996). Models of cerebellum and motor learning. Behav. Brain Sci.19, 369-383.
34. Howard, I. S., Ingram, J. N., and Wolpert, D. M. (2010). Context-dependent partitioning of motor learning in bimanual movements.J. Neurophysiol. 104, 2082-2091. doi: 10.1152/jn.00299.2010
35. Hwang, E. J., and Shadmehr, R. (2005). Internal models of limb dynamic and the encoding of limb state. J. Neural Eng. 2, 266-278. doi: 10.1088/1741-2560/2/3/S09
36. Ito, M. (1984). The Cerebellum and Neural Control. NewYork, NY:RavenPress.
37. Ito, M. (2008). Control of mental activities by internal models in the cerebellum. Nat. Rev. Neurosci.9, 304-313. doi: 10.1038/nrn2332
38. Ito, M. (2013). Error detection and representation in the olivo-cerebellar system. Front. Neural Circuits.7:1. doi: 10.3389/fncir.2013.00001
39. Izawa, J., Pekny, S.E., Marko, M.K., Haswell, C.C., Shadmehr, R., andMostofsky, S. H. (2012). Motor learning relies on integrated sensory inputs in ADHD, but over-selectively on proprioception in autism spectrum conditions. Autism Res. 5, 124-136. doi: 10.1002/aur.1222
40. Kalveram, K. T., Schinauer, T., Beirle, S., Richter, S., and Jansen-Osmann, P. (2005). Threading neural feedforward into a mechanical spring:how biology exploits physics in limb control. Biol. Cybern.92, 229-240. doi: 10.1007/s00422-005-0542-6
41. Kawato, M., Furukawa, K., and Suzuki, R. (1987). A hierarchical neural-network model for control and learning of voluntary movement. Biol. Cybern.57, 169-185. doi: 10.1007/BF00364149
42. Kawato, M., and Gomi, H. (1992). A computational model of four regions of the cerebellum based on feedback-error learning. Biol. Cybern.68, 95-103. doi: 10.1007/BF00201431
43. Keating, J. G., and Thach, W. T. (1995). Nonclock behavior of inferior olive neurons. Interspike interval of Purkinje cell complex spike discharge in the awake behaving monkey is random.J. Neurophysiol.73, 1329-1340.
44. Kettner, R. E., Mahamud, S., Leung, H., Sittko, N., Houk, J. C., Peterson, B. W., et al. (1997). Prediction of complex two-dimensional trajectories by a cerebellar model of smooth pursuit eye movement.J. Neurophysiol.77, 2115-2130.
45. Kistler, W. M., and Leo Van Hemmen, J. (1999). Delayed reverberation through time windows as a key to cerebellar function. Biol. Cybern.81, 373-380. doi: 10.1007/s004220050569
46. Krichmar, J. L., Ascoli, G. A., Hunter, L., and Olds, J. L. (1997). A model of cerebellar saccadic motor learning using qualitative reasoning. Biol. Artif. Comput. 1240, 133-145.
47. Llinas, R., and Welsh, J. P. (1993). On the cerebellum and motor learning. Curr. Opin. Neurobiol.3, 958-965. doi: 10.1016/0959-4388(93)90168-X
48. Luque, N. R., Garrido, J. A., Carrillo, R. R., Coenen, O. J. M. D., and Ros, E. (2011a). Cerebellar input configuration toward object model abstraction in manipulation tasks. IEEETrans. NeuralNetw. 22, 1321-1328. doi: 10.1109/TNN.2011.2156809
49. Luque, N. R., Garrido, J. A., Carrillo, R. R., Coenen, O. J. M. D., and Ros, E. (2011b). Cerebellarlike corrective model inference engine for manipulation tasks. IEEE Trans. Syst. Man Cybern. B Cybern. 41, 1299-1312. doi: 10.1109/TSMCB.2011.2138693
50. Luque, N. R., Garrido, J. A., Carrillo, R. R., Tolu, S., and Ros, E. (2011c). Adaptive cerebellar spiking model embedded in the control loop: context switching and robustness against noise. Int. J. Neural Syst. 21, 385-401. doi: 10.1142/S0129065711002900
51. Marr, D. (1969). A theory of cerebellar cortex.J. Physiol.202, 437-470.
52. Medina, J., and Mauk, M. (1999). Simulations of cerebellar motor learning: computational analysis of plasticity at the mossy fiber synapse.J. Neurosci. 19, 7140-7151.
53. Medina, J. F., and Mauk, M. D. (2000). Computer simulation of cerebellar information processing. Nat. Neurosci.3(Suppl.), 1205-1211. doi: 10.1038/81486
54. Miall, R. C., Weir, D. J., Wolpert, D. M., and Stein, J. F. (1993). Is the cerebellum a Smith predictor?J. Mot. Behav.25, 203-216. doi: 10.1080/00222895.1993. 9942050
55. Nakano, E., Imamizu, H., Osu, R., Uno, Y., Gomi, H., Yoshioka, T., et al. (1999). Quantitative examinations of internal representations for arm trajectory planning. minimum commanded torque change model. J. Neurophysiol. 81, 2140-2155.
56. Ohtsuki, G., Piochon, C., and Hansel, C. (2009). Climbing fiber signaling and cerebellar gain control. Front. Neurosci.3, 1-11. doi: 10.3389/neuro.03. 004.2009
57. Ostojic, S., and Brunel, N. (2011). From spiking neuron models to linear-nonlinear models. PLoS Comput. Biol.7:e1001056. doi: 10.1371/journal.pcbi.1001056
58. Passot, J.-B., Luque, N. R., and Arleo, A. (2013). Coupling internal cerebellar models enhances online adaptation and supports offline consolidation in sensorimotor tasks. Front. Comput. Neurosci. 7:95. doi: 10.3389/fncom. 2013.00095
59. Petkos, G., Toussaint, M., and Vijayakumar, S. (2006). "Learning multiple models of non-linear dynamics for control under varying contexts, " inProceedings International Conference on Artificial Neural Networks (ICANN '06) (Athens), 1-10.
60. Pugh, J., and Raman, I. (2006). Potentiation of mossy NMDA receptor activation followed by postinhibitory rebound current. Neuron51, 113-123. doi: 10.1016/j.neuron.2006.05.021
61. Racine, R., Wilson, D., Gingell, R., and Sunderland, D. (1986). Long-term potentiation in the interpositus and vestibular nuclei in the rat. Exp. Brain Res.63, 158-162. doi: 10.1007/BF00235658
62. Rokni, D., Llinas, R., and Yarom, Y. (2008). The morpho/functional discrepancy in the cerebellar cortex: looks alone are deceptive. Front. Neurosci.2, 192-198. doi: 10.3389/neuro.01.036.2008
63. Rothganger, F. H., and Anastasio, T. J. (2009). Using input minimization to train a cerebellar model to simulate regulation of smooth pursuit. Biol. Cybern.101, 339-359. doi: 10.1007/s00422-009-0340-7
64. Schweighofer, N., Arbib, M. A., and Kawato, M. (1998). Role of the cerebellum in reaching movements in human. I. Distributed inverse dynamics control. Eur. J. Neurosci.10, 86-94. doi: 10.1046/j.1460-9568.1998.00006.x
65. Schweighofer, N., Doya, K., and Lay, F. (2001). Unsupervised learning of granule cell sparse codes enhances cerebellar adaptive control. Neuroscience103, 35-40. doi: 10.1016/S0306-4522(00)00548-0
66. Sergio, L. E., Hamel-Pâquet, C., and Kalaska, J. F. (2005). Motor cortex neural correlates of output kinematics and kinetics during isometric-force and arm-reaching tasks.J. Neurophysiol.94, 2353-2378. doi: 10.1152/jn.00989.2004
67. Shadmehr, R., Smith, M. A., and Krakauer, J. W. (2010). Error correction, sensory prediction, and adaptation in motor control. Annu. Rev. Neurosci.33, 89-108. doi: 10.1146/annurev-neuro-060909-153135
68. Siciliano, B., and Khatib, O. (2008). Springer Handbook of Robotics. Berlin; Heidelberg: Springer-Verlag. doi: 10.1007/978-3-540-30301-5
69. Smith, M. A., Ghazizadeh, A., and Shadmehr, R. (2006). Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4:e179. doi: 10.1371/journal.pbio.0040179
70. Solinas, S., Nieus, T., and D'Angelo, E. (2010). A realistic large-scale model of the cerebellum granular layer predicts circuit spatio-temporal filtering properties. Front. Cell. Neurosci.4:12. doi: 10.3389/fncel.2010.00012
71. Spitzer, M. (2000). "Hidden layers, " inThe Mind Within the Net: Models of Learning, Thinking And Acting, ed M. Spitzer (Cambridge: The MIT Press), 115-136.
72. Strata, P. (2009). David Marr's theory of cerebellar learning: 40 years later.J. Physiol. 587, 5519-5520. doi: 10.1113/jphysiol.2009.180307
73. Stroeve, S. (1997). A learning feedback and feedforward neuromuscular control model for two degrees of freedom humanarm movements. Hum. Mov. Sci.16, 621-651. doi: 10.1016/S0167-9457(97)00009-2
74. Swain, R. A., Kerr, A. L., and Thompson, R. F. (2011). The cerebellum: a neural system for the study of reinforcement learning. Front. Behav. Neurosci.5:8. doi: 10.3389/fnbeh.2011.00008
75. Todorov, E. (2000). Direct cortical control of muscle activation in voluntary arm movements: a model. Nat. Neurosci.3, 391-398. doi: 10.1038/73964
76. Todorov, E. (2004). Optimality principles in sensorimotor control (review). Nat. Neurosci.7, 907-915. doi: 10.1038/nn1309
77. Turrigiano, G. G., and Nelson, S. B. (2004). Homeostatic plasticity in the developing nervous system. Nat. Rev. Neurosci.5, 97-107. doi: 10.1038/nrn1327
78. Uusisaari, M., and De Schutter, E. (2011). The mysterious microcircuitry of the cerebellar nuclei. J. Physiol. 589, 3441-3457. doi: 10.1113/jphysiol.2010. 201582
79. Uusisaari, M., and Knöpfel, T. (2011). Functional classification of neurons in the mouse lateral cerebellar nuclei. Cerebellum10, 637-646. doi: 10.1007/s12311-010-0240-3
80. Van Der Smagt, P. (2000). Benchmarking cerebellar control. Rob. Auton. Syst.32, 237-251. doi: 10.1016/S0921-8890(00)00090-7
81. Welberg, L. (2009). Cerebellum: an olive branch to two theories. Nat. Rev. Neurosci. 10:468. doi: 10.1038/nrn2674
82. Witney, A. G., Goodbody, S. J., and Wolpert, D. M. (2000). Learning and decay of prediction in object manipulation.J. Neurophysiol.84, 334-343.
83. Wolpert, D. M., and Miall, R. C. (1996). Forward models for physiological motor control. Neural Netw.9, 1265-1279. doi: 10.1016/S0893-6080(96)00035-4
84. Wolpert, W., and Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Netw.11, 1317-1329. doi: 10.1016/S0893-6080(98)00066-5
85. Yamazaki, T., and Tanaka, S. (2005). Neural modeling of an internal clock. Neural Comput.17, 1032-1058. doi: 10.1162/0899766053491850
86. Yamazaki, T., and Tanaka, S. (2007a). The cerebellum as a liquid state machine. Neural Netw.20, 290-297. doi: 10.1016/j.neunet.2007.04.004
87. Yamazaki, T., and Tanaka, S. (2007b). A spiking network model for passage-oftime representation in the cerebellum. Eur. J. Neurosci.26, 2279-2292. doi: 10.1111/j.1460-9568.2007.05837.x
88. Yamazaki, T., and Tanaka, S. (2009). Computational models of timing mechanisms in the cerebellar granular layer. Cerebellum8, 423-432. doi: 10.1007/s12311-009-0115-7
89. Zhang, W., and Linden, W. D. (2006). Long-term depression at the mossy fiber-deep cerebellar nucleus synapse. J. Neurosci. 26, 6935-6944. doi: 10.1523/JNEUROSCI.0784-06.2006