A selectionist account of de novo action learning

Current Opinion in Neurobiology

Volumn 21, Issue 4, 2011, Pages 579-586

  Costa, Rui M ^a  

^a INSTITUTO GULBENKIAN DE CIÊNCIA   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

BASAL GANGLION; BRAIN CORTEX; BRAIN REGION; DE NOVO ACTION LEARNING; HUMAN; LEARNING; LEARNING THEORY; LONG TERM POTENTIATION; MOTOR PERFORMANCE; MOVEMENT PERCEPTION; NERVE CELL PLASTICITY; NERVE CELL STIMULATION; NEUROANATOMY; NEUROFEEDBACK; NONHUMAN; ORIENTING REFLEX; PATTERN RECOGNITION; PRIORITY JOURNAL; RESPONSE TIME; REVIEW; STIMULUS RESPONSE; VENTRAL TEGMENTUM;

ANIMALS; CHOICE BEHAVIOR; DOPAMINE; HUMANS; MOTOR ACTIVITY; MOTOR NEURONS; NEURAL PATHWAYS; PHYSICAL STIMULATION; PROBLEM-BASED LEARNING; REACTION TIME;

EID: 80052944607     PISSN: 09594388     EISSN: 18736882     Source Type: Journal    
DOI: 10.1016/j.conb.2011.05.004     Document Type: Review

References (63)

1. Grillner S. Neurobiological bases of rhythmic motor acts in vertebrates. Science 1985, 228:143-149.
2. Marder E., Bucher D. Central pattern generators and the control of rhythmic movements. Curr Biol 2001, 11:R986-R996.
3. Hull C.L. A Behavior System 1952, Wiley, New York.
4. Sherrington C.S. The Integrative Action of the Nervous System 1906, Charles Scribner's Sons, New York.
5. el Manira A., Shupliakov O., Fagerstedt P., Grillner S. Monosynaptic input from cutaneous sensory afferents to fin motoneurons in lamprey. J Comp Neurol 1996, 369:533-542.
6. Eisen J.S., Marder E. Mechanisms underlying pattern generation in lobster stomatogastric ganglion as determined by selective inactivation of identified neurons. III. Synaptic connections of electrically coupled pyloric neurons. J Neurophysiol 1982, 48:1392-1415.
7. Grillner S., Wallen P. On peripheral control mechanisms acting on the central pattern generators for swimming in the dogfish. J Exp Biol 1982, 98:1-22.
8. Smith J.C., Ellenberger H.H., Ballanyi K., Richter D.W., Feldman J.L. Pre-Botzinger complex: a brainstem region that may generate respiratory rhythm in mammals. Science 1991, 254:726-729.
9. Bucher D., Prinz A.A., Marder E. Animal-to-animal variability in motor pattern production in adults and during growth. J Neurosci 2005, 25:1611-1619.
10. Grace A.A., Bunney B.S. The control of firing pattern in nigral dopamine neurons: single spike firing. J Neurosci 1984, 4:2866-2876.
11. Plenz D., Kital S.T. A basal ganglia pacemaker formed by the subthalamic nucleus and external globus pallidus. Nature 1999, 400:677-682.
12. Changeux J.P., Dehaene S. Neuronal models of cognitive functions. Cognition 1989, 33:63-109.
13. Olveczky B.P., Andalman A.S., Fee M.S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol 2005, 3:e153.
14. Kao M.H., Doupe A.J., Brainard M.S. Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 2005, 433:638-643.
15. Costa R.M. Plastic corticostriatal circuits for action learning: what's dopamine got to do with it?. Ann N Y Acad Sci 2007, 1104:172-191.
16. Edelman G. Neural Darwinism. The Theory of Neuronal Group Selection 1987, Basic Books, New York.
17. Carrillo-Reid L., Tecuapetla F., Tapia D., Hernandez-Cruz A., Galarraga E., Drucker-Colin R., Bargas J. Encoding network states by striatal cell assemblies. J Neurophysiol 2008, 99:1435-1450.
18. Jaidar O., Carrillo-Reid L., Hernandez A., Drucker-Colin R., Bargas J., Hernandez-Cruz A. Dynamics of the Parkinsonian striatal microcircuit: entrainment into a dominant network state. J Neurosci 2010, 30:11326-11336.
19. Carlsson A. Biochemical and pharmacological aspects of Parkinsonism. Acta Neurol Scand Suppl 1972, 51:11-42.
20. Costa R.M., Lin S.C., Sotnikova T.D., Cyr M., Gainetdinov R.R., Caron M.G., Nicolelis M.A. Rapid alterations in corticostriatal ensemble coordination during acute dopamine-dependent motor dysfunction. Neuron 2006, 52:359-369.
21. Burkhardt J.M., Jin X., Costa R.M. Dissociable effects of dopamine on neuronal firing rate and synchrony in the dorsal striatum. Front Integr Neurosci 2009, 3:28.
22. Brown P., Oliviero A., Mazzone P., Insola A., Tonali P., Di Lazzaro V. Dopamine dependency of oscillations between subthalamic nucleus and pallidum in Parkinson's disease. J Neurosci 2001, 21:1033-1038.
23. Bar-Gad I., Elias S., Vaadia E., Bergman H. Complex locking rather than complete cessation of neuronal activity in the globus pallidus of a 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine-treated primate in response to pallidal microstimulation. J Neurosci 2004, 24:7410-7419.
24. Goldberg J.A., Boraud T., Maraton S., Haber S.N., Vaadia E., Bergman H. Enhanced synchrony among primary motor cortex neurons in the 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine primate model of Parkinson's disease. J Neurosci 2002, 22:4639-4653.
25. Goldberg J.A., Rokni U., Boraud T., Vaadia E., Bergman H. Spike synchronization in the cortex/basal-ganglia networks of Parkinsonian primates reflects global dynamics of the local field potentials. J Neurosci 2004, 24:6003-6010.
26. Raz A., Frechter-Mazar V., Feingold A., Abeles M., Vaadia E., Bergman H. Activity of pallidal and striatal tonically active neurons is correlated in mptp-treated monkeys but not in normal monkeys. J Neurosci 2001, 21:RC128.
27. Cruz A.V., Mallet N., Magill P.J., Brown P., Averbeck B.B. Effects of dopamine depletion on network entropy in the external globus pallidus. J Neurophysiol 2009, 102:1092-1102.
28. Gale S.D., Perkel D.J. A basal ganglia pathway drives selective auditory responses in songbird dopaminergic neurons via disinhibition. J Neurosci 2010, 30:1027-1037.
29. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
30. Goard M., Dan Y. Basal forebrain activation enhances cortical coding of natural scenes. Nat Neurosci 2009, 12:1444-1449.
31. Hahnloser R.H., Kozhevnikov A.A., Fee M.S. An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature 2002, 419:65-70.
32. Costa R.M., Cohen D., Nicolelis M.A. Differential corticostriatal plasticity during fast and slow motor skill learning in mice. Curr Biol 2004, 14:1124-1134.
33. Komiyama T., Sato T.R., O'Connor D.H., Zhang Y.X., Huber D., Hooks B.M., Gabitto M., Svoboda K. Learning-related fine-scale specificity imaged in motor cortex circuits of behaving mice. Nature 2010, 464:1182-1186.
34. Ganguly K., Carmena J.M. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol 2009, 7:e1000153.
35. Dombeck D.A., Graziano M.S., Tank D.W. Functional clustering of neurons in motor cortex determined by cellular resolution imaging in awake behaving mice. J Neurosci 2009, 29:13751-13760.
36. Dragoi G., Tonegawa S. Preplay of future place cell sequences by hippocampal cellular assemblies. Nature 2011, 469:397-401.
37. Yin H.H., Mulcare S.P., Hilario M.R., Clouse E., Holloway T., Davis M.I., Hansson A.C., Lovinger D.M., Costa R.M. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat Neurosci 2009, 12:333-341.
38. Choi W.Y., Balsam P.D., Horvitz J.C. Extended habit training reduces dopamine mediation of appetitive response expression. J Neurosci 2005, 25:6729-6733.
39. Ljungberg T., Apicella P., Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 1992, 67:145-163.
40. Rioult-Pedotti M.S., Friedman D., Donoghue J.P. Learning-induced LTP in neocortex. Science 2000, 290:533-536.
41. Jackson A., Mavoori J., Fetz E.E. Long-term motor cortex plasticity induced by an electronic neural implant. Nature 2006, 444:56-60.
42. Reynolds J.N., Hyland B.I., Wickens J.R. A cellular mechanism of reward-related learning. Nature 2001, 413:67-70.
43. Benecke R., Rothwell J.C., Dick J.P., Day B.L., Marsden C.D. Disturbance of sequential movements in patients with Parkinson's disease. Brain 1987, 110(Pt 2):361-379.
44. Agostino R., Berardelli A., Formica A., Accornero N., Manfredi M. Sequential arm movements in patients with Parkinson's disease, Huntington's disease and dystonia. Brain 1992, 115(Pt 5):1481-1495.
45. Phillips J.G., Chiu E., Bradshaw J.L., Iansek R. Impaired movement sequencing in patients with Huntington's disease: a kinematic analysis. Neuropsychologia 1995, 33:365-369.
46. Fujii N., Graybiel A.M. Representation of action sequence boundaries by macaque prefrontal cortical neurons. Science 2003, 301:1246-1249.
47. Barnes T.D., Mao J.B., Hu D., Kubota Y., Dreyer A.A., Stamoulis C., Brown E.N., Graybiel A.M. Advance-cueing produces enhanced action-boundary patterns of spike activity in the sensorimotor striatum. J Neurophysiol 2011.
48. Meyer-Luehmann M., Thompson J.F., Berridge K.C., Aldridge J.W. Substantia nigra pars reticulata neurons code initiation of a serial pattern: implications for natural action sequences and sequential disorders. Eur J Neurosci 2002, 16:1599-1608.
49. Jin X., Costa R.M. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 2010, 466:457-462.
50. Dickinson A. Actions and habits: the development of behavioural autonomy. Philos Trans R Soc Lond 1985, B308:67-78.
51. Adams C. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q J Exp Psychol Comp Physiol Psychol 1982, 34B:77-98.
52. Adams C., Dickinson A. Actions and habits: variations in associative representations during instrumental learning. Information Processing in Animals: Memory Mechanisms 1981, 143-166. N. Spear, R. Miller (Eds.).
53. Yin H.H., Knowlton B.J., Balleine B.W. Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur J Neurosci 2005, 22:505-512.
54. Yin H.H., Ostlund S.B., Knowlton B.J., Balleine B.W. The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 2005, 22:513-523.
55. Balleine B.W., Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 1998, 37:407-419.
56. Corbit L.H., Muir J.L., Balleine B.W. Lesions of mediodorsal thalamus and anterior thalamic nuclei produce dissociable effects on instrumental conditioning in rats. Eur J Neurosci 2003, 18:1286-1294.
57. Yin H.H., Knowlton B.J., Balleine B.W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 2004, 19:181-189.
58. Killcross S., Coutureau E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex 2003, 13:400-408.
59. Hilario M.R.F., Clouse E., Yin H.H., Costa R.M. Endocannabinoid signaling is critical for habit formation. Front in Integr Neurosci 2007, 1:6. 10.3389/neuro.07/006.2007.
60. Hilario M.R., Costa R.M. High on habits. Front Neurosci 2008, 2:208-217.
61. Yu C., Gupta J., Chen J.F., Yin H.H. Genetic deletion of A2A adenosine receptors in the striatum selectively impairs habit formation. J Neurosci 2009, 29:15100-15103.
62. Gremel C.M., Costa R.M. Orbitofrontal and striatal circuits encode the shift between goal-directed and habitual actions. Preceedings of Cosyne 2011, Salt Lake City, Utah, p. 85.
63. Eusebio A., Thevathasan W., Doyle Gaynor L., Pogosyan A., Bye E., Foltynie T., Zrinzo L., Ashkan K., Aziz T., Brown P. Deep brain stimulation can suppress pathological synchronisation in parkinsonian patients. J Neurol Neurosurg Psychiatry 2010.