Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances

Journal of Neurophysiology

Volumn 98, Issue 4, 2007, Pages 2038-2057

  Fiete, Ila R ^a,b,e   Fee, Michale S ^c,d   Seung, H Sebastian ^d  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^d MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^e CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL BEHAVIOR; ARTICLE; BRAIN REGION; ERROR; LEARNING; MATHEMATICAL MODEL; MEMORY; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NERVE CONDUCTION; PREDICTION; PRIORITY JOURNAL; REINFORCEMENT; SINGING; SONGBIRD; SPIKE; SYNAPTIC TRANSMISSION; TIME;

ALGORITHMS; ANIMALS; FINCHES; LEARNING; MODELS, NEUROLOGICAL; MOTOR NEURONS; NEURAL CONDUCTION; NEURONAL PLASTICITY; NEURONS; PITCH PERCEPTION; REINFORCEMENT (PSYCHOLOGY); SYNAPSES; VOCALIZATION, ANIMAL;

EID: 35348872545     PISSN: 00223077     EISSN: 15221598     Source Type: Journal    
DOI: 10.1152/jn.01311.2006     Document Type: Article

References (74)

1. Alon U, Surette M, Barkai N, Leibler S. Robustness in bacterial chemotaxis. Nature 397: 168-171, 1999.
2. Barto AG, Anandan P. Pattern-recognizing stochastic learning automata. IEEE Trans Syst Man Cybern 15: 360-375, 1985.
3. Barto AG, Sutton RS, Anderson CW. Neuronlike adaptive elements that can solve difficult learning control-problems. IEEE Trans Syst Man Cybern 13: 834-846, 1983.
4. Baxter J, Bartlett P. Infinite-horizon policy-gradient estimation. J Artif Intell Res 15: 319-350, 2001.
5. Beckers G, Suthers R, ten Cate C. Pure-tone birdsong by resonance filtering of harmonic overtones. Proc Natl Acad Sci USA 100: 7372-7376, 2003.
6. Bottjer S, Miesner E, Arnold A. Forebrain lesions disrupt development but not maintenance of song in passerine birds. Science 224: 901-903, 1984.
7. Brainard M, Doupe A. Auditory feedback in learning and maintenance of vocal behaviour. Nat Rev Neurosci 1: 31-40, 2000a.
8. Brainard M, Doupe A. Interruption of a basal ganglia-forebrain circuit prevents plasticity of learned vocalizations. Nature 404: 762-766, 2000b.
9. Brainard M, Doupe A. What songbirds teach us about learning. Nature 417: 351-358, 2002.
10. Canady R, Burd G, Devoogd T, Nottebohm F. Effect of testosterone on input received by an identified neuron type of the canary song system: a Golgi/electron microscopy/degeneration study. J Neurosci 8: 3770-3784, 1988.
11. Cauwenberghs G. A fast stochastic error-descent algorithm for supervised learning and optimization. In: Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 1993, vol. 5, p. 244-251.
12. Dayan P. Reinforcement comparison. In: Proceedings of the 1990 Connectionist Models Summer School, edited by Touretzky DS, Elman JL, Sejnowski TJ, Hinton GE. San Mateo, CA: Morgan Kaufmann, 1990, p. 45-51.
13. Dembo A, Kailath T. Model-free distributed learning. IEEE Trans Neural Netw 1: 58-70, 1990.
14. Doupe A. A neural circuit specialized for vocal learning. Curr Opin Neurobiol 3: 104-111, 1993.
15. Doya K, Sejnowski T. A computational model of birdsong learning by auditory experience and auditory feedback. In: Central Auditory Processing and Neural Modeling, edited by Brugge J, Poon P. New York: Plenum, 1998, p. 77-88.
16. Doya K, Sejnowski T. A computational model of avian song learning. In: The New Cognitive Neurosciences, edited by Gazzaniga M. Cambridge, MA: MIT Press, 2000, p. 469-482.
17. Elemans C, Larsen O, Hoffmann M, van Leeuwen J. Quantitative modelling of the biomechanics of the avian syrinx. Anim Biol 53: 183-193, 2004.
18. Farries M. The avian song system in comparative perspective. Ann NY Acad Sci 1016: 61-76, 2004.
19. Fee M, Kozhevnikov A, Hahnloser R. Neural mechanisms of vocal sequence generation in the songbird. Ann NY Acad Sci 1016: 153-170, 2004.
20. Fee M, Shraiman B, Pesaran B, Mitra P. The role of nonlinear dynamics of the syrinx in the vocalizations of a songbird. Nature 395: 67-71, 1998.
21. Fiete I, Hahnloser R, Fee M, Seung H. Temporal sparseness of the premotor drive is important for rapid learning in a neural network model of birdsong. J Neurophysiol 92: 2274-2282, 2004.
22. Fiete I, Seung H. Gradient learning in spiking neural networks by dynamic perturbation of conductances. Phys Rev Lett 97: 048104, 2006.
23. Fletcher NH. Bird song - a quantitative acoustic model. J Theor Biol 135: 455-481, 1988.
24. Goller F, Larsen O. A new mechanism of sound generation in songbirds. Proc Natl Acad Sci USA 94: 14787-14791, 1997.
25. Gurney M. Hormonal control of cell form and number in the zebra finch song system. J Neurosci 1: 658-673, 1981.
26. Hahnloser R, Kozhevnikov A, Fee M. An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature 419: 65-70, 2002.
27. Herrmann K, Arnold A. The development of afferent projections to the robust archistriatal nucleus in male zebra finches: a quantitative electron microscopic study. J Neurosci 11: 2063-2074, 1991.
28. Hessler N, Doupe A. Singing-related neural activity in a dorsal forebrain-basal ganglia circuit of adult zebra finches. J Neurosci 19: 10461-10481, 1999a.
29. Hessler N, Doupe A. Social context modulates singing-related neural activity in the songbird forebrain. Nat Neurosci 2: 209-211, 1999b.
30. Immelmann K. Song development in the zebra finch and in other estrildid finches. In: Bird Vocalizations, edited by Hinde RA. New York: Cambridge Univ. Press, 1969, p. 61-74.
31. Johnson F, Soderstrom K, Whitney O. Quantifying song bout production during zebra finch sensory-motor learning suggests a sensitive period for vocal practice. Behav Brain Res 131: 57-65, 2002.
32. Kao M, Doupe A, Brainard M. Contributions of an avian basal ganglia forebrain circuit to real-time modulation of song. Nature 433: 638-643, 2005.
33. Kittelberger J, Mooney R. Lesions of an avian forebrain nucleus that disrupt song development alter synaptic connectivity and transmission in the vocal premotor pathway. J Neurosci 19: 9385-9398, 1999.
34. Konishi M. The role of auditory feedback in the control of vocalization in the white-crowned sparrow. Z Tierpsychol 22: 770-783, 1965.
35. Leonardo A. Experimental test of the birdsong error-correction model. Proc Natl Acad Sci USA 101: 16935-16940, 2004.
36. Leonardo A, Fee M. Ensemble coding of vocal control in birdsong. J Neurosci 25: 652-661, 2005.
37. Leslie K, Nelson S, Turrigiano G. Postsynaptic depolarization scales quantal amplitude in cortical pyramidal neurons. J Neurosci 21: RC170, 2001.
38. Margoliash D. Evaluating theories of bird song learning: implications for future directions. J Comp Physiol A Sens Neural Behav Physiol 188: 851-866, 2002.
39. Marler P, Tamura M. Culturally transmitted patterns of vocal behavior in sparrows. Science 146: 1483-1486, 1964.
40. Mello C, Pinaud R, Ribeiro S. Noradrenergic system of the zebra finch brain: immunocytochemical study of dopamine-beta-hydroxylase. J Comp Neurol 400: 207-228, 1998.
41. Mooney R. Synaptic basis for developmental plasticity in a birdsong nucleus. J Neurosci 12: 2464-2477, 1992.
42. Nick T, Konishi M. Neural auditory selectivity develops in parallel with song. J Neurobiol 62: 469-481, 2005.
43. Nottebohm F, Stokes T, Leonard C. Central control of song in the canary, Serinus canarius. J Comp Neurol 165: 457-486, 1976.
44. Nowicki S. Vocal tract resonances in oscine bird sound production: evidence from birdsongs in a helium atmosphere. Nature 325: 53-55, 1987.
45. Olveczky B, Andalman A, Fee M. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol 3: e153, 2005.
46. Perkel D. Origin of the anterior forebrain pathway. Ann NY Acad Sci 1016: 736-748, 2004.
47. Price P. Developmental determinants of structure in zebra finch song. J Comp Physiol Psychol 93: 268-277, 1979.
48. Rabiner L, Schafer R. Digital Processing of Speech Signals. Englewood Cliffs, NJ: Prentice-Hall, 1978.
49. Reiner A, Perkel D, Mello C, Jarvis E. Songbirds and the revised avian brain nomenclature. Ann NY Acad Sci 1016: 77-108, 2004.
50. Sakaguchi H, Saito N. Developmental changes in axon terminals visualized by immunofluorescence for the growth-associated protein, gap-43, in the robust nucleus of the archistriatum of the zebra finch. Brain Res Dev Brain Res 95: 245-251, 1996.
51. Scharff C, Nottebohm F. A comparative study of the behavioral deficits following lesions of various parts of the zebra finch song system: implications for vocal learning. J Neurosci 11: 2896-2913, 1991.
52. Seung H. Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40: 1063-1073, 2003.
53. Shea S, Margoliash D. Basal forebrain cholinergic modulation of auditory activity in the zebra finch song system. Neuron 40: 1213-1226, 2003.
54. Simpson H, Vicario D. Brain pathways for learned and unlearned vocalizations differ in zebra finches. J Neurosci 10: 1541-1556, 1990.
55. Solis M, Perkel D. Noradrenergic modulation of activity in a vocal control nucleus in vitro. J Neurophysiol 95: 2265-2276, 2006.
56. Stark H, Scheich H. Dopaminergic and serotonergic neurotransmission systems are differentially involved in auditory cortex learning: a long-term microdialysis study of metabolites. J Neurochem 68: 691-697, 1997.
57. Stark L, Perkel D. Two-stage, input-specific synaptic maturation in a nucleus essential for vocal production in the zebra finch. J Neurosci 19: 9107-9116, 1999.
58. Suaud-Chagny M, Dugast C, Chergui K, Msghina M, Gonon F. Uptake of dopamine released by impulse flow in the rat mesolimbic and striatal systems in vivo. J Neurochem 65: 2603-2611, 1995.
59. Suthers R, Goller F, Pytte C. The neuromuscular control of birdsong. Philos Trans R Soc Lond 354: 927-939, 1999.
60. Suthers R, Margoliash D. Motor control of birdsong. Curr Opin Neurobiol 12: 684-690, 2002.
61. Tchernichovski O, Lints T, Mitra P, Nottebohm F. Vocal imitation in zebra finches is inversely related to model abundance. Proc Natl Acad Sci USA 96: 12901-12904, 1999.
62. Tchernichovski O, Mitra P, Lints T, Nottebohm F. Dynamics of the vocal imitation process: how a zebra finch learns its song. Science 291: 2564-2569, 2001.
63. Titze I. The physics of small-amplitude oscillation of the vocal folds. J Acoust Soc Am 83: 1536-1552, 1988.
64. Troyer T, Bottjer S. Birdsong: models and mechanisms. Curr Opin Neurobiol 11: 721-726, 2001.
65. Troyer T, Doupe A. An associational model of birdsong sensorimotor learning. I. Efference copy and the learning of song syllables. J Neurophysiol 84: 1204-1223, 2000.
66. Warner RW. The anatomy of the syrinx in passerine birds. J Zool (Lond) 168: 381-393, 1972.
67. Werfel J, Xie X, Seung HS. Learning curves for stochastic gradient descent in linear feedforward networks. In: Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2003, vol. 16, p. 1197-1204.
68. Werfel J, Xie X, Seung HS. Learning curves for stochastic gradient descent in linear feedforward network. Neural Comput 17: 12699-12718, 2005.
69. Wild J. Descending projections of the songbird nucleus robustus archistriatalis. J Comp Neurol 338: 225-241, 1993.
70. Wild J. Neural pathways for the control of birdsong production. J Neurobiol 33: 653-670, 1997.
71. Wild J. Functional neuroanatomy of the sensorimotor control of singing. Ann NY Acad Sci 1016: 438-462, 2004.
72. Williams R. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learn 8: 229-256, 1992.
73. Xie X, Seung H. Learning in neural networks by reinforcement of irregular spiking. Phys Rev E Stat Nonlin Soft Matter Phys 69: 041909, 2004.
74. Yu A, Margoliash D. Temporal hierarchical control of singing in birds. Science 273: 1871-1875, 1986.