The cerebellum: A neural system for the study of reinforcement learning

Frontiers in Behavioral Neuroscience

Volumn , Issue MARCH, 2011, Pages

  Swain, Rodney A ^a   Kerr, Abigail L ^b   Thompson, Richard F ^c  

^a UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^c UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Classical conditioning; Inferior olive; Reinforcer; Reward

Indexed keywords

ARTICLE; BRAIN DEPTH STIMULATION; BRAIN REGION; CEREBELLUM; CONDITIONING; ELECTROSTIMULATION; EYELID REFLEX; FRONTAL LOBE; INFERIOR OLIVE; LEARNING; LOCOMOTION; MOTIVATION; MOTOR ACTIVITY; NEUROSCIENCE; NONHUMAN; OCCIPITAL LOBE; PARIETAL LOBE; PREFRONTAL CORTEX; PSYCHOLOGY; RABBIT; REINFORCEMENT; SENSORIMOTOR FUNCTION; SENSORY STIMULATION; TEMPORAL LOBE; THALAMUS VENTRAL NUCLEUS;

EID: 84855348337     PISSN: 16625153     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnbeh.2011.00008     Document Type: Article

References (58)

1. Allen, G., and Tsukahara, N. (1974). Cerebrocerebellar communication systems. Physiol. Rev. 54, 957-1006.
2. Andreasen, N. C., and Pierson, R. (2008). The role of the cerebellum in schizophrenia. Biol. Psychiatry 64, 81-88.
3. Bauer, D., Kerr, A., and Swain, R. (2011). Cerebellar dentate nuclei lesions reduce motivation. Neurobiol. Learn. Mem. 95, 166-175.
4. Bauman, M., and Kemper, T. L. (1985). Histoanatomic observations of the brain in early infantile autism. Neurology 35, 866-874.
5. Brogden, W. J., and Gantt, W. H. (1942). Intraneural conditioning. Arch. Neurol. Psychiatry 48, 437-455.
6. Caston, J., Chianale, C., Delhaye-Bouchaud, N., and Mariani, J. (1998). Role of the cerebellum in exploration behavior. Brain Res. 808, 232-237.
7. Courchesne, E., Townsend, J., Akshoomoff, N. A., Saitoh, O., Yeung-Courchesne, R., Lincoln, A. J., James, H. E., Haas, R. H., Schriebman, L., and Lau, L. (1994). Impairment in shifting attention in autistic and cerebellar patients. Behav. Neurosci. 108, 848-865.
8. D'Agata, V., Drago, F., Serapide, F., and Cicirata, F. (1993). Effects of cerebellectomy on motivation-related behavior: a time-course study. Physiol. Behav. 53, 173-176.
9. Dayan, P., and Balleine, B. W. (2002). Reward, motivation, and reinforcement learning. Neuron 36, 285-298.
10. Eccles, J. C., Ito, M., and Szentagothai, J. (1967). The Cerebellum as a Neuronal Machine. New York: Springer-Verlag.
11. Fatemi, S. H., Halt, A. R., Realmuto, G., Earle, J., Kist, D. A., Thuras, P., and Merz, A. (2002). Purkinje cell size is reduced in cerebellum of patients with autism. Cell. Mol. Neurobiol. 22, 171-175.
12. Foy, M. R., and Thompson, R. F. (1986). Single-unit analysis of Purkinje cell discharge in classically conditioned and untrained rabbits. Soc. Neurosci. Abstr. 12, 518.
13. Gormezano, I., Kehoe, E. J., and Marshall, B. S. (1983). "Twenty years of classical conditioning research with the rabbit," in Progress in Psychobiology and Physiological Psychology, Vol. 10, eds J. M. Sprague and A. N. Epstein (New York, NY: Academic Press), 197-275.
14. Hodos, W. (1961). Progressive ratio as a measure of reward strength. Science 134, 943-944.
15. Hodos, W., and Kalman, G. (1963). Effects of increment size and reinforcer volume on progressive ratio performance. J. Exp. Anal. Behav. 6, 387-392.
16. Ichinohe, N., Mori, F., and Shoumura, K. (2000). A di-synaptic projection from the lateral cerebellar nucleus to the laterodorsal part of the striatum via the central lateral nucleus of the thalamus in the rat. Brain Res. 880, 191-197.
17. Jiang, M. C., Alheid, G. F., Nunzi, M. G., and Houk, J. C. (2002). Cerebellar input to magnocellular neurons in the red nucleus of the mouse: synaptic analysis in horizontal brain slices incorporating cerebellarrubral pathways. Neuroscience 110, 105-121.
18. Kamin, L. J. (1968). "Attention-like processes in classical conditioning," in Miami Symposium on the Prediction of Behavior: Aversive Stimulation, ed. M. R. Jones (Miami, FL: University of Miami Press), 9-33.
19. Kelley, A. E., Smith-Roe, S. L., and Holahan, M. R. (1997). Response-reinforcement learning is dependent on N-methyl-d-aspartate receptor activation in the nucleus accumbens core. Proc. Natl. Acad. Sci. U.S.A. 94, 12174-12179.
20. Kelly, R. M., and Strick, P. L. (2003). Cerebellar loops with motor cortex and prefrontal cortex of a nonhuman primate. J. Neurosci. 23, 8432-8444.
21. Kettner, R. E., Mahamud, S., Leung, H. C., Sitkoff, N., Houk, J. C., Peterson, B. W., and Barto, A. G. (1997). Prediction of complex two-dimensional trajectories by a cerebellar model of smooth pursuit eye movement. J. Neurophysiol. 77, 2115-2130.
22. Kim, J. J., Krupa, D. J., and Thompson, R. F. (1998). Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science 279, 570-573.
23. Kitazawa, S., Kimura, T., and Yin, P. B. (1998). Cerebellar complex spikes encode both destinations and errors in arm movement. Nature 392, 494-497.
24. Lalonde, R., Manseau, M., and Botez, M. I. (1988a). Spontaneous alternation and exploration in staggerer mutant mice. Behav. Brain Res. 27, 273-276.
25. Lalonde, R., Manseau, M., and Botez, M. I. (1988b). Spontaneous alternation and exploration in weaver mutant mice. Behav. Brain Res. 32, 111-114.
26. Lalonde, R., and Strazielle, C. (2003). Motor coordination, exploration, and spatial learning in a natural mouse mutation (nervous) with Purkinje cell degeneration. Behav. Genet. 33, 59-66.
27. Leiner, H. C., Leiner, A. L., and Dow, R. S. (1986). Does the cerebellum contribute to mental skills? Behav. Neurosci. 100, 443-454.
28. Leiner, H. C., Leiner, A. L., and Dow, R. S. (1987). Cerebro-cerebellar learning loops in apes and humans. Ital. J. Neurol. Sci. 8, 425-436.
29. Loucks, R. B. (1935). The experimental delimitation of neural structures essential for learning: the attempt to condition striped muscle response with faradization of the sigmoid gyri. J. Psychol. 1, 5-44.
30. Martin, P., and Albers, M. (1995). Cerebellum and schizophrenia: a selective review. Schizophr. Bull. 21, 241-250.
31. Mauk, M. D., Steinmetz, J. E., and Thompson, R. F. (1986). Classical conditioning using stimulation of the inferior olive as the unconditioned stimulus. Proc. Natl. Acad. Sci. U.S.A. 83, 5349-5353.
32. Mauk, M. D., and Thompson, R. F. (1987). Retention of classically conditioned eyelid responses following acute decerebration. Brain Res. 403, 89-95.
33. McCormick, D. A., Steinmetz, J. E., and Thompson, R. F. (1985). Lesions of the inferior olivary complex cause extinction of the classically conditioned eyelid response. Brain Res. 359, 120-130.
34. Middleton, F. A., and Strick, P. L. (1997). Cerebellar output channels. Neurobiology 41, 61-81.
35. Middleton, F. A., and Strick, P. L. (2000). Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Res. Rev. 31, 236-250.
36. Middleton, F. A., and Strick, P. L. (2001). Cerebellar projections to the prefrontal cortex of the primate. J. Neurosci. 21, 700-712.
37. Mis, R. W., and Moore, J. W. (1973). Effects of preacquisition US exposure on classical conditioning of the rabbit's nictitating membrane response. Learn. Motiv. 4, 108-114.
38. Moore, J. W., and Blazis, D. E. J. (1989). "Stimulation of a classically conditioned response: a cerebellar network implementation of the Sutton-BartoDesmond model," in Neural Models of Plasticity, eds J. H. Byrne and W. O. Berry (New York, NY: Academic Press), 187-207.
39. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., and Dolan, R. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452-454.
40. Percheron, G., Francois, C., Talbi, B., Yelnik, J., and Fenelon G. (1996). The primate motor thalamus. Brain Res. Rev. 22, 93-181.
41. Rapaport, M., van Reekum, R., and Mayberg, H. (2000). The role of the cerebellum in cognition and behavior: a selective review. J. Neuropsychiatry Clin. Neurosci. 12, 193-198.
42. Rapoport, M. (2001). The cerebellum in psychiatric disorders. Int. Rev. Psychiatry 13, 295-301.
43. Rescorla, R. A. (1969). Pavlovian conditioned inhibition. Psychol. Bull. 72, 77-74.
44. Rescorla, R. A., and Wagner, A. R. (1972). "A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement," in Classical Conditioning II: Current Research Theory, eds A. H. Black and W. F. Prokasy (New York, NY: Appleton-Century-Crofts), 64-99.
45. Schmahmann, J. D. (2001). The cerebellar cognitive affective syndrome: clinical correlations of the dysmetria of thought hypothesis. Int. Rev. Psychiatry 13, 313-322.
46. Schmahmann, J. D., and Sherman, J. C. (1997). Cerebellar cognitive affective syndrome. Int. Rev. Neurobiol. 41, 433-440.
47. Sears, L. L., and Steinmetz, J. E. (1991). Dorsal accessory inferior olive activity diminishes during acquisition of the rabbit classically conditioned eyelid response. Brain Res. 545, 114-122.
48. Shinkman, P. G., Swain, R. A., and Thompson, R. F. (1996). Classical conditioning with electrical stimulation of cerebellum as both conditioned and unconditioned stimulus. Behav. Neurosci. 110, 914-921.
49. Skjoldager, P., Pierre, P. J., and Mittleman, G. (1993). Reinforcer magnitude and progressive ratio responding in the rat: effects of increased effort, prefeeding, and extinction. Learn. Motiv. 24, 303-343.
50. Strick, P. L., Dum, R. P., and Fiez, J. A. (2009). Cerebellum and nonmotor function. Annu. Rev. Neurosci. 32, 413-434.
51. Swain, R. A., Shinkman, P. G., Nordholm, A. F., and Thompson, R. F. (1992). Cerebellar stimulation as an unconditioned stimulus in classical conditioning. Behav. Neurosci. 106, 739-750.
52. Swain, R. A., Shinkman, P. G., Thompson, J. K., Grethe, J. S., and Thompson, R. F. (1999). Essential neuronal pathways for reflex and conditioned response initiation in an intracerebellar stimulation paradigm and the impact of unconditioned stimulus preexposure on learning rate. Neurobiol. Learn. Mem. 71, 167-193.
53. Swain, R. A., and Thompson, R. F. (1993). In search of engrams. Ann. N. Y. Acad. Sci. 702, 27-39.
54. Thoma, P, Bellebaum, C., Koch, B., Schwarz, M., and Daum, I. (2008). The cerebellum is involved in rewardbased reversal learning. Cerebellum 7, 433-443.
55. Thompson, R. F. (1986). The neurobiology of learning and memory. Science 233, 941-947.
56. Thompson, R. F., and Gluck, M. A. (1991). "Brain substrates of basic associative learning and memory," in Perspectives on Cognitive Neuroscience, eds R. G. Lister and H. J. Weingartner (NewYork, NY: Oxford University Press), 25-45.
57. Thompson, R. F., Swain, R., Clark, R., and Shinkman, P. (2000). Intracerebellar conditioning Brogden and Gantt revisited. Behav. Brain Res. 110, 3-11.
58. Wagner, A. R., Thomas, E., and Norton, T. (1967). Conditioning with electrical stimulation of motor cortex: evidence of a possible source of emotion. J. Comp. Physiol. Psychol. 64, 191-199.