The hippocampal-VTA loop: The role of novelty and motivation in controlling the entry of information into long-term memory

Intrinsically Motivated Learning in Natural and Artificial Systems

Volumn 9783642323751, Issue , 2013, Pages 235-254

  Otmakhova, Nonna ^a   Duzel, Emrah ^b,c,d   Deutch, Ariel Y ^e   Lisman, John ^a  

^a BRANDEIS UNIVERSITY   (United States)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c OTTO VON GUERICKE UNIVERSITY   (Germany)

^d GERMAN CENTER FOR NEURODEGENERATIVE DISEASES DZNE   (Germany)

^e VANDERBILT UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AMINES; NEURONS; NEUROPHYSIOLOGY;

DEPENDENT PROCESS; DOPAMINE; DOPAMINE NEURONS; LONG TERM MEMORY; NOVEL INFORMATION; PROPERTY; SUPERIOR COLLICULUS;

BRAIN;

EID: 84882750935     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-3-642-32375-1_10     Document Type: Chapter

References (100)

1. Adcock, R., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., Gabrieli, J.: Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron 50, 507-517 (2006)
2. Arsenault, M., Parent, A., Seguela, P., Descarries, L.: Distribution and morphological characteristics of dopamine-immunoreactive neurons in the midbrain of the squirrel monkey (saimiri sciureus). J. Comp. Neurol. 267(4), 489-506 (1988)
3. Ballarini, F., Moncada, D., Martinez, M., Alen, N., Viola, H.: Behavioral tagging is a general mechanism of long-term memory formation. Proc. Natl. Acad. Sci. U. S. A. 106(34), 14599-604 (2009)
4. Bethus, I., Tse, D., Morris, R.: Dopamine and memory: Modulation of the persistence of memory for novel hippocampal nmda receptor-dependent paired associates. J. Neurosci. 30(5), 1610-1618 (2010)
5. Borgkvist, A., Malmlof, T., Feltmann, K., Lindskog, M., Schilstrom, B.: Dopamine in the hippocampus is cleared by the norepinephrine transporter. Int. J. Neuropsychopharmacol. 1-10 (2011)
6. Bromberg-Martin, E., Matsumoto, M., Hikosaka, O.: Distinct tonic and phasic anticipatory activity in lateral habenula and dopamine neurons. Neuron 67(1), 144-155 (2010a)
7. Bromberg-Martin, E., Matsumoto, M., Hikosaka, O.: Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron 68(5), 815-834 (2010b)
8. Bromberg-Martin, E., Matsumoto, M., Nakahara, H., Hikosaka, O.: Multiple timescales of memory in lateral habenula and dopamine neurons. Neuron 67(3), 499-510 (2010c)
9. Brown, M., Henny, P., Bolam, J., Magill, P.: Activity of neurochemically heterogeneous dopamin-ergic neurons in the substantia nigra during spontaneous and driven changes in brain state. J. Neurosci. 29(9), 2915-2925 (2009)
10. Bunzeck, N., Doeller, C., Fuentemilla, L., Dolan, R., Duzel, E.: Reward motivation accelerates the onset of neural novelty signals in humans to 85 milliseconds. Curr. Biol. 19(15), 1294-1300 (2009)
11. Bunzeck, N., Duzel, E.: Absolute coding of stimulus novelty in the human substantia nigra/vta. Neuron 51(3), 369-379 (2006)
12. Bunzeck, N., Schutze, H., Stallforth, S., Kaufmann, J., Duzel, S., Heinze, H., Duzel, E.: Mesolimbic novelty processing in older adults. Cereb. Cortex 17(12), 2940-2948 (2007)
13. Cheng, S., Frank, L.: New experiences enhance coordinated neural activity in the hippocampus. Neuron 57(2), 303-313 (2008)
14. Coulter, C., Happe, H., Bergman, D., Murrin, L.C.: Localization and quantification of the dopamine transporter: Comparison of [3h]win 35,428 and [125i]rti-55. Brain Res. 690(2), 217-224 (1995)
15. Cragg, S., Nicholson, C., Kume-Kick, J., Tao, L., Rice, M.: Dopamine-mediated volume transmission in midbrain is regulated by distinct extracellular geometry and uptake. J. Neurophysiol. 85(4), 1761-1771 (2001)
16. Csicsvari, J., O'Neill, J., Allen, K., Senior, T.: Place-selective firing contributes to the reverseorder reactivation of ca1 pyramidal cells during sharp waves in open-field exploration. Eur. J. Neurosci. 26(3), 704-716 (2007)
17. Curet, O., Dennis, T., Scatton, B.: The formation of deaminated metabolites of dopamine in the locus coeruleus depends upon noradrenergic neuronal activity. Brain Res. 335(2), 297-301 (1985)
18. Deadwyler, S., Hampson, R.: Differential but complementary mnemonic functions of the hippocampus and subiculum. Neuron 42(3), 465-476 (2004)
19. Deutch, A., Goldstein, M., Roth, R.: Activation of the locus coeruleus induced by selective stimulation of the ventral tegmental area. Brain Res. 363(2), 307-314 (1986)
20. Devoto, P., Flore, G., Pani, L., Gessa, G.: Evidence for co-release of noradrenaline and dopamine from noradrenergic neurons in the cerebral cortex. Mol. Psychiatry 6(6), 657-664 (2001)
21. Devoto, P., Flore, G., Saba, P., Fa, M., Gessa, G.: Co-release of noradrenaline and dopamine in the cerebral cortex elicited by single train and repeated train stimulation of the locus coeruleus. BMC Neurosci. 6(31) (2005)
22. Duzel, E., Bunzeck, N., Guitart-Masip, M., Duzel, S.: Novelty-related motivation of anticipation and exploration by dopamine (nomad): Implications for healthy aging. Neurosci. Biobehav. Rev. 34(5), 660-669 (2010)
23. Duzel, E., Guitart-Masip, M., Weiskopf, N., Kanowski, M.: The fMRI Book, chapter Midbrain fMRI: Applications, Limitations and Challenges (2012, in revision)
24. Fenker, D., Frey, J., Schuetze, H., Heipertz, D., Heinze, H., Duzel, E.: Novel scenes improve recollection and recall of words. J. Cogn. Neurosci. 20(7), 1250-1265 (2008)
25. Frey, U., Matthies, H., Reymann, K.: The effect of dopaminergic d1 receptor blockade during tetanization on the expression of long-term potentiation in the rat ca1 region in vitro. Neurosci. Lett. 129(1), 111-114 (1991)
26. Frey, U., Morris, R.: Synaptic tagging: Implications for late maintenance of hippocampal long-term potentiation. Trends Neurosci. 21(5), 181-188 (1998)
27. Frey, U., Schroeder, H., Matthies, H.: Dopaminergic antagonists prevent long-term maintenance of posttetanic ltp in the ca1 region of rat hippocampal slices. Brain Res. 522(1), 69-75 (1990)
28. Fuxe, K., Hokfelt, T., Johansson, O., Jonsson, G., Lidbrink, P., Ljungdahl, A.: The origin of the dopamine nerve terminals in limbic and frontal cortex. evidence for meso-cortico dopamine neurons. Brain Res. 82(2), 349-355 (1974)
29. Fyhn, M., Molden, S., Hollup, S., Moser, M., Moser, E.: Hippocampal neurons responding to firsttime dislocation of a target object. Neuron 35(3), 555-566 (2002)
30. Gasbarri, A., Campana, E., Pacitti, C., Hajdu, F., Tombol, T.: Organization of the projections from the ventral tegmental area of tsai to the hippocampal formation in the rat. J. Hirnforsch. 32(4), 429-437 (1991)
31. Gasbarri, A., M.G., P., Campana, E., Pacitti, C.: Anterograde and retrograde tracing of projections from the ventral tegmental area to the hippocampal formation in the rat. Brain Res. Bull. 33(4), 445-452 (1994a)
32. Gasbarri, A., Packard, M., Sulli, A., Pacitti, C., Innocenzi, R., Perciavalle, V.: The projections of the retrorubral field a8 to the hippocampal formation in the rat. Exp. Brain Res. 112(2), 244-252 (1996)
33. Gasbarri, A., Sulli, A., Packard, M.: The dopaminergic mesencephalic projections to the hippocam-pal formation in the rat. Prog. Neuropsychopharmacol. Biol. Psychiatry 21(1), 1-22 (1997)
34. Gasbarri, A., Verney, C., Innocenzi, R., Campana, E., Pacitti, C.: Mesolimbic dopaminergic neurons innervating the hippocampal formation in the rat: A combined retrograde tracing and immunohistochemical study. Brain Res. 668(1-2), 71-79 (1994b)
35. Geffard, M., Buijs, R., Seguela, P., Pool, C., Le Moal, M.: First demonstration of highly specific and sensitive antibodies against dopamine. Brain Res. 294(1), 161-165 (1984a)
36. Geffard, M., Kah, O., Onteniente, B., Seguela, P., Le Moal, M., Delaage, M.: Antibodies to dopamine: Radioimmunological study of specificity in relation to immunocytochemistry. J. Neurochem. 42(6), 1593-1599 (1984b)
37. Geffard, M., Seguela, P., Heinrich-Rock, A.: Antisera against catecholamines: Specificity studies and physicochemical data for anti-dopamine and anti-p-tyramine antibodies. Mol. Immunol. 21(6), 515-522 (1984c)
38. Giese, K., Fedorov, N., Filipkowski, R., Silva, A.: Autophosphorylation at thr286 of the alpha calcium-calmodulin kinase ii in ltp and learning. Science 279 (5352), 870-873 (1998)
39. Goldman-Rakic, P., Leranth, C., Williams, S., Mons, N., Geffard, M.: Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex. Proc. Natl. Acad. Sci. U. S. A. 86(22), 9015-9019 (1989)
40. Guitart-Masip, M., Bunzeck, N., Stephan, K., Dolan, R., Duzel, E.: Contextual novelty changes reward representations in the striatum. J. Neurosci. 30(5), 1721-1726 (2010)
41. Guitart-Masip, M., Fuentemilla, L., Bach, D., Huys, Q., Dayan, P., Dolan, R., Duzel, E.: Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J. Neurosci. 31(21), 7867-7875 (2011)
42. Hokfelt, T., Fuxe, K., Johansson, O., Ljungdahl, A.: Pharmaco-histochemical evidence of the existence of dopamine nerve terminals in the limbic cortex. Eur. J. Pharmacol. 25(1), 108-112 (1974a)
43. Hokfelt, T., Ljungdahl, A., Fuxe, K., Johansson, O.: Dopamine nerve terminals in the rat limbic cortex: Aspects of the dopamine hypothesis of schizophrenia. Science 184(133), 177-179 (1974b)
44. Huang, Y., Kandel, E.: D1/d5 receptor agonists induce a protein synthesis-dependent late potentiation in the ca1 region of the hippocampus. Proc. Natl. Acad. Sci. U. S. A. 92(7), 2446-2450 (1995)
45. Ishikawa, K., Ott, T., McGaugh, J.: Evidence for dopamine as a transmitter in dorsal hippocampus. Brain Res. 232(1), 222-226 (1982)
46. Kakade, S., Dayan, P.: Dopamine: Generalization and bonuses. Neural Netw. 15(4-6), 549-559 (2002)
47. Karlsson, M., Frank, L.: Network dynamics underlying the formation of sparse, informative representations in the hippocampus. J. Neurosci. 28(52), 14271-14281 (2008)
48. Krebs, R., Heipertz, D., Schuetze, H., Duzel, E.: Novelty increases the mesolimbic functional connectivity of the substantia nigra/ventral tegmental area (sn/vta) during reward anticipation: Evidence from high-resolution fmri. NeuroImage 58(2), 647-655 (2011)
49. Krebs, R., Schott, B., Duzel, E.: Personality traits are differentially associated with patterns of reward and novelty processing in the human substantia nigra/ventral tegmental area. Biol. Psychiatry 65(2), 103-110 (2009)
50. Kwon, O., Paredes, D., Gonzalez, C., Neddens, J., Hernandez, L., Vullhorst, D., Buonanno, A.: Neuregulin-1 regulates ltp at ca1 hippocampal synapses through activation of dopamine d4 receptors. Proc. Natl. Acad. Sci. U. S. A. 105(40), 15587-1592 (2008)
51. Legault, M., Wise, R.: Novelty-evoked elevations of nucleus accumbens dopamine: Dependence on impulse flow from the ventral subiculum and glutamatergic neurotransmission in the ventral tegmental area. Eur. J. Neurosci. 13(4), 819-828 (2001)
52. Lemon, N., Manahan-Vaughan, D.: Dopamine d1/d5 receptors gate the acquisition of novel information through hippocampal long-term potentiation and long-term depression. J. Neurosci. 26(29), 7723-7729 (2006)
53. Li, S., Cullen, W., Anwyl, R., Rowan, M.: Dopamine-dependent facilitation of ltp induction in hippocampal ca1 by exposure to spatial novelty. Nat. Neurosci. 6(5), 526-531 (2003)
54. Lisman, J., Grace, A.: The hippocampal-vta loop: Controlling the entry of information into longterm memory. Neuron 46(5), 703-713 (2005)
55. Lisman, J., Grace, A., Duzel, E.: A neohebbian framework for episodic memory; role of dopamine-dependent late ltp. Trends Neurosci. 34(10), 536-547 (2011)
56. Ljungberg, T., Apicella, P., Schultz, W.: Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67(1), 145-163 (1992)
57. Luo, A., Tahsili-Fahadan, P., Wise, R., Lupica, C., Aston-Jones, G.: Linking context with reward: A functional circuit from hippocampal ca3 to ventral tegmental area. Science 333 (6040), 353-357 (2011)
58. Martig, A., Mizumori, S.: Ventral tegmental area and substantia nigra neural correlates of spatial learning. Learn. Mem. 18(4), 260-271 (2011)
59. Mirolli, M., Baldassarre, G.: Functions and mechanisms of intrinsic motivations: The knowledge versus competence distinction. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 49-72. Springer, Berlin (2012)
60. Navakkode, S., Sajikumar, S., Frey, J.: Synergistic requirements for the induction of dopaminergic d1/d5-receptor-mediated ltp in hippocampal slices of rat ca1 in vitro. Neuropharmacology 52(7), 1547-1554 (2007)
61. Nehmzow, U., Gatsoulis, Y., Kerr, E., Condell, J., Siddique, N.H., McGinnity, M.T.: Novelty detection as an intrinsic motivation for cumulative learning robots. In: Baldassarre, G., Mirolli, M. (eds.) Intrinsically Motivated Learning in Natural and Artificial Systems, pp. 185-207. Springer, Berlin (2012)
62. Neugebauer, F., Korz, V., Frey, J.: Modulation of extracellular monoamine transmitter concentrations in the hippocampus after weak and strong tetanization of the perforant path in freely moving rats. Brain Res. 1273, 29-38 (2009)
63. Nitz, D., McNaughton, B.: Differential modulation of ca1 and dentate gyrus interneurons during exploration of novel environments. J. Neurophysiol. 91(2), 863-872 (2004)
64. Oades, R., Halliday, G.: Ventral tegmental (a10) system: Neurobiology. 1. anatomy and connectivity. Brain Res. 434(2), 117-165 (1987)
65. Onteniente, B., Geffard, M., Calas, A.: Ultrastructural immunocytochemical study of the dopamin-ergic innervation of the rat lateral septum with anti-dopamine antibodies. Neuroscience 13(2), 385-393 (1984)
66. Otmakhova, N., Lisman, J.: D1/d5 dopamine receptor activation increases the magnitude of early long-term potentiation at ca1 hippocampal synapses. J. Neurosci. 16(23), 7478-7486 (1996)
67. Pacholczyk, T., Blakely, R., Amara, S.: Expression cloning of a cocaine- and antidepressant-sensitive human noradrenaline transporter. Nature 350 (6316), 350-354 (1991)
68. Quintin, L., Hilaire, G., Pujol, J.: Variations in 3,4-dihydroxyphenylacetic acid concentration are correlated to single cell firing changes in the rat locus coeruleus. Neuroscience 18(4), 889-899 (1986)
69. Redgrave, P., Gurney, K.: The short-latency dopamine signal: A role in discovering novel actions? Nat. Rev. Neurosci. 7(12), 967-975 (2006)
70. Redondo, R., Morris, R.: Making memories last: The synaptic tagging and capture hypothesis. Nat. Rev. Neurosci. 12(1), 17-30 (2011)
71. Rossato, J., Bevilaqua, L., Izquierdo, I., Medina, J., Cammarota, M.: Dopamine controls persistence of long-term memory storage. Science 325 (5943), 1017-1020 (2009)
72. Sawaguchi, T., Goldman-Rakic, P.: The role of d1-dopamine receptor in working memory: Local injections of dopamine antagonists into the prefrontal cortex of rhesus monkeys performing an oculomotor delayed-response task. J. Neurophysiol. 71(2), 515-528 (1994)
73. Scatton, B., Dennis, T., Curet, O.: Increase in dopamine and dopac levels in noradrenergic terminals after electrical stimulation of the ascending noradrenergic pathways. Brain Res. 298(1), 193-196 (1984)
74. Scatton, B., Javoy-Agid, F., Rouquier, L., Dubois, B., Agid, Y.: Reduction of cortical dopamine, noradrenaline, serotonin and their metabolites in parkinson's disease. Brain Res. 275(2), 321-328 (1983)
75. Scatton, B., Rouquier, L., Javoy-Agid, F., Agid, Y.: Dopamine deficiency in the cerebral cortex in parkinson disease. Neurology 32(9), 1039-1040 (1982)
76. Scatton, B., Simon, H., Le Moal, M., Bischoff, S.: Origin of dopaminergic innervation of the rat hippocampal formation. Neurosci. Lett. 18(2), 125-131 (1980)
77. Schott, B., Seidenbecher, C., Fenker, D., Lauer, C., Bunzeck, N., Bernstein, H., Tischmeyer, W., Gundelfinger, E., Heinze, H., Duzel, E.: The dopaminergic midbrain participates in human episodic memory formation: Evidence from genetic imaging. J. Neurosci. 26(5), 1407-1417 (2006)
78. Schroeter, S., Apparsundaram, S., Wiley, R., Miner, L., Sesack, S., Blakely, R.: Immunolocalization of the cocaine- and antidepressant-sensitive l-norepinephrine transporter. J. Comp. Neurol. 420(2), 211-232 (2000)
79. Schultz, W.: Behavioral dopamine signals. Trends Neurosci. 30(5), 203-210 (2007a)
80. Schultz, W.: Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 30, 259-288 (2007b)
81. Schultz, W., Apicella, P., Ljungberg, T.: Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13(3), 900-913 (1993)
82. Schwab, M., Javoy-Agid, F., Agid, Y.: Labeled wheat germ agglutinin (wga) as a new, highly sensitive retrograde tracer in the rat brain hippocampal system. Brain Res. 152(1), 145-150 (1978)
83. Shohamy, D., Wagner, A.: Integrating memories in the human brain: Hippocampal-midbrain encoding of overlapping events. Neuron 60(2), 378-389 (2008)
84. Simon, H., Le Moal, M., Calas, A.: Efferents and afferents of the ventral tegmental-a10 region studied after local injection of [3h]leucine and horseradish peroxidase. Brain Res. 178(1), 17-40 (1979)
85. Smith, C.C., Greene, R.W.: CNS dopamine transmission mediated by noradrenergic innervation. J. Neurosci. 32(18), 6072-6080 (2012)
86. Sountsov, P., Santucci, D.M., Lisman, J.: A biologically plausible transform for visual recognition that is invariant to translation, scale, and rotation. Front. Comput. Neurosci. 5(53) (2011)
87. Swanson, L.: The projections of the ventral tegmental area and adjacent regions: A combined fluorescent retrograde tracer and immunofluorescence study in the rat. Brain Res. Bull. 9(1-6), 321-53 (1982)
88. Swanson, L., Hartman, B.: The central adrenergic system. an immunofluorescence study of the location of cell bodies and their efferent connections in the rat utilizing dopamine-beta-hydroxylase as a marker. J. Comp. Neurol. 163(4), 467-505 (1975)
89. Swanson, L., Kohler, C., Bjorklund, A.: The limbic region. I: The septohippocampal system. In: Bjorklund A, Hokfelt T.S.L. (eds.) Integrated Systems of the CNS, pp. 125-269. Elsevier, Amsterdam (1987)
90. Swant, J., Wagner, J.: Dopamine transporter blockade increases ltp in the ca1 region of the rat hippocampus via activation of the d3 dopamine receptor. Learn. Mem. 13(2), 161-167 (2006)
91. Van Eden, C., Hoorneman, E., Buijs, R., Matthijssen, M., Geffard, M., Uylings, H.: Immunocytochemical localization of dopamine in the prefrontal cortex of the rat at the light and electron microscopical level. Neuroscience 22(3), 849-862 (1987)
92. Van Elzakker, M., Fevurly, R., Breindel, T., Spencer, R.: Environmental novelty is associated with a selective increase in fos expression in the output elements of the hippocampal formation and the perirhinal cortex. Learn. Mem. 15(12), 899-908 (2008)
93. Verney, C., Baulac, M., Berger, B., Alvarez, C., Vigny, A., Helle, K.: Morphological evidence for a dopaminergic terminal field in the hippocampal formation of young and adult rat. Neuroscience 14(4), 1039-1052 (1985)
94. Wang, S., Redondo, R., Morris, R.: Relevance of synaptic tagging and capture to the persistence of long-term potentiation and everyday spatial memory. Proc. Natl. Acad. Sci. U. S. A. 107, 19537-19542 (2010)
95. Whitlock, J., Heynen, A., Shuler, M., Bear, M.: Learning induces long-term potentiation in the hippocampus. Science 313 (5790), 1093-1097 (2006)
96. Wittmann, B., Bunzeck, N., Dolan, R., Duzel, E.: Anticipation of novelty recruits reward system and hippocampus while promoting recollection. Neuroimage 38(1), 194-202 (2007)
97. Wittmann, B., Schott, B., Guderian, S., Frey, J., Heinze, H., Duzel, E.: Reward-related fmri activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent longterm memory formation. Neuron 45(3), 459-467 (2005)
98. Wolosin, S., Liang, J., Zeithamova, D., Schmandt, N., Preston, A.: Reward modulation of hippocampal subregions during motivated associative encoding. Soc. Neurosci. (2009)
99. Wyss, J., Swanson, L., Cowan, W.: A study of subcortical afferents to the hippocampal formation in the rat. Neuroscience 4(4), 463-476 (1979)
100. Zoli, M., Torri, C., Ferrari, R., Jansson, A., Zini, I., Fuxe, K., Agnati, L.: The emergence of the volume transmission concept. Brain Res. Brain Res. Rev. 26(2-3), 136-47 (1998)