Vicarious trial and error

Nature Reviews Neuroscience

Volumn 17, Issue 3, 2016, Pages 147-159

  Redish, A David ^a  

^a UNIVERSITY OF MINNESOTA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATION; BEHAVIOR; DECISION MAKING; DORSOLATERAL STRIATUM; ENVIRONMENT; HIPPOCAMPUS; HIPPOCAMPUS THETA RHYTHM; HUMAN; INFORMATION; MAZE TEST; NEUROPHYSIOLOGY; NONHUMAN; ORBITAL CORTEX; PREDICTION; PRIORITY JOURNAL; RANDOMIZED CONTROLLED TRIAL (TOPIC); REVIEW; REWARD; SPACE; VENTRAL STRIATUM; VICARIOUS TRIAL AND ERROR BEHAVIOR; ANIMAL; BIOLOGICAL MODEL; BRAIN; CYTOLOGY; MENTAL FUNCTION; NERVE CELL; PHYSIOLOGY; RAT;

ANIMALS; BRAIN; DECISION MAKING; HUMANS; MENTAL PROCESSES; MODELS, BIOLOGICAL; NEURONS; RATS;

EID: 84959096800     PISSN: 1471003X     EISSN: 14710048     Source Type: Journal    
DOI: 10.1038/nrn.2015.30     Document Type: Review

References (172)

1. Muenzinger, K. F., & Gentry, E. Tone discrimination in white rats. J. Comp. Psychol. 12, 195-206 (1931)
2. Tolman, E. C. Prediction of vicarious trial and error by means of the schematic sowbug. Psychol. Rev. 46, 318-336 (1939)
3. Tolman, E. C. Cognitive maps in rats and men. Psychol. Rev. 55, 189-208 (1948)
4. Hull, C. L. Principles of Behavior (Appleton-Century-Crofts, 1943)
5. Turing, A. On computable numbers, with an application to the entscheidungs problem. Proc. Lond. Math. Soc. 42, 230-265 (1937)
6. Wiener, N. Cybernetics, or Control and Communications in the Animal and the Machine (Hermann, 1948)
7. Shannon, C. A mathematical theory of communication. Bell System Techn. J. 27, 379-423, 623-656 (1948)
8. Newell, A., Shaw, J. C., & Simon, H. A. in Proc. Int. Conf. Information Process. [online], http://bitsavers. informatik.uni-stuttgart.de/pdf/rand/ipl/P-1584- Reporton-A-General-Problem-Solving-Program- Feb59.pdf (UNESCO, 1959)
9. Simon, H. A behavioral model of rational choice. Q. J. Econ. 69, 99-118 (1955)
10. O'Keefe, J., & Nadel, L The Hippocampus as a Cognitive Map (Clarendon Press, 1978)
11. Wilson, M. A., & McNaughton, B. L. Dynamics of the hippocampal ensemble code for space. Science 261, 1055-1058 (1993)
12. Zhang, K. Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory. J. Neurosci. 16, 2112-2126 (1996)
13. Brown, E. N., Frank, L. M., Tang, D., Quirk, M. C., & Wilson, M. A. A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells. J. Neurosci. 18, 7411-7425 (1998)
14. Johnson, A., & Redish, A. D. Neural ensembles CA3 transiently encode paths forward animal decision point. J. Neurosci. 27, 12176-12189 (2007)
15. Foster, D. J., & Wilson, M. A. Hippocampal theta sequences. Hippocampus 17, 1093-1099 (2007)
16. Pfeiffer, B. E., & Foster, D. J. Hippocampal place-cell sequences depict future paths to remembered goals. Nature 497, 74-79 (2013)
17. O'Craven, K. M., & Kanwisher, N. Mental imagery of faces and places activates corresponding stimulus-specific brain regions. J. Cogn. Neurosci. 12, 1013-1023 (2000)
18. Pearson J., Naselaris, T., Holmes, E. A., & Kosslyn S. M. Mental imagery: functional mechanisms clinical application. Trends Cogn. Sci. 19, 590-602 (2015)
19. Johnson, A., Fenton, A. A., Kentros, C., & Redish, A. D. Looking for cognition in the structure within the noise. Trends Cogn. Sci. 13, 55-64 (2009)
20. Payne, J., Bettman, J., & Johnson, E. The Adaptive Decision Maker (Cambridge Univ. Press, 1993)
21. Gilbert, D. T., & Wilson, T. D. Prospection: experiencing the future. Science 317, 1351-1354 (2007)
22. Rangel, A., Camerer, C., & Montague, P. R. A framework for studying the neurobiology of value based decision making. Nat. Rev. Neurosci. 9, 545-556 (2008)
23. Kurth-Nelson, Z., Bickel, W. K., & Redish, A. D. A theoretical account of cognitive effects in delay discounting. Eur. J. Neurosci. 35, 1052-1064 (2012)
24. Redish, A. D. The Mind within the Brain: How we Make Decisions and How Those Decisions Go Wrong (Oxford Univ. Press, 2013)
25. Buckner, R. L., & Carroll, D. C. Self-projection and the brain. Trends Cognitive Sci. 11, 49-57 (2007)
26. Schacter, D. L., & Addis, D. R. in Predictions in the Brain: Using Our Past to Generate a Future (ed. Bar, M.) 58-69 (Oxford Univ. Press, 2011)
27. Gilbert, D. T., & Wilson, T. D. Why the brain talks to itself: sources of error in emotional prediction. Phil. Trans. R Soc. B 364, 1335-1341 (2009)
28. Phelps, E., Lempert, K. M., & Sokol-Hessner, P. Emotion and decision making: multiple modulatory circuits. Annu. Rev. Neurosci. 37, 263-287 (2014)
29. Redish, A. D., Schultheiss, N. W., & Carter, E. C. The computational complexity of valuation and motivational forces in decision-making processes. Curr. Top. Behav. Neurosci. http://dx.doi.org/10.1007/7854-2015-375 (2015)
30. Redish, A. D., Jensen, S., Johnson, A., & Kurth Nelson, Z. Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychol. Rev. 114, 784-805 (2007)
31. Gershman, S. J., Blei, D., & Niv, Y. Context, learning and extinction. Psychol. Rev. 117, 197-209 (2010)
32. Niv, Y., et al. Reinforcement learning in multidimensional environments relies on attention mechanisms. J. Neurosci. 35, 8145-8157 (2015)
33. Johnson, A., & Crowe, D. A. Revisiting Tolman, his theories and cognitive maps. Cogn. Critique 1, 43-72 (2009)
34. Redish, A D. Beyond the Cognitive Map: From Place Cells to Episodic Memory (MIT Press, 1999)
35. Daw, N. D., Niv, Y., & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704-1711 (2005)
36. Winstanley, C A., et al. in Cognitive Search: Evolution, Algorithms, and the Brain (eds Hills, T., et al.) 125-156 (MIT Press, 2012)
37. McKenzie, S., et al. Hippocampal representation of related and opposing memories develop within distinct, hierarchically organized neural schemas. Neuron 83, 202-215 (2014)
38. Lichtenstein, S., & Slovic, P. (eds) The Construction of Preference (Cambridge Univ. Press, 2006)
39. Kahneman, D. Thinking, Fast, and Slow (Farrar, 2011)
40. Benoit, R. G., Gilbert, S. J., & Burgess, P. W. A neural mechanism mediating the impact of episodic prospection on farsighted decisions. J. Neurosci. 31, 6771-6779 (2011
41. Peters, J., & Büchel, C. Episodic future thinking reduces reward delay discounting through an enhancement of prefrontal-mediotemporal interactions. Neuron 66, 138-148 (2010)
42. Kwan, D., et al. Future decision-making without episodic mental time travel. Hippocampus 22, 1215-1219 (2012)
43. Hassabis, D., Kumaran, D., Vann, S. D., & Maguire, E. A. Patients with hippocampal amnesia cannot imagine experiences. Proc. Natl Acad. Sci. USA 104, 1726-1731 (2007)
44. Lebreton, M., et al. A critical role for the hippocampus in the valuation of imagined outcomes. PLoS Biol. 11, e1001684 (2013)
45. Wang, J. X., Cohen, N. J., & Voss, J. L. Covert rapid action-memory simulation (crams): a hypothesis of hippocampal-prefrontal interactions for adaptive behavior. Neurobiol. Learn. Mem. 117, 22-33 (2015)
46. Spiers, H. J., & Gilbert, S. J. Solving the detour problem in navigation: a model of prefrontal and hippocampal interactions. Front. Hum. Neurosci. 9, 125 (2015)
47. Hikosaka, O., et al. Parallel neural networks for learning sequential procedures. Trends Neurosci. 22, 464-471 (1999)
48. Lee, S. W., Shimoko, S., & O'Doherty, J. P. Neural computations underlying arbitration between model-based and model-free learning. Neuron 81, 687-699 (2014)
49. Dezfouli, A., & Balleine, B. Habits, action sequences and reinforcement learning. Eur. J. Neurosci. 35, 1036-1051 (2012)
50. Johnson, A., van der Meer, M. A., & Redish, A. D. Integrating hippocampus and striatum in decision-making. Curr. Opin. Neurobiol. 17, 692-697 (2007)
51. Gardner, R. S., et al. A secondary working memory challenge preserves primary place strategies despite overtraining. Learn. Mem. 20, 648-656 (2013)
52. Schmidt, B. J., Papale, A. E., Redish, A. D., & Markus, E. J. Conflict between place and response navigation strategies: effects on vicarious trial and error (VTE) behaviors. Learn. Mem. 20, 130-138 (2013)
53. Smith, K. S., & Graybiel, A. M. A dual operator view of habitual behavior reflecting cortical striatal dynamics. Neuron 79, 361-374 (2013)
54. Meer, M. A. A., Johnson, A., Schmitzer-Torbert, N. C., & Redish, A. D. Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 67, 25-32 (2010)
55. Thorn, C. A., Atallah, H., Howe, M., & Graybiel, A. M. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66, 781-795 (2010)
56. Regier, P. S., Amemiya, S., & Redish, A. D. Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task. J. Neurophysiol. 114, 1399-1416 (2015)
57. Niv, Y., Joel, D., & Dayan, P. A normative perspective on motivation. Trends Cogn. Sci. 10, 375-381 (2006)
58. Johnson, A., Varberg, Z., Benhardus, J., Maahs, A., & Schrater, P. The hippocampus and exploration: dynamically evolving behavior and neural representations. Front. Hum. Neurosci. 6, 216 (2012)
59. Balleine, B. W., & Dickinson, A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37, 407-419 (1998)
60. Liljeholm, M., Tricomi, E., O'Doherty, J. P., & Balleine, B. W. Neural correlates of instrumental contingency learning: differential effects of actionreward conjunction and disjunction. J. Neurosci. 31, 2474-2480 (2011)
61. Krajbich, I., & Rangel, A. Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proc. Natl Acad. Sci. USA 108, 13852-13857 (2011)
62. Packard, M. G., & McGaugh, J. L. Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol. Learn. Mem. 65, 65-72 (1996)
63. Yin, H. H., Knowlton, B., & Balleine, B. W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181-189 (2004)
64. Gupta, A. S., van der Meer, M. A. A., Touretzky, D. S., & Redish, A. D. Hippocampal replay is not a simple function of experience. Neuron 65, 695-705 (2010)
65. Blumenthal, A., Steiner, A., Seeland, K. D., & Redish, A. D. Effects of pharmacological manipulations of NMDA-receptors on deliberation in the Multiple T task. Neurobiol. Learn. Mem. 95, 376-384 (2011)
66. Steiner, A., & Redish, A. D. The road not taken: neural correlates of decision making in orbitofrontal cortex. Front. Decision Neurosci. 6, 131 (2012)
67. Steiner, A., & Redish, A. D. Behavioral and neurophysiological correlates of regret in rat decision-making on a neuroeconomic task. Nat. Neurosci. 17, 995-1002 (2014)
68. Papale, A., Stott, J. J., Powell, N. J., Regier, P. S., & Redish, A. D. Interactions between deliberation and delay-discounting in rats. Cogn. Affect. Behav. Neurosci. 12, 513-526 (2012)
69. Atance, C. M., & O'Neill, D. K. Episodic future thinking. Trends Cogn. Sci. 5, 533-539 (2001)
70. Seidenbecher, T., Laxmi, T. R., Stork, O., & Pape, H. C. Amygdalar and hippocampal theta rhythm synchronization during fear memory retrieval. Science 301, 846-850 (2003)
71. Schmidt, R., et al. Single-Trial phase precession in the hippocampus. J. Neurosci. 29, 13232-13241 (2009)
72. Gupta, A. S., van der Meer, M. A. A., Touretzky, D. S., & Redish, A. D. Segmentation of spatial experience by hippocampal ? sequences. Nat. Neurosci. 15, 1032-1039 (2012)
73. Hassabis, D., & Maguire, E. A. in Predictions in the Brain: Using our Past to Generate a Future (ed. Bar, M.) 70-82 (Oxford Univ. Press, 2011)
74. Howard, L. R., et al. The hippocampus and entorhinal cortex encode the path and Euclidean distances to goals during navigation. Curr. Biol. 24, 1331-1340 (2014)
75. Wikenheiser, A. M., & Redish, A. D. Hippocampal theta sequences reflect current goals. Nat. Neurosci. 18, 289-294 (2015)
76. Royer, S., Sirota, A., Patel, J., & Buzsaki, G. Distinct representations and theta dynamics in dorsal and ventral hippocampus. J. Neurosci. 30, 1777-1787 (2010)
77. Stensola, H., et al. The entorhinal grid map is discretized. Nature 492, 72-78 (2012)
78. Dragoi, G., & Buzsaki, G. Temporal encoding of place sequences by hippocampal cell assemblies. Neuron 50, 145-157 (2006)
79. Lisman, J., & Redish, A. D. Prediction, sequences and the hippocampus. Phil. Trans. R. Soc. B 364, 1193-1201 (2009)
80. Buzsaki, G. Hippocampal sharp wave-ripple: a cognitive biomarker for episodic memory and planning. Hippocampus 25, 1073-1188 (2015)
81. Diba, K., & Buzsaki, G. Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 10, 1241-1242 (2007)
82. Yamamoto, J., Suh, J., Takeuchi, D., & Tonegawa, S. Successful execution of working memory linked to synchronized high-frequency gamma oscillations. Cell 157, 845-857 (2014)
83. Schmitzer-Torbert, N. C., & Redish, A. D. Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task. J. Neurophysiol. 91, 2259-2272 (2004)
84. Tse, D., et al. Schemas and memory consolidation. Science 316, 76-82 (2007)
85. Hu, D., & Amsel, A. A simple test of the vicarious trial and-error hypothesis of hippocampal function. Proc. Natl Acad. Sci. USA 92, 5506-5509 (1995)
86. Bett, D., et al. The neural substrates of deliberative decision making: contrasting effects of hippocampus lesions on performance and vicarious trial-And-error behavior in a spatial memory task and a visual discrimination task. Front. Behav. Neurosci. 6, 70 (2012)
87. Bett, D., Murdoch, L. H., Wood, E. R., & Dudchenko, P. A. Hippocampus, delay discounting, and vicarious trial-And-error. Hippocampus 25, 643-654 (2015)
88. Barnes, C. A., Nadel, L., & Honig, W. K. Spatial memory deficit in senescent rats. Can. J. Psychol. 34, 29-39 (1980)
89. Breton, Y. A., Seeland K. D., & Redish, A. D. Aging impairs deliberation and behavioral flexibility in inter-Temporal choice. Front. Aging Neurosci. 7, 41 (2015)
90. Robbe, D., et al. Cannabinoids reveal importance of spike timing coordination in hippocampal function. Nat. Neurosci. 9, 1526-1533 (2006)
91. Robbe, D., & Buzsaki, G. Alteration of theta timescale dynamics of hippocampal place cells by a cannabinoid is associated with memory impairment. J. Neurosci. 29, 12597-12605 (2009)
92. Papale, A. E. Hippocampal Representations on the Spatial Delay Discounting Task. Thesis, Univ. Minnesota (2015)
93. Amemiya, S., Noji, T., Kubota, N., Nishijima, T., & Kita, I. Noradrenergic modulation of vicarious trial-And-error behavior during a spatial decision-making task in rats. Neuroscience 265, 291-301 (2014)
94. Amemiya, S., & Redish, A. D. Manipulating decisiveness in decision making: effects of clonidine on hippocampal search strategies. J. Neurosci. 36, 814-827 (2016)
95. Seifert, W Neurobiology of the Hippocampus (Academic Press, 1983)
96. Teyler, T. J. and DiScenna, P. The hippocampal memory indexing theory. Behav. Neurosci. 100, 147-154 (1986)
97. Nadel, L., Samsonovich, A., Ryan, L., & Moscovitch, M. Multiple trace theory of human memory: computational, neuroimaging, and neuropsychological results. Hippocampus 10, 352-368 (2000)
98. Doll, B. B., Duncan, K. D., Simon, D. A., Shohamy, D., & Daw, N. D. Model-based choices involve prospective neural activity. Nat. Neurosci. 18, 767-772 (2015)
99. Killcross, S., & Coutureau, E. Coordination of actions and habits in the medial prefrontral cortex of rats. Cereb. Cortex 13, 400-408 (2003)
100. Sharpe, M. J., & Killcross, S. The prelimbic cortex directs attention toward predictive cues during fear learning. Learn. Mem. 22, 289-293 (2015)
101. Spellman, T., et al. Hippocampal-prefrontal input supports spatial encoding in working memory. Nature 522, 309-314 (2015)
102. Ito, H. T., Zhang, S. J., Witter, M. P., Moser, E. I., & Moser, M. B. A prefrontal-Thalamo-hippocampal circuit for goal-directed spatial navigation. Nature 522, 50-55 (2015)
103. Kolb, B. in The Cerebral Cortex of the Rat (eds Kolb & Tees R. C.) 437-458 (MIT Press, 1990)
104. Dalley, J. W., Cardinal, R. N., & Robbins, T. W. Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates. Neurosci. Biobehav. Rev. 28, 771-784 (2004)
105. Wallis, J. D. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat. Neurosci. 15, 13-19 (2012)
106. Jung, M. W., Qin, Y., McNaughton, B. L., & Barnes, C. A. Firing characteristics of deep layer neurons in prefrontal cortex in rats performing spatial working memory tasks. Cereb. Cortex 8, 437-450 (1998)
107. Powell, N. J., & Redish, A. D. Complex neural codes in rat prelimbic cortex are stable across days on a spatial decision task. Front. Behav. Neurosci. 8, (2014)
108. Jones, M. W., & Wilson, M. A. Theta rhythms coordinate hippocampal-prefrontal interactions in a spatial memory task. PLoS Biol. 3, e402 (2005)
109. Hyman, J. M., Zilli, E. A., Paley, A. M., & Hasselmo, M. E. Working memory performance correlates with prefrontal-hippocampal theta interactions but not with prefrontal neuron firing rates. Front. Integr.Neurosci. 4, 2 (2010
110. Ragozzino, M. E., Detrick, S., & Kesner, R. P. Involvement of the prelimbic-infralimbic areas of the rodent prefrontal cortex in behavioral flexibility for place and response learning. J. Neurosci. 19, 4585-4594 (1999)
111. Rich, E. L., & Shapiro, M. L. Prelimbic/infralimbic inactivation impairs memory for multiple task switches, but not flexible selection of familiar tasks. J. Neurosci. 27, 4747-4755 (2007)
112. Benchenane, K., et al. Coherent theta oscillations and reorganization of spike timing in the hippocampal-prefrontal network upon learning. Neuron 66, 921-936 (2010)
113. O'Neill, P. K., Gordon, J. A., & Sigurdsson, T. Theta oscillations in the medial prefrontal cortex are modulated by spatial working memory and synchronize with the hippocampus through its ventral subregion. J. Neurosci. 33, 14211-14224 (2013)
114. Engel, A. K., Fries, P., & Singer, W. Dynamic predictions: oscillations and synchrony in top-down processing. Nat. Rev. Neurosci. 2, 704-716 (2001)
115. Kucewicz, M. T., Tricklebank, M. D., Bogacz, R., & Jones, M. W. Dysfunctional prefrontal cortical network activity and interactions following cannabinoid receptor activation. J. Neurosci. 31, 15560-15568 (2011)
116. Mogenson, G. J., Jones, D. L., & Yim, C. Y. From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14, 69-97 (1980)
117. McDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y., & Schoenbaum, G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 31, 2700-2705 (2011)
118. Jones, J. L., et al. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338, 953-956 (2012)
119. Padoa-Schioppa, C., & Assad, J. A. Neurons in the orbitofrontal cortex encode economic value. Nature 441, 223-226 (2006)
120. Strait, C. E., Blanchard, T. C., & Hayden, B. Y. Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron 82, 1357-1366 (2014)
121. Strait, C. E., Sleezer, B. J., & Hayden, B. Y. Signatures of value comparison in ventral striatum neurons. PLoS Biol. 13, e1002173 (2015)
122. Bechara, A., Tranel, D., & Damasio, H. Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. Brain 123, 2189-2202 (2000)
123. Zald, D. H., & Rauch, S. L. (eds) The Orbitofrontal Cortex (Oxford Univ. Press, 2008)
124. Fellows, L. K. Deciding how to decide: ventromedial frontal lobe damage affects information acquisition in multi-Attribute decision making. Brain 129, 944-952 (2006)
125. Bray, S., Shimojo, S., & O'Doherty, J. P. Human medial orbitofrontal cortex is recruited during experience of imagined and real rewards. J. Neurophysiol. 103, 2506-2512 (2010)
126. Tremblay, L., & Schultz, W. Relative reward preference in primate orbitofrontal cortex. Nature 398, 704-708 (1999)
127. Nicola, S. M., Yun, I. A., Wakabayashi, K. T., & Fields, H. L. Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. J. Neurophysiol. 91, 1840-1865 (2004)
128. Roitman, M. F., Wheeler, R. A., & Carelli, R. M. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587-597 (2005)
129. Stott, J. J., & Redish, A. D. A functional difference in information processing between orbitofrontal cortex ventral striatum during decision-making behaviour. Phil. Trans. R. Soc. B 369, 20130472 (2014)
130. German, P. W., & Fields, H. L. Rat nucleus accumbens neurons persistently encode locations associated with morphine reward. J. Neurophysiol. 97, 2094-2106 (2007)
131. Moorman, D. E., & Aston-Jones, G. Orbitofrontal cortical neurons encode expectation-driving initiation of reward-seeking. J. Neurosci. 34, 10234-10246 (2014)
132. van der Meer, M. A. A., & Redish, A. D. Covert expectation-of reward in rat ventral striatum at decision points. Front. Integr. Neurosci. 3, 1-15 (2009)
133. Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. Orbitofrontal cortex as a cognitive map of task space. Neuron 81, 267-279 (2014)
134. Fenton, A. A., et al. Attention-like modulation of hippocampus place cell discharge. J. Neurosci. 30, 4613-4625 (2010)
135. Wikenheiser, A., & Redish, A. D. Changes in reward contingency modulate the trial to trial variability of hippocampal place cells. J. Neurophysiol. 106, 589-598 (2011)
136. Berke, J. D., & Eichenbaum, H. Striatal versus hippocampal representations during win-stay maze performance. J. Neurophysiol. 101, 1575-1587 (2009)
137. Schmitzer-Torbert, N. C., & Redish, A. D. Task-dependent encoding of space and events by striatal neurons is dependent on neural subtype. Neuroscience 153, 349-360 (2008)
138. Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V., & Graybiel, A. M. Building neural representations of habits. Science 286, 1746-1749 (1999)
139. Jin, X., & Costa, R. M. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 466, 457-462 (2010)
140. Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z., & Graybiel, A. M. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158-1161 (2005)
141. Barnes, T. D., et al. Advance cueing produces enhanced action-boundary patterns of spike activity in the sensorimotor striatum. J. Neurophysiol. 105, 1861-1878 (2011)
142. Kubota, Y., et al. Stable encoding of task structure coexists with flexible coding of task events in sensorimotor striatum. J. Neurophysiol. 102, 2142-2160 (2009)
143. Corbit, L. H., & Balleine, B. W. The role of the hippocampus in instrumental conditioning. J. Neurosci. 20, 4233-4239 (2000)
144. Ragozzino, M. E., Ragozzino, K. E., Mizumori, S. J. Y., & Kesner, R. P. The role of the dorsomedial striatum in behavioral flexibility for repsonse and visual cue discrimination learning. Behav. Neurosci. 116, 105-115 (2002)
145. Ostlund, S. B., Winterbauer, N. E., & Balleine, B. W. Evidence of action sequence chunking in goal-directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex. J. Neurosci. 29, 8280-8287 (2009)
146. Corbit, L. H., & Janak, P. H. Posterior dorsomedial striatum is critical for both selective instrumental and Pavlovian reward learning. Eur. J. Neurosci. 31, 1312-1321 (2010)
147. Bower, G. H. in Studies in Mathematical Learning Theory (eds Bush, R. R., & Estes, W. K.) 109-124 (Stanford Univ. Press, 1959)
148. Whishaw, I. Q., & Kolb, B. The Behavior of the Laboratory Rat: A Handbook with Tests (Oxford Univ. Press, 2004)
149. Monaco, J. D., Rao, G., Roth, E. D., & Knierim, J. J. Attentive scanning behavior drives one-Trial potentiation of hippocampal place fields. Nat. Neurosci. 17, 725-731 (2014)
150. Baddeley, A. Working Memory, Thought and Action (Oxford Univ. Press, 2007)
151. Fuster, J The Prefrontal Cortex (Academic Press, 2008)
152. Guthrie, E. R. The Psychology of Learning (Harpers, 1935)
153. Rodgers, R. J., Cao, B. J., Dalvi, A., & Holmes, A. Animals models of anxiety: an ethological perspective. Brazil. J. Med. Biol. Res. 30, 289-304 (1997)
154. Jackson, J. C., & Redish, A. D. Detecting dynamical changes within a simulated neural ensemble using a measure of representational quality. Network 14, 629-645 (2003)
155. Adhikari, A. Distributed circuits underlying anxiety. Front. Behav. Neurosci. 8, 112 (2014)
156. Allsop, S. A., Vander Weele, C. M., Wichmann, R., & Tye, K. M. Optogenetic insights on the relationship between anxiety-related behaviors and social deficits. Front. Behav. Neurosci. 8, 241 (2014)
157. Adhikari, A., Topiwala, M. A., & Gordon, J. A. Synchronized activity between the ventral hippocampus and the medial prefrontal cortex during anxiety. Neuron 28, 257-269 (2010)
158. Adhikari, A., Topwala, M., & Gordon, J. A. Single units in the medial prefrontal cortex with anxiety-related firing patterns are preferentially influenced by ventral hippocampal activity. Neuron 71, 898-910 (2011)
159. Grewal, S. S., Shepherd, J. K., Bill, D. J., Fletcher, A., & Dourish, C. T. Behavioural and pharmacological characterisation of the canopy stretched attend posture test as a model of anxiety in mice and rats. Psychopharmacology 133, 29-38 (1997)
160. Choi, J. S., & Kim, J. J. Amygdala regulates risk of predation in rats foraging in a dynamic fear environment. Proc. Natl Acad. Sci. USA 107, 21773-21777 (2010)
161. Amir, A., Lee, S. C., Headley, D. B., Herzallah, M. M., & Pare, D. Amygdala signaling during foraging in a hazardous environment. J. Neurosci. 35, 12994-13005 (2015)
162. Voss, J. L., et al. Spontaneous revisitation during visual exploration as a link among strategic behavior, learning, and the hippocampus. Proc. Natl Acad. Sci. USA 108, E402-E409 (2011)
163. Reutskaja, E., Nagel, R., Camerer, C. F., & Rangel, A. Search dynamics in consumer choice under time pressure: an eye-Tracking study. Am. Econom. Rev. 101, 900-926 (2011)
164. Treisman, A. M., & Gelade, G. A feature-integration theory of attention. Cogn. Psychol. 12, 97-136 (1980)
165. Kruse, J. M., Overmier, J. B., Konz, W. A., & Rokke, E. Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific. Learn. Motiv. 14, 165-181 (1983)
166. Balleine, B. W., & Ostlund, S. B. Still at the choice-point: action selection and initiation in instrumental conditioning. Ann. NY Acad. Sci. 1104, 147-171 (2007)
167. Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. Human Pavlovian-instrumental transfer. J. Neurosci. 28, 360-368 (2008)
168. Bray, S., Rangel, A., Shimojo, S., Balleine, B., & O'Doherty, J. P. The neural mechanisms underlying the influence of Pavlovian cues on human decision making. J. Neurosci. 28, 5861-5866 (2008)
169. Damasio, A. Looking for Spinoza: Joy, Sorrow, and the Feeling Brain (Mariner, 2003)
170. Balleine, B. W. in The Behavior of the Laboratory Rat: A Handbook with Tests (eds Whishaw, I. Q., & Kolb, B.) 436-446 (Oxford Univ. Press, 2004)
171. Gold, J. I., & Shadlen, M. N. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36, 299-308 (2002)
172. Hanks, T. D., et al. Distinct relationships of parietal and prefrontal cortices to evidence accumulation. Nature 520, 220-223 (2015