Context, Learning, and Extinction

Psychological Review

Volumn 117, Issue 1, 2010, Pages 197-209

  Gershman, Samuel J ^a   Blei, David M ^b   Niv, Yael ^a  

^a PRINCETON UNIVERSITY   (United States)

^b Princeton University   (United States)

Author keywords

Bayesian; classical conditioning; hippocampus; latent inhibition; renewal

Indexed keywords

ARTICLE; BAYES THEOREM; CONDITIONED REFLEX; HIPPOCAMPUS; HUMAN; LEARNING; PHYSIOLOGY; PSYCHOLOGICAL MODEL; REINFORCEMENT; SEMANTICS;

BAYES THEOREM; CONDITIONING, CLASSICAL; EXTINCTION, PSYCHOLOGICAL; HIPPOCAMPUS; HUMANS; LEARNING; MODELS, PSYCHOLOGICAL; SEMANTICS;

EID: 74049117596     PISSN: 0033295X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0017808     Document Type: Article

References (73)

1. Aldous D. Exchangeability and related topics. École d'Été de probabilités de Saint-Flour xiii (1985), Springer, Berlin, Germany 1-198
2. Anderson J. The adaptive nature of human categorization. Psychological Review 98 (1991) 409-429
3. Blaisdell A.P., Sawa K., Leising K.J., and Waldmann M.R. Causal reasoning in rats. Science 311 (2006, February 17) 1020-1022
4. Bouton M.E. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin 114 (1993) 80-99
5. Bouton M.E. Context and behavioral processes in extinction. Learning and Memory 11 (2004) 485-494
6. Bouton M.E., and Bolles R.C. Contextual control of the extinction of conditioned fear. Learning and Motivation 10 (1979) 445-466
7. Bouton M.E., and King D.A. Contextual control of the extinction of conditioned fear: Tests for the associative value of the context. Journal of Experimental Psychology: Animal Behavioral Processes 9 (1983) 248-265
8. Brandon S.E., Vogel E.H., and Wagner A.R. A componential view of configural cues in generalization and discrimination in Pavlovian conditioning. Behavioral Brain Research 110 (2000) 67-72
9. Brogden W. Sensory preconditioning. Journal of Experimental Psychology 25 (1939) 323-332
10. Brown S., and Steyvers M. Detecting and predicting changes. Cognitive Psychology 58 (2009) 49-67
11. Capaldi E. The effect of different amounts of alternating partial reinforcement on resistance to extinction. American Journal of Psychology 70 (1957) 451-452
12. Capaldi E. The effect of different amounts of training on the resistance to extinction of different patterns of partially reinforced responses. Journal of Comparative and Physiological Psychology 51 (1958) 367-371
13. Corbit L.H., and Balleine B.W. The role of the hippocampus in instrumental conditioning. Journal of Neuroscience 20 (2000) 4233-4239
14. Courville, A. C. (2006). A latent cause theory of classical conditioning. Unpublished doctoral dissertation, Pittsburgh, PA.
15. Courville A.C., Daw N., Gordon G.J., and Touretzky D.S. Model uncertainty in classical conditioning. In: Thrun S., Saul L., and Schölkopf B. (Eds). Advances in neural information processing systems (2004), MIT Press, Cambridge, MA 977-984
16. Courville A.C., Daw N.D., and Touretzky D.S. Similarity and discrimination in classical conditioning: A latent variable account. In: Saul L.K., Weiss Y., and Bottou L. (Eds). Advances in neural information processing systems (2002), MIT Press, Cambridge, MA 313-320
17. Daw N., and Courville A. The rat as particle filter. In: Platt J., Koller D., Singer Y., and Roweis S. (Eds). Advances in neural information processing systems (2008), MIT Press, Cambridge, MA 369-376
18. Daw N., Courville A., and Touretzky D. Representation and timing in theories of the dopamine system. Neural Computation 18 (2006) 1637-1677
19. Dayan P., and Abbott L. Theoretical neuroscience: Computational and mathematical modeling of neural systems (2001), MIT Press, Cambridge, MA
20. Dayan P., Kakade S., and Montague P.R. Learning and selective attention. Nature Neuroscience 3 Suppl (2000) 1218-1223
21. Dayan P., and Long T. Statistical models of conditioning. In: Thrun S., Saul L., and Schölkopf B. (Eds). Advances in neural information processing systems (1998), MIT Press, Cambridge, MA 117-123
22. Dayan P., Niv Y., Seymour B., and Daw N. The misbehavior of value and the discipline of the will. Neural Networks 19 (2006) 1153-1160
23. Doboli S., Minai A.A., and Best P.J. Latent attractors: A model for context- dependent place representations in the hippocampus. Neural Computation 12 (2000) 1009-1043
24. Fearnhead P. Particle filters for mixture models with an unknown number of components. Journal of Statistics and Computing 14 (2004) 11-21
25. Foster D.J., Morris R.G., and Dayan P. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10 (2000) 1-16
26. Frohardt R.J., Guarraci F.A., and Bouton M.E. The effects of neurotoxic hippocampal lesions on two effects of context after fear extinction. Behavioral Neuroscience 114 (2000) 227-240
27. Fuhs M.C., and Touretzky D.S. Context learning in the rodent hippocampus. Neural Computation 19 (2007) 3173-3215
28. Gelman A., Carlin J., Stern H., and Rubin D. Bayesian data analysis (2003), Chapman and Hall/CRC, Boca Raton, FL
29. Gluck M., and Myers C. Hippocampal mediation of stimulus representation: A computational theory. Hippocampus 3 (1993) 491-516
30. Grossberg S. Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors. Biological Cybernetics 23 (1976) 121-134
31. Hall G., and Honey R.C. Contextual effects in conditioning, latent inhibition, and habituation: Associative and retrieval functions of contextual cues. Journal of Experimental Psychology: Animal Behavior Processes 15 (1989) 232-241
32. Hasselmo M.E., Bodelón C., and Wyble B. A proposed function for hippocampal theta rhythm: Separate phases of encoding and retrieval enhance reversal of prior learning. Neural Computation 14 (2002) 793-817
33. Hasselmo M.E., and Eichenbaum H. Hippocampal mechanisms for the context-dependent retrieval of episodes. Neural Networks 18 (2005) 1172-1190
34. Hertz J., Krogh A., and Palmer R. Introduction to the theory of neural computation (1991), Addison Wesley, Boston, MA
35. Hirsh R. The hippocampus and contextual retrieval of information from memory: A theory. Behavioral Biology 12 (1974) 421-444
36. Holland P.C. Excitation and inhibition in unblocking. Journal of Experimental Psychology: Animal Behavioral Processes 14 (1988) 261-279
37. Honey R.C., and Good M. Selective hippocampal lesions abolish the contextual specificity of latent inhibition and conditioning. Behavioral Neuroscience 107 (1993) 23-33
38. Howard M.W., Fotedar M.S., Datey A.V., and Hasselmo M.E. The temporal context model in spatial navigation and relational learning: Toward a common explanation of medial temporal lobe function across domains. Psychological Review 112 (2005) 75-116
39. Jarrard L. On the role of the hippocampus in learning and memory in the rat. Behavioral and Neural Biology 60 (1993) 9-26
40. Ji J., and Maren S. Electrolytic lesions of the dorsal hippocampus disrupt renewal of conditional fear after extinction. Learning and Memory 12 (2005) 270-276
41. Ji J., and Maren S. Hippocampal involvement in contextual modulation of fear extinction. Hippocampus 17 (2007) 749-758
42. Johnson A., van der Meer M.A., and Redish A.D. Integrating hippocampus and striatum in decision-making. Current Opinion in Neurobiology 17 (2007) 692-697
43. Kakade S., and Dayan P. Acquisition and extinction in autoshaping. Psychological Review 109 (2002) 2002
44. Kemp C., and Tenenbaum J.B. The discovery of structural form. Proceedings of the National Academy of Sciences of the United States of America 105 (2008) 10687-10692
45. Larrauri J.A., and Schmajuk N.A. Attentional, associative, and configural mechanisms in extinction. Psychological Review 115 (2008) 640-676
46. Lisman J.E., and Grace A.A. The hippocampal-VTA loop: Controlling the entry of information into long-term memory. Neuron 46 (2005) 703-713
47. Love B.C., and Gureckis T.M. Models in search of a brain. Cognitive, Affective, & Behavioral Neuroscience 7 (2007) 90-108
48. Marr D. Simple memory: A theory for archicortex. Philosophical Transactions of the Royal Society London B, Biological Sciences 262 (1971) 23-81
49. Marr D. Vision (1980), Freeman, San Francisco, CA
50. Martin P.D., and Berthoz A. Development of spatial firing in the hippocampus of young rats. Hippocampus 12 (2002) 465-480
51. McClelland J.L., McNaughton B.L., and O'Reilly R.C. Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review 102 (1995) 419-457
52. McNaughton B.L., and Morris R.G.M. Hippocampal synaptic enhancement and information storage within a distributed memory system. Trends in Neurosciences 10 (1987) 408-415
53. Nadel L. The role of the hippocampus in declarative memory: A comment on Zola-Morgan, Squire, and Ramus (1994). Hippocampus 5 (1995) 232-239
54. Nadel L., and MacDonald L. Hippocampus: Cognitive map or working memory?. Behavioral and Neural Biology 29 (1980) 405-409
55. O'Keefe J., and Nadel L. The hippocampus as a cognitive map (1978), Oxford University Press, New York, NY
56. O'Reilly R.C., and McClelland J.L. Hippocampal conjunctive encoding, storage, and recall: Avoiding a trade-off. Hippocampus 4 (1994) 661-682
57. Pearce J.M., and Bouton M.E. Theories of associative learning in animals. Annual Review of Psychology 52 (2001) 111-139
58. Pitman J. Combinatorial stochastic processes (Notes for Saint Flour Summer School; Technical Report no. 621) (2002) Department of Statistics, University of California, Berkeley
59. Quirk G.J., Garcia R., and Gonzàlez-Lima F. Prefrontal mechanisms in extinction of conditioned fear. Biological Psychiatry 60 (2006) 337-343
60. Redish A. Beyond the cognitive map: From place cells to episodic memory (1999), MIT Press, Cambridge, MA
61. Redish A., Jensen S., Johnson A., and Kurth-Nelson A. Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychological Review 114 (2007) 784-805
62. Ren L., Dunson D.B., and Carin L. The dynamic hierarchical dirichlet process. Proceedings of the 25th international conference on machine learning (2008), ACM, Helsinki, Finland 824-831
63. Rescorla R.A., and Wagner A.R. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black A., and Prokasy W. (Eds). Classical conditioning ii: Current research and theory (1972), Appleton-Century-Crofts, New York 64-99
64. Rhodes S.E., and Killcross A.S. Lesions of rat infra-limbic cortex enhance renewal of extinguished appetitive Pavlovian responding. European Journal of Neuroscience 25 (2007) 2498-2503
65. Rudy J.W. Contextual conditioning and auditory cue conditioning dissociate during development. Behavioral Neuroscience 107 (1993) 887-891
66. Rudy J.W., and O'Reilly R.C. Contextual fear conditioning, conjunctive representations, pattern completion, and the hippocampus. Behavioral Neuroscience 113 (1999) 867-880
67. Sanborn A.N., Griffiths T.L., and Navarro D.J. A more rational model of categorization. In: Sun R., and Miyake N. (Eds). Proceedings of the 28th Annual Conference of the Cognitive Science Society (2006), Erlbaum, Mahwah, NJ 726-731
68. Schmajuk N.A., Buhusi C., and Gray J.A. An attentional-configural model of classical conditioning. Journal of Mathematical Psychology 40 (1996) 358
69. Schultz W., Dayan P., and Montague P.R. A neural substrate of prediction and reward. Science 275 (1997, March 14) 1593-1599
70. Sutton R.S., and Barto A.G. Reinforcement learning: An introduction (1998), MIT Press, Cambridge, MA
71. Wilson A., Brooks D.C., and Bouton M.E. The role of the rat hippocampal system in several effects of context in extinction. Behavioral Neuroscience 109 (1995) 828-836
72. Yap C.S., and Richardson R. Latent inhibition in the developing rat: An examination of context-specific effects. Developmental Psychobiology 47 (2005) 55-65
73. Yap C.S., and Richardson R. Extinction in the developing rat: An examination of renewal effects. Developmental Psychobiology 49 (2007) 565-575