Learning latent structure: Carving nature at its joints

Current Opinion in Neurobiology

Volumn 20, Issue 2, 2010, Pages 251-256

  Gershman, Samuel J ^a   Niv, Yael ^a  

^a PRINCETON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION MAKING; FEEDBACK SYSTEM; HUMAN; LEARNING ALGORITHM; LEARNING THEORY; MENTAL FUNCTION; MENTAL PERFORMANCE; MOTIVATION; NONHUMAN; PERCEPTION; PRIORITY JOURNAL; REINFORCEMENT; REVIEW; REWARD; SENSORY STIMULATION; STATE DEPENDENT LEARNING; STIMULUS RESPONSE;

ALGORITHMS; ANIMALS; ARTIFICIAL INTELLIGENCE; BRAIN; HUMANS; LEARNING; MODELS, ANIMAL; MODELS, NEUROLOGICAL; NEUROPSYCHOLOGY; PERCEPTION; PSYCHOMOTOR PERFORMANCE; REINFORCEMENT (PSYCHOLOGY);

EID: 77952541839     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.conb.2010.02.008     Document Type: Review

References (52)

1. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press.
2. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275(5306):1593.
3. Houk J.C., Adams J.L., Barto A.G. A model of how the basal ganglia generate and use neural signals that predict reinforcement. Models of Information Processing in the Basal Ganglia 1995, 249-270.
4. Kemp C., Tenenbaum J.B. Structured statistical models of inductive reasoning. Psychol Rev 2009, 116(1):20-58.
5. Mach E. The analysis of sensations 1897.
6. Von Helmholtz H. Handbuch der Physiologischen Optik 1867, Voss.
7. Knill D.C., Richards W. Perception as Bayesian inference 1996, Cambridge University Press.
8. Doya K., Ishii S., Alexandre P., Rao R.P.N. Bayesian Brain 2007, MIT Press.
9. Chater N., Oaksford M. The Probabilistic Mind: Prospects for Bayesian Cognitive Science 2008, Oxford University Press.
10. Daw N.D., Niv Y., Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 2005, 8(12):1704-1711.
11. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. Classical Conditioning II: Current Research and Theory 1972, 64-99. Appleton-Century-Crofts, New York, NY. A.H. Black, W.F. Prokasy (Eds.).
12. Gershman S.J., Blei D.M., Niv Y. Context, learning, and extinction. Psychol Rev 2010, 117(1):197-209.
13. Courville Aaron C., Daw Nathaniel D., Touretzky David S. Similarity and discrimination in classical conditioning: a latent variable account. Advances in Neural Information Processing Systems, vol 17 2002, 313-320. MIT Press, Cambridge, MA. L.K. Saul, Y. Weiss, L. Bottou (Eds.).
14. Courville Aaron C., Daw Nathaniel D., Gordon Geoffrey J., Touretzky David S. Model uncertainty in classical conditioning. Advances in Neural Information Processing Systems, vol 16 2004, MIT Press, Cambridge, MA. S. Thrun, L.K. Saul, B. Schölkopf (Eds.).
15. Daw N.D., Courville A.C., Touretzky D.S. Representation and timing in theories of the dopamine system. Neural Comput 2006, 18(7):1637-1677.
16. Daw N., Courville A. The pigeon as particle filter. Adv Neural Inform Process Syst 2008, 20:369-376.
17. Yi M.S.K., Steyvers M., Lee M. Modeling human performance in restless bandits with particle filters. J Problem Solving 2009, 2(2):5.
18. Vul E., Frank M., Alvarez G., Tenenbaum J. Explaining human multiple object tracking as resource-constrained approximate inference in a dynamic probabilistic model. Advances in Neural Information Processing Systems, vol 22 2009, 1955-1963. Y. Bengio, D. Schuurmans, J. Lafferty, C.K.I. Williams, A. Culotta (Eds.).
19. Gershman S., Vul E., Tenenbaum J. Perceptual multistability as Markov Chain Monte Carlo inference. Advances in Neural Information Processing Systems vol 22 2009, 611-619. Y. Bengio, D. Schuurmans, J. Lafferty, C.K.I. Williams, A. Culotta (Eds.).
20. Williams R.J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 1992, 8(3):229-256.
21. Acuna D., Schrater P.R. Structure learning in human sequential decision-making. Advances in Neural Information Processing Systems, vol 21 2009, 1-8. D. Koller, D. Schuurmans, Y. Bengio, L. Bottou (Eds.).
22. Timothy E.J., Behrens W., Mark, Woolrich E., Mark, Walton F.S., Matthew, Rushworth Learning the value of information in an uncertain world. Nat Neurosci 2007, 10(9):1214-1221.
23. Owen A.M., Roberts A.C., Hodges J.R., Robbins T.W. Contrasting mechanisms of impaired attentional set-shifting in patients with frontal lobe damage or Parkinson's disease. Brain 1993, 116(5):1159.
24. Milner B. Effects of different brain lesions on card sorting: the role of the frontal lobes. Arch Neurol 1963, 9(1):90.
25. Barch D.M., Braver T.S., Nystrom L.E., Forman S.D., Noll D.C., Cohen J.D. Dissociating working memory from task difficulty in human prefrontal cortex. Neuropsychologia 1997, 35(10):1373-1380.
26. O'Reilly R.C., Frank M.J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput 2006, 18(2):283-328.
27. Bouton M.E., Bolles R.C. Contextual control of the extinction of conditioned fear. Learn Motiv 1979, 10(4):445-466.
28. Nakajima S., Tanaka S., Urushihara K., Imada H. Renewal of extinguished lever-press responses upon return to the training context. Learn Motiv 2000, 31(4):416-431.
29. Rescorla R.A., Heth C.D. Reinstatement of fear to an extinguished conditioned stimulus. J Exp Psychol 1975, 1:88-96.
30. Bouton M.E., Bolles R.C. Role of conditioned contextual stimuli in reinstatement of extinguished fear. J Exp Psychol 1979, 5:368-378.
31. Robbins S.J. Mechanisms underlying spontaneous recovery in autoshaping. J Exp Psychol: Anim Behav Process 1990, 16(3):235-249.
32. Rescorla R.A. Spontaneous recovery. Learn Mem 2004, 11(5):501.
33. Bouton M.E. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychol Bull 1993, 114:80-180.
34. Redish A.D., Jensen S., Johnson A., Kurth-Nelson Z. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 2007, 114(3):784-805.
35. Ji J., Maren S. Electrolytic lesions of the dorsal hippocampus disrupt renewal of conditional fear after extinction. Learn Mem 2005, 12(3):270.
36. Yap C.S.L., Richardson R. Extinction in the developing rat: an examination of renewal effects. Dev Psychobiol 2007, 49(6):565-575.
37. Zilli E.A., Hasselmo M.E. Modeling the role of working memory and episodic memory in behavioral tasks. Hippocampus 2008, 18(2):193.
38. Braver T.S., Barch D.M., Cohen J.D. Cognition, and control in schizophrenia: a computational model of dopamine and prefrontal function. Biol Psychiatry 1999, 46(3):312-328.
39. Braver T.S., Cohen J.D. On the control of control: the role of dopamine in regulating prefrontal function and working memory. Control of Cognitive Processes: Attention and Performance XVIII 2000, 713-737.
40. Rougier N.P., Noelle D.C., Braver T.S., Cohen J.D., O'Reilly R.C. Prefrontal cortex and flexible cognitive control: rules without symbols. Proc Natl Acad Sci U S A 2005, 102(20):7338.
41. Jazbec S., Pantelis C., Robbins T., Weickert T., Weinberger D.R., Goldberg T.E. Intra-dimensional/extra-dimensional set-shifting performance in schizophrenia: impact of distractors. Schizophr Res 2007, 89(1-3):339-349.
42. Ceaser A.E., Goldberg T.E., Egan M.F., McMahon R.P., Weinberger D.R., Gold J.M. Set-shifting ability and schizophrenia: a marker of clinical illness or an intermediate phenotype?. Biol Psychiatry 2008, 64(9):782-788.
43. Leeson V.C., Robbins T.W., Matheson E., Hutton S.B., Ron M.A., Barnes T.R.E., Joyce E.M. Discrimination learning, reversal, and set-shifting in first-episode schizophrenia: stability over six years and specific associations with medication type and disorganization syndrome. Biol Psychiatry 2009, 66(6):586-593.
44. Todd M.T., Niv Y., Cohen J.D. Learning to use working memory in partially observable environments through dopaminergic reinforcement. Neural Information Processing Systems 2009, Citeseer.
45. Peshkin L., Meuleau N., Kaelbling L.P. Learning policies with external memory. Proceedings of the Sixteenth International Conference (ICML'99) 1999, 307.
46. Dayan P. Bilinearity, rules, and prefrontal cortex. Front Comput Neurosci 2007, 1.
47. Bellman R. Dynamic Programming 1957, Princeton University Press.
48. Braun D.A., Mehring C., Wolpert D.M. Structure learning in action. Behavioral Brain Research 2010, 206:157-165.
49. Braun D.A., Aertsen A., Wolpert D.M., Mehring C. Motor task variation induces structural learning. Curr Biol 2009, 19(4):352-357.
50. Gershman S.J., Pesaran B., Daw N.D. Human reinforcement learning subdivides structured action spaces by learning effector-specific values. J Neurosci 2009, 29(43):13524.
51. Palminteri S., Boraud T., Lafargue G., Dubois B., Pessiglione M. Brain hemispheres selectively track the expected value of contralateral options. J Neurosci 2009, 29(43):13465.
52. David J.C., MacKay Information Theory, Inference and Learning Algorithms 2003, Cambridge University Press.