Assembling old tricks for new tasks: A neural model of instructional learning and control

Journal of Cognitive Neuroscience

Volumn 25, Issue 6, 2013, Pages 843-851

  Huang, Tsung Ren ^a   Hazy, Thomas E ^a   Herd, Seth A ^a   O'Reilly, Randall C ^a  

^a UNIVERSITY OF COLORADO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE BEHAVIOR; ANTERIOR CINGULATE; ARTICLE; BASAL GANGLION; HABITUATION; HIPPOCAMPUS; HUMAN; LEARNING; MEMORY CONSOLIDATION; MOTOR CORTEX; NEUROANATOMY; NEUROIMAGING; NEUROPHYSIOLOGY; POSTERIOR PARIETAL CORTEX; PREMOTOR CORTEX; PRIORITY JOURNAL; SIMULATION; STIMULUS RESPONSE;

EID: 84877005580     PISSN: 0898929X     EISSN: 15308898     Source Type: Journal    
DOI: 10.1162/jocn_a_00365     Document Type: Article

References (53)

1. Alvarez, P., & Squire, L. R. (1994). Memory consolidation and the medial temporal lobe: A simple network model. Proceedings of the National Academy of Sciences, U.S.A., 91, 7041-7045.
2. Andersen, R. A., & Buneo, C. A. (2002). Intentional maps in posterior parietal cortex. Annual Review of Neuroscience, 25, 189-220.
3. Ashby, F. G., Turner, B. O., & Horvitz, J. C. (2010). Cortical and basal ganglia contributions to habit learning and automaticity. Trends in Cognitive Science, 14, 191-232.
4. Bayley, P. J., Frascino, J. C., & Squire, L. R. (2005). Robust habit learning in the absence of awareness and independent of the medial temporal lobe. Nature, 436, 550-553.
5. Biele, G., Rieskamp, J., & Gonzalez, R. (2009). Computational models for the combination of advice and individual learning. Cognitive Science, 33, 206-242.
6. Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Sciences, 8, 539-546.
7. Brass, M., Wenke, D., Spengler, S., & Waszak, F. (2009). Neural correlates of overcoming interference from instructed and implemented stimulus-response associations. Journal of Neuroscience, 29, 1766-1772.
8. Brasted, P. J., Bussey, T. J., Murray, E. A., & Wise, S. P. (2005). Conditional motor learning in the nonspatial domain: Effects of errorless learning and the contribution of the fornix to one-trial learning. Behavioral Neuroscience, 119, 662-676.
9. Bunge, S. A. (2004). How we use rules to select actions: A review of evidence from cognitive neuroscience. Cognitive, Affective, and Behavioral Neuroscience, 4, 564-579.
10. Casey, B. J., Thomas, K. M., Davidson, M. C., Kunz, K., & Franzen, P. L. (2002). Dissociating striatal and hippocampal function developmentally with a stimulus-response compatibility task. Journal of Neuroscience, 22, 8647-8652.
11. Cavina-Pratesi, C., Valyear, K. F., Culham, J. C., Köhler, S., Obhi, S. S., Marzi, C. A., et al. (2006). Dissociating arbitrary stimulus-response mapping from movement planning during preparatory period: Evidence from event-related functional magnetic resonance imaging. Journal of Neuroscience, 26, 2704-2713.
12. Cohen-Kdoshay, O., & Meiran, N. (2009). The representation of instructions operates like a prepared reflex: Flanker compatibility effects that are found in the first trial following S-R instructions. Experimental Psychology, 56, 128-133.
13. Cole, M. W., Bagic, A., Kass, R., & Schneider, W. (2010). Prefrontal dynamics underlying rapid instructed task learning reverse with practice. Journal of Neuroscience, 30, 14245-14254.
14. Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201-215.
15. Dayan, P. (2007). Bilinearity, rules and prefrontal cortex. Frontiers in Computational Neuroscience, 1, 1.
16. Doll, B. B., Jacobs, W. J., Sanfey, A. G., & Frank, M. J. (2009). Instructional control of reinforcement learning: A behavioral and neurocomputational investigation. Brain Research, 1299, 74-94.
17. Duvernoy, H. M. (2005). The human hippocampus: Functional anatomy, vascularization and serial sections with MRI (3rd ed.). Berlin: Springer-Verlag.
18. Goldman-Rakic, P. S., Selemon, L. D., & Schwartz, M. L. (1984). Dual pathways connecting the dorsolateral prefrontal cortex with the hippocampal formation and parahippocampal cortex in the rhesus monkey. Neuroscience, 12, 719-743.
19. Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neuroscience, 15, 20-25.
20. Grol, M. J., de Lange, F. P., Verstraten, F. A. J., Passingham, R. E., & Toni, I. (2006). Cerebral changes during performance of overlearned arbitrary visuomotor associations. Journal of Neuroscience, 26, 117-125.
21. Hartstra, E., Kuhn, S., Verguts, T., & Brass, M. (2011). The implementation of verbal instructions: An fMRI study. Human Brain Mapping, 32, 1811-1824.
22. Hasselmo, M. E., Bodelon, C., & Wyble, B. P. (2002). A proposed function for hippocampal theta rhythm: Separate phases of encoding and retrieval enhance reversal of prior learning. Neural Computation, 14, 793-817.
23. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2006). Banishing the homunculus: Making working memory work. Neuroscience, 139, 105-118.
24. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2007). Towards an executive without a homunculus: Computational models of the prefrontal cortex/basal ganglia system. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences, 362, 1601-1613.
25. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2010). Neural mechanisms of acquired phasic dopamine responses in learning. Neuroscience and Biobehavioral Reviews, 34, 701-720.
26. Krichmar, J. L. (2008). The neuromodulatory system-A framework for survival and adaptive behavior in a challenging world. Adaptive Behavior, 16, 385-399.
27. Lavenex, P., & Amaral, D. G. (2000). Hippocampal-neocortical interaction: A hierarchy of associativity. Hippocampus, 10, 420-430.
28. Li, J., Delgado, M. R., & Phelps, E. A. (2011). How instructed knowledge modulates the neural systems of reward learning. Proceedings of the National Academy of Sciences, U.S.A., 108, 55-60.
29. Lisman, J., & Grace, A. A. (2005). The hippocampal-VTA loop: Control of entry into long-term memory. Neuron, 46, 703-713.
30. McClelland, J. L. (2001). Failures to learn and their remediation: A Hebbian account. In J. L. McClelland & S. Siegler (Eds.), Mechanisms of cognitive development: Behavioral and neural approaches (pp. 97-121). Mahwah, NJ: Erlbaum.
31. McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102, 419-457.
32. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167-202.
33. Noelle, D. C., & Cottrell, G. W. (1995). A connectionist model of instruction following. In J. D. Moore & J. F. Lehman (Eds.), Proceedings of the 17th Annual Conference of the Cognitive Science Society (pp. 369-374). Hillsdale, NJ: Erlbaum.
34. Noelle, D. C., & Cottrell, G. W. (1996). Modeling interference effects in instructed category learning. In G. W. Cottrell (Ed.), Proceedings of the 18th Annual Conference of the Cognitive Science Society (pp. 475-480). Hillsdale, NJ: Erlbaum.
35. Norman, K. A., & O'Reilly, R. C. (2003). Modeling hippocampal and neocortical contributions to recognition memory: A complementary learning systems approach. Psychological Review, 110, 611-646.
36. O'Reilly, R. C., & Frank, M. J. (2006). Making working memory work: A computational model of learning in the frontal cortex and basal ganglia. Neural Computation, 18, 283-328.
37. O'Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007). PVLV: The primary value and learned value Pavlovian learning algorithm. Behavioral Neuroscience, 121, 31-49.
38. O'Reilly, R. C., Hazy, T. E., & Herd, S. A. (in press). The Leabra cognitive architecture: How to play 20 principles with nature and win! The Oxford Handbook of Cognitive Science. Oxford: Oxford University Press
39. O'Reilly, R. C., & McClelland, J. L. (1994). Hippocampal conjunctive encoding, storage, and recall: Avoiding a tradeoff. Hippocampus, 4, 661-682.
40. O'Reilly, R. C., & Munakata, Y. (2000). Computational explorations in cognitive neuroscience: Understanding the mind by simulating the brain. Cambridge, MA: MIT Press.
41. Pasupathy, A., & Miller, E. K. (2005). Different time courses for learning-related activity in the prefrontal cortex and striatum. Nature, 433, 873-876.
42. Paus, T. (2001). Primate anterior cingulate cortex: Where motor control, drive and cognition interface. Nature Reviews Neuroscience, 2, 417-424.
43. Petrides, M. (1985). Deficits on conditional associative-learning task after frontal-and temporal-lobe lesions in man. Neuropsychologia, 23, 601-614.
44. Petrides, M. (1997). Visuo-motor conditional associative learning after frontal and temporal lesions in the human brain. Neuropsychologia, 35, 989-997.
45. Rossetti, Y., Revol, P., McIntosh, R., Pisella, L., Rode, G., Danckert, J., et al. (2005). Visually guided reaching: Posterior parietal lesions cause a switch from fast visuomotor to slow cognitive control. Neuropsychologia, 43, 162-177.
46. Ruge, H., & Wolfensteller, U. (2010). Rapid formation of pragmatic rule representations in the human brain during instruction-based learning. Cerebral Cortex, 20, 1656-1667.
47. Schindler, I., Rice, N. J., McIntosh, R. D., Rossetti, Y., Vighetto, A., & Milner, A. D. (2004). Automatic avoidance of obstacles is a dorsal stream function: Evidence from optic ataxia. Nature Neuroscience, 7, 779-784.
48. Simons, J. S., & Spiers, H. (2003). Prefrontal and medial temporal lobe interactions in long-term memory. Nature Reviews Neuroscience, 4, 637-648.
49. Suzuki, W. A. (2007). Integrating associative learning signals across the brain. Hippocampus, 17, 842-850.
50. Walsh, M. M., & Anderson, J. R. (2011). Modulation of the feedback-related negativity by instruction and experience. Proceedings of the National Academy of Sciences, U.S.A., 108, 19048-19053.
51. Wenke, D., Gaschler, R., Nattkemper, D., & Frensch, P. A. (2009). Strategic influences on implementing instructions for future actions. Psychological Research, 73, 587-601.
52. Wise, S. P., & Murray, E. A. (1999). Role of the hippocampal system in conditional motor learning: Mapping antecedents to action. Hippocampus, 9, 101-117.
53. Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews Neuroscience, 7, 464-476.