Distinct fronto-striatal couplings reveal the double-faced nature of response–outcome relations in instruction-based learning

Cognitive, Affective and Behavioral Neuroscience

Volumn 15, Issue 2, 2015, Pages 349-364

  Ruge, Hannes ^a   Wolfensteller, Uta ^a  

^a DRESDEN UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

Differential outcomes; Fronto hippocampal coupling; Fronto striatal coupling; Goal directed action; Ideomotor theory; Instruction based learning; Model based learning

Indexed keywords

OXYGEN;

ADOLESCENT; ADULT; ANALYSIS OF VARIANCE; AUDITORY STIMULATION; BLOOD; BRAIN MAPPING; CORPUS STRIATUM; FACE; FEMALE; FRONTAL LOBE; HUMAN; IMAGE PROCESSING; INSTRUMENTAL CONDITIONING; MALE; NERVE TRACT; NUCLEAR MAGNETIC RESONANCE IMAGING; PATTERN RECOGNITION; PHOTOSTIMULATION; PHYSIOLOGY; REACTION TIME; VASCULARIZATION; YOUNG ADULT;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; ANALYSIS OF VARIANCE; BRAIN MAPPING; CONDITIONING, OPERANT; CORPUS STRIATUM; FACE; FEMALE; FRONTAL LOBE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; NEURAL PATHWAYS; OXYGEN; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION; REACTION TIME; YOUNG ADULT;

EID: 84940002733     PISSN: 15307026     EISSN: None     Source Type: Journal    
DOI: 10.3758/s13415-014-0325-4     Document Type: Article

References (70)

1. Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9, 357–381.
2. Ashby, F. G., Turner, B. O., & Horvitz, J. C. (2010). Cortical and basal ganglia contributions to habit learning and automaticity. Trends in Cognitive Sciences, 14, 208–215. doi:10.1016/j.tics.2010.02.001
3. Balleine, B. W., & Ostlund, S. B. (2007). Still at the choice-point. Annals of the New York Academy of Sciences, 1104, 147–171.
4. Bergmann, H. C., Rijpkema, M., Fernández, G., & Kessels, R. P. C. (2012). Distinct neural correlates of associative working memory and long-term memory encoding in the medial temporal lobe. NeuroImage, 63, 989–997.
5. Brewer, J. B., Zhao, Z., Desmond, J. E., Glover, G. H., & Gabrieli, J. D. E. (1998). Making memories: Brain activity that predicts how well visual experience will be remembered. Science, 281, 1185–1187. doi:10.1126/science.281.5380.1185
6. Brovelli, A., Nazarian, B., Meunier, M., & Boussaoud, D. (2011). Differential roles of caudate nucleus and putamen during instrumental learning. NeuroImage, 57, 1580–1590.
7. Clark, J. J., Hollon, N. G., & Phillips, P. E. M. (2012). Pavlovian valuation systems in learning and decision making. Current Opinion in Neurobiology, 22, 1054–1061.
8. Cole, M. W., Laurent, P., & Stocco, A. (2012). Rapid instructed task learning: A new window into the human brain’s unique capacity for flexible cognitive control. Cognitive, Affective, & Behavioral Neuroscience, 13, 1–22. doi:10.3758/s13415-012-0125-7
9. Colwill, R. M., & Rescorla, R. A. (1985). Postconditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology: Animal Behavior Processes, 11, 120–132. doi:10.1037/0097-7403.11.1.120
10. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69, 1204–1215.
11. de Wit, S., & Dickinson, A. (2009). Associative theories of goal-directed behaviour: A case for animal–human translational models. Psychological Research, 73, 463–476.
12. Dolan, R. J., & Dayan, P. (2013). Goals and habits in the brain. Neuron, 80, 312–325.
13. Doll, B. B., Jacobs, W. J., Sanfey, A. G., & Frank, M. J. (2009). Instructional control of reinforcement learning: a behavioral and neurocomputational investigation. Brain Research, 1299, 74–94.
14. Doll, B. B., Simon, D. A., & Daw, N. D. (2012). The ubiquity of model-based reinforcement learning. Current Opinion in Neurobiology, 22, 1075–1081.
15. Dumontheil, I., Thompson, R., & Duncan, J. (2011). Assembly and use of new task rules in fronto-parietal cortex. Journal of Cognitive Neuroscience, 23, 168–182.
16. Duncan, J., Emslie, H., Williams, P., Johnson, R., & Freer, C. (1996). Intelligence and the frontal lobe: the organization of goal-directed behavior. Cognitive Psychology, 30, 257–303.
17. Elsner, B., & Hommel, B. (2001). Effect anticipation and action control. Journal of Experimental Psychology: Human Perception and Performance, 27, 229–240.
18. Elsner, B., Hommel, B., Mentschel, C., Drzezga, A., Prinz, W., Conrad, B., & Siebner, H. (2002). Linking actions and their perceivable consequences in the human brain. NeuroImage, 17, 364–372.
19. Estévez, A. F., Fuentes, L. J., Mari-Beffa, P., González, C., & Alvarez, D. (2001). The differential outcome effect as a useful tool to improve conditional discrimination learning in children. Learning and Motivation, 32, 48–64.
20. Fouquet, C., Babayan, B. M., Watilliaux, A., Bontempi, B., Tobin, C., & Rondi-Reig, L. (2013). Complementary roles of the hippocampus and the dorsomedial striatum during spatial and sequence-based navigation behavior. PLoS ONE, 8, e67232. doi:10.1371/journal.pone.0067232
21. Friedrich, A. M., & Zentall, T. R. (2011). A differential-outcome effect in pigeons using spatial hedonically nondifferential outcomes. Learning & Behavior, 39, 68–78.
22. Friston, K. J., Buechel, C., Fink, G. R., Morris, J., Rolls, E., & Dolan, R. J. (1997). Psychophysiological and modulatory interactions in neuroimaging. NeuroImage, 6, 218–229.
23. Gitelman, D. R., Penny, W. D., Ashburner, J., & Friston, K. J. (2003). Modeling regional and psychophysiologic interactions in fMRI: The importance of hemodynamic deconvolution. NeuroImage, 19, 200–207.
24. Glascher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66, 585–595.
25. Gourley, S. L., Lee, A. S., Howell, J. L., Pittenger, C., & Taylor, J. R. (2010). Dissociable regulation of instrumental action within mouse prefrontal cortex. European Journal of Neuroscience, 32, 1726–1734.
26. Greenwald, A. G. (1970). Sensory feedback mechanisms in performance control—with special reference to ideo-motor mechanism. Psychological Review, 77, 73–99. doi:10.1037/0033-295X.77.1.73
27. Hartstra, E., Kuhn, S., Verguts, T., & Brass, M. (2011). The implementation of verbal instructions: An fMRI study. Human Brain Mapping, 32, 1811–1824. doi:10.1002/hbm.21152
28. Huang, T.-R., Hazy, T. E., Herd, S. A., & O’Reilly, R. C. (2013). Assembling old tricks for new tasks: A neural model of instructional learning and control. Journal of Cognitive Neuroscience, 25, 843–851. doi:10.1162/jocn_a_00365
29. Johnson, A., van der Meer, M. A. A., & Redish, A. D. (2007). Integrating hippocampus and striatum in decision-making. Current Opinion in Neurobiology, 17, 692–697.
30. Kennedy, P. J., & Shapiro, M. L. (2009). Motivational states activate distinct hippocampal representations to guide goal-directed behaviors. Proceedings of the National Academy of Sciences, 106, 10805–10810.
31. Killcross, S., & Coutureau, E. (2003). Coordination of actions and habits in the medial prefrontal cortex of rats. Cerebral Cortex, 13, 400–408.
32. Legge, E. L. G., & Spetch, M. L. (2009). The differential outcomes effect (DOE) in spatial localization: An investigation with adults. Learning and Motivation, 40, 313–328.
33. Li, J., Delgado, M. R., & Phelps, E. A. (2011). How instructed knowledge modulates the neural systems of reward learning. Proceedings of the National Academy of Sciences, 108, 55–60.
34. Luria, A. R. (1973). The working brain: An introduction to neuropsychology. New York, NY: Basic Books.
35. McLaren, D. G., Ries, M. L., Xu, G., & Johnson, S. C. (2012). A generalized form of context-dependent psychophysiological interactions (gPPI): A comparison to standard approaches. NeuroImage, 61, 1277–1286.
36. McNamee, D., Rangel, A., & O’Doherty, J. P. (2013). Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex. Nature Neuroscience, 16, 479–485.
37. Melcher, T., Winter, D., Hommel, B., Pfister, R., Dechent, P., & Gruber, O. (2013). The neural substrate of the ideomotor principle revisited: Evidence for asymmetries in action-effect learning. Neuroscience, 231, 13–27.
38. Meyer-Lindenberg, A. S., Olsen, R. K., Kohn, P. D., Brown, T., Egan, M. F., Weinberger, D. R., & Berman, K. F. (2005). Regionally specific disturbance of dorsolateral prefrontal–hippocampal functional connectivity in schizophrenia. Archives of General Psychiatry, 62, 379–386. doi:10.1001/archpsyc.62.4.379
39. Mok, L. W., & Overmier, J. B. (2007). The differential outcomes effect in normal human adults using a concurrent-task within-subjects design and sensory outcomes. Psychological Record, 57, 187–200.
40. Monsell, S. (1996). Control of mental processes. In V. Bruce (Ed.), Unsolved mysteries of the mind (pp. 93–148). Hove, UK: Erlbaum/Taylor & Francis.
41. Noelle, D. C. (1997). A connectionist model of instructed learning. Unpublished PhD dissertation, University of California, San Diego.
42. Noonan, M. P., Kolling, N., Walton, M. E., & Rushworth, M. F. S. (2012). Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement. European Journal of Neuroscience, 35, 997–1010.
43. Noonan, M. P., Mars, R. B., & Rushworth, M. F. S. (2011). Distinct roles of three frontal cortical areas in reward-guided behavior. Journal of Neuroscience, 31, 14399–14412.
44. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452–454. doi:10.1126/science.1094285
45. O’Reilly, R. C., & Rudy, J. W. (2001). Conjunctive representations in learning and memory: Principles of cortical and hippocampal function. Psychological Review, 108, 311–345.
46. Oberauer, K. (2009). Design for a working memory. In B. H. Ross (Ed.), The psychology of learning and motivation: advances in research and theory (Vol. 51, pp. 45–100). San Diego, CA: Elsevier Academic Press. doi:10.1016/S0079-7421(09)51002-X
47. Ramamoorthy, A., & Verguts, T. (2012). Word and deed: A computational model of instruction following. Brain Research, 1439, 54–65.
48. Ranganath, C., Cohen, M. X., & Brozinsky, C. J. (2005). Working memory maintenance contributes to long-term memory formation: Neural and behavioral evidence. Journal of Cognitive Neuroscience, 17, 994–1010. doi:10.1162/0898929054475118
49. Rolls, E. T. (2010). A computational theory of episodic memory formation in the hippocampus. Behavioural Brain Research, 215, 180–196.
50. Rorden, C., Karnath, H.-O., & Bonilha, L. (2007). Improving lesion–symptom mapping. Journal of Cognitive Neuroscience, 19, 1081–1088.
51. Rose, M., Haider, H., & Büchel, C. (2010). The emergence of explicit memory during learning. Cerebral Cortex, 20, 2787–2797.
52. Ruge, H., Krebs, R. M., & Wolfensteller, U. (2012). Early markers of ongoing action-effect learning. Frontiers in Psychology, 3, 522.
53. Ruge, H., & Wolfensteller, U. (2010). Rapid formation of pragmatic rule representations in the human brain during instruction-based learning. Cerebral Cortex, 20, 1656–1667.
54. Ruge, H., & Wolfensteller, U. (2013). Functional integration processes underlying the instruction-based learning of novel goal-directed behaviors. NeuroImage, 68, 162–172.
55. Seger, C. A., & Spiering, B. J. (2011). A critical review of habit learning and the basal ganglia. Frontiers in Systems Neuroscience, 5, 66. doi:10.3389/fnsys.2011.00066
56. Shin, Y. K., Proctor, R. W., & Capaldi, E. J. (2010). A review of contemporary ideomotor theory. Psychological Bulletin, 136, 943–974.
57. Stocco, A., Lebiere, C., O’Reilly, R. C., & Anderson, J. R. (2012). Distinct contributions of the caudate nucleus, rostral prefrontal cortex, and parietal cortex to the execution of instructed tasks. Cognitive, Affective, & Behavioral Neuroscience, 12, 611–628. doi:10.3758/s13415-012-0117-7
58. Summerfield, C., Greene, M., Wager, T., Egner, T., Hirsch, J., & Mangels, J. (2006). Neocortical connectivity during episodic memory formation. PLoS Biology, 4, e128. doi:10.1371/journal.pbio.0040128
59. Tanaka, S. C., Balleine, B. W., & O’Doherty, J. P. (2008). Calculating consequences: Brain systems that encode the causal effects of actions. Journal of Neuroscience, 28, 6750–6755.
60. Tolman, E. C. (1948). Cognitive maps in rats and men. Psychological Review, 55, 189–208. doi:10.1037/h0061626
61. Trapold, M. A. (1970). Are expectancies based upon different positive reinforcing events discriminably different. Learning and Motivation, 1, 129–140.
62. Tricomi, E. M., Delgado, M. R., & Fiez, J. A. (2004). Modulation of caudate activity by action contingency. Neuron, 41, 281–292.
63. Tulving, E., & Markowitsch, H. J. (1998). Episodic and declarative memory: Role of the hippocampus. Hippocampus, 8, 198–204.
64. Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., & Joliot, M. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. NeuroImage, 15, 273–289. doi:10.1006/nimg.2001.0978
65. Urcuioli, P. J. (2005). Behavioral and associative effects of differential outcomes in discrimination learning. Learning & Behavior, 33, 1–21.
66. Valentin, V. V., Dickinson, A., & O’Doherty, J. P. (2007). Determining the neural substrates of goal-directed learning in the human brain. Journal of Neuroscience, 27, 4019–4026.
67. Wolfensteller, U., & Ruge, H. (2011). On the timescale of stimulus-based action-effect learning. Quarterly Journal of Experimental Psychology, 64, 1273–1289. doi:10.1080/17470218.2010.546417
68. Wolfensteller, U., & Ruge, H. (2012). Frontostriatal mechanisms in instruction-based learning as a hallmark of flexible goal-directed behavior. Frontiers in Psychology, 3, 192. doi:10.3389/fpsyg.2012.00192
69. Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews Neuroscience, 7, 464–476.
70. Zilli, E. A., & Hasselmo, M. E. (2008). Modeling the role of working memory and episodic memory in behavioral tasks. Hippocampus, 18, 193–209.