Aversive Pavlovian responses affect human instrumental motor performance

Frontiers in Neuroscience

Volumn , Issue OCT, 2012, Pages

  Rigoli, Francesco ^a,b   Pavone, Enea Francesco ^c,d   Pezzulo, Giovanni ^b,e  

^a UNIVERSITY OF SIENA   (Italy)

^b CNR   (Italy)

^c SAPIENZA UNIVERSITY OF ROME   (Italy)

^d IRCCS FONDAZIONE SANTA LUCIA   (Italy)

^e CNR Consiglio Nazionale delle Ricerche   (Italy)

Author keywords

Controllability; Goal directed; Habitual; Pavlovian; Reinforcement learning

Indexed keywords

ADULT; ARTICLE; AVERSIVE BEHAVIOR; AVERSIVE PAVLOVIAN RESPONSE; CONDITIONING; ECONOMICS; FEMALE; HUMAN; INSTRUMENTAL MOTOR PERFORMANCE; MALE; MENTAL DISEASE; MOTOR PERFORMANCE; NORMAL HUMAN; SOCIAL BEHAVIOR; TASK PERFORMANCE;

EID: 84870953994     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2012.00134     Document Type: Article

References (66)

1. Amat, J., Baratta, M., Paul, E., Bland, S., Watkins, L., and Maier, S. (2005). Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphé nucleus. Nat. Neurosci. 8, 365-371.
2. Balleine, B., and Dickinson, A. (1998). Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37, 407-419.
3. Balleine, B., and O'Doherty, J. (2009). Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacol-ogy 35, 48-69.
4. Blake, D., Strata, F., Churchland, A., and Merzenich, M. (2002). Neural correlates of instrumental learning in primary auditory cortex. Proc. Natl. Acad. Sci. U.S.A. 99, 10114.
5. Blanchard, R., and Blanchard,D. (1989). Attack and defense in rodents as ethoexperimental models for the study of emotion. Prog. Neu-ropsychopharmacol. Biol. Psychiatry 13:S3-S14.
6. Bornstein, A. M., and Daw, N. (2011). Multiplicity of control in the basal ganglia: computational roles of stri-atal subregions. Curr. Opin. Neuro-biol. 21, 374-380.
7. Boureau, Y., and Dayan, P. (2010). Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharma-cology 36, 74-97.
8. Bray, S., Rangel, A.,Shimojo, S., Balleine, B., O'Doherty, and J. P. (2008). The neural mechanisms underlying the influence of Pavlovian cues on human decision making. J. Neurosci. 28, 5861.
9. Breland, K., and Breland, M. (1966). Animal Behavior. Oxford: Macmillan.
10. Brown, P., and Jenkins,H. (1968). Auto-shaping of the pigeon's key-peck. J. Exp. Anal. Behav. 11, 1.
11. Butler, T., Pan, H., Tuescher, O., Engelien, A., Goldstein, M., Epstein, J., et al. (2007). Human fear-related motor neurocircuitry. Neuroscience 150, 1-7.
12. Colwill, R., and Rescorla, R. (1988). Associations between the discriminative stimulus and the reinforcer in instrumental learning. J. Exp. Psychol. Anim. Behav. Process. 14, 155.
13. Crockett, M. J., Clark, L., and Robbins, T. W. (2009). Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J. Neurosci. 29, 11993.
14. Davis, M. (1992). The role of the amygdala in fear and anxiety. Annu. Rev. Neurosci. 15, 353-375.
15. Trial-by-trial data analysis using computational models
16. Daw, N., Niv, Y., and Dayan, P. (2005). Uncertainty-based competition between prefrontal and dor-solateralstriatal systems for behavioral control. Nat. Neurosci. 8, 1704-1711.
17. Dayan, P., Niv, Y., Seymour, B., and Daw, N. D. (2006). The misbehavior of value and the discipline of the will. Neural Netw. 19, 1153-1160.
18. Dayan,P.,and Seymour,B. (2008)."Val-ues and actions in aversion," in Neu-roeconomics: Decision Making and the Brain, eds P.W. Glimcher,E.Fehr, A. Rangel,C.Camerer,andR.A.Pol-drack (New York: Academic Press), 175-191.
19. Dickinson, A., and Pearce, J. (1977). Inhibitory interactions between appetitive and aversive stimuli. Psychol. Bull. 84, 690.
20. Doyon, J., Penhune, V., and Ungerlei-der, L. G. (2003). Distinct contribution of the cortico-striatal and cortico-cerebellar systems to motor skill learning. Neuropsychologia 41, 252-262.
21. Eisterbrook, J. A. (1959). The effect of emotion on cue utilization and the organization of behavior. Psychol. Rev. 66, 183-201.
22. Estes, W., and Skinner, B. (1941). Some quantitative properties of anxiety. J. Exp. Psychol. 29, 390.
23. Fanselow, M. (1994). Neural organization of the defensive behavior system responsible for fear. Psychon. Bull. Rev. 1, 429-438.
24. A functional behavioristic approach to aversively motivated behavior: predatory imminence as a determinant of the topography of defensive behavior
25. Glascher, J., Daw, N., Dayan, P., and O'Doherty, J. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585-595.
26. Gray, J. (1982). The Psychology of Fear and Stress. Cambridge: University Press.
27. Guitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., et al. (2011). Action dominates valence in anticipatory representations in the human stria-tum and dopaminergic midbrain. J. Neurosci. 31, 7867.
28. Hershberger, W. (1986). An approach through the looking-glass. Learn. Behav. 14, 443-451.
29. Holland, P. (2004). Relations between Pavlovian-instrumental transfer and reinforcer devaluation. J. Exp. Psy-chol. Anim. Behav. Process. 30, 104.
30. Occasion setting in Pavlovian conditioning
31. Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding
32. Huys, Q., and Dayan, P. (2009). A Bayesian formulation of behavioral control. Cognition 113, 314-328.
33. Keay, K., and Bandler, R. (2001). Parallel circuits mediating distinct emotional coping reactions to different types of stress. Neurosci. Biobehav. Rev. 25, 669-678.
34. Langer, L. J., and Imber, L. G. (1979). When practice makes imperfect: the debilitating effects of over-learning. J. Pers. Soc. Psychol. 37, 2014-2024.
35. Lipp, O. V., and Edwards, M. S. (2002). Effect of instructed extinction on verbal and autonomic indices of Pavlovian learning with fear-relevant and fear-irrelevant conditioned stimuli. J. Psychophysiol. 16, 176-186.
36. Loewenstein,G. F., and O'Donoghue, T. (2004). Animal Spirits: Affective and Deliberative Processes in Economic Behavior: http://ssrn.com/abstract=539843 or http://dx.doi.org/10.2139/ssrn.539843
37. Maier, S., and Watkins, L. (2005). Stressor controllability and learned helplessness: the roles of the dorsal raphé nucleus, serotonin, and corticotropin-releasing factor. Neu-rosci. Biobehav. Rev. 29, 829-841.
38. McNaughton, N., and Corr, P. (2004). A two-dimensional neuropsychol-ogy of defense: fear/anxiety and defensive distance. Neurosci. Biobe-hav. Rev. 28, 285-305.
39. Mineka, S., Cook, M., and Miller, S. (1984). Fear conditioned with escapable and inescapable shock: Effects of a feedback stimulus. J. Exp. Psychol. Anim. Behav. Process. 10, 307.
40. Mineka, S., and Hendersen, R. (1985). Controllability and predictability in acquired motivation.Annu.Rev. Psy-chol. 36, 495-529.
41. Mobbs, D., Marchant, J., Hassabis, D., Seymour, B., Tan, G., Gray, M., Petrovic, P., Dolan, R., and Frith, C. (2009). From threat to fear: the neural organization of defensive fear systems in humans. J. Neurosci. 29, 12236.
42. Morse, W., Mead, R., and Kelleher, R. (1967). Modulation of elicited behavior by a fixed-interval schedule of electric shock presentation. Science 157, 215.
43. Neiss, R. (1988). Reconceptualizing Arousal: Psychological States in Motor Performance. Psychol. Bull. 103, 345-366.
44. Niv, Y., Daw, N., Joel, D., and Dayan, P. (2007). Tonic dopamine: opportunity costs and the control of response vigor. Psy-chopharmacology (Berl.) 191, 507-520.
45. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., and Dolan, R. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452.
46. Overmier, J. B., Bull, J. A., and Pack, K. (1971).Oninstrumental response interaction as explaining the influ-encesofPavlovian CS+suponavoid-ance behavior. Learn. Motiv. 2, 103-112.
47. Padoa-Schioppa, C., and Assad, J. (2006). Neurons in the orbitofrontal cortex encode economic value. Nature 441, 223-226.
48. Pennartz, C. M., Ito, R., Verschure, P. F., Battaglia, F. P., and Robbins, T. W. (2011). The hippocampal-striatal axis in learning, prediction and goal-directed behavior. Trends Neurosci. 34, 548-559.
49. Pezzulo, G., and Castelfranchi, C. (2009). Thinking as the control of imagination: a conceptual framework for goal-directed systems. Psy-chol. Res. 73, 559-577.
50. Pezzulo, G., and Rigoli, F. (2011). The value of foresight: How prospection affects decision-making. Front. Decis. Neurosci. 5:79. doi:10.3389/fnins.2011.00079
51. Pleger, B., Blankenburg, F., Ruff, C., Driver, J., and Dolan, R. (2008). Reward facilitates tactile judgments and modulates hemodynamic responses in human primary somatosensory cortex. J. Neurosci. 28, 8161.
52. Rescorla, R., and Solomon, R. (1967). Two-process learning theory: relationships between Pavlovian conditioning and instrumental learning. Psychol. Rev. 74, 151.
53. Rigoli, F., Pavone, E., and Pezzulo, G. (2011). "Interaction of goal-directed and Pavlovian systems in aversive domains," in Proceedings of CogSci 2011, Boston.
54. Schell, A. M., and Dawson, M. E., and Marinkovic, K. (1991). Effects of potentially phobic conditioned stimuli on retention, reconditioning, and extinction of the conditioned skin conductance response. Psychophysiology 28, 140-153.
55. Schoenbaum, G., Setlow, B., Sad-doris, M., and Gallagher, M. (2003). Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39, 855-867.
56. Simon, D., and Daw, N. (2011). Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. 31, 5526.
57. Solway,A.,and Botvinick,M.M. (2012). Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. Psychol. Rev. 119, 120-154.
58. Sutton, R., and Barto, A. (1998). Reinforcement Learning: An Introduction. Cambridge: University Press.
59. Talmi, D., Seymour, B., Dayan, P., and Dolan, R. J. (2008). Human Pavlovian-instrumental transfer. J. Neurosci. 28, 360.
60. Vuilleumier, P., and Driver, J. (2007). Modulation of visual processing by attention and emotion: window son causal interactions between human brain regions. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 837.
61. Williams, D., and Williams, H. (1969). Auto-maintenance in the pigeon: sustained pecking despite contingentnon-reinforcement. J. Exp. Anal. Behav. 12, 511.
62. Wunderlich, K.,Dayan,P.,and Dolan,R. (2012). Mapping value-based planning and extensively trained choice in the human brain. Nat. Neurosci. 15, 786-793.
63. Yerkes, R. M., and Dodson, J. D. (1908). The Relationship of Strength of Stimulus to Rapidity of Habit-Formation. J. Comp. Neurol. Psychol. 18, 459-482.
64. Yin, H., Ostlund, S., and Balleine, B. (2008). Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. Eur. J. Neurosci. 28, 1437-1448.