Social cognition as reinforcement learning: Feedback modulates emotion inference

Journal of Cognitive Neuroscience

Volumn 28, Issue 9, 2016, Pages 1270-1282

  Zaki, Jamil ^a   Kallman, Seth ^b   Elliott Wimmer, G ^c   Ochsner, Kevin ^b   Shohamy, Daphna ^b  

^a STANFORD UNIVERSITY   (United States)

^b Och Spine at New York Presbyterian Hospitals   (United States)

^c UNIVERSITY MEDICAL CENTER HAMBURG EPPENDORF   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIORAL RESEARCH; COGNITIVE SYSTEMS; NEUROPHYSIOLOGY;

DOMAIN SPECIFIC; FEEDBACK LEARNING; INTERNAL STATE; REINFORCEMENT LEARNING MODELS; REINFORCEMENT MECHANISMS; SINGLE SHOTS; SOCIAL COGNITION; SOCIAL INFORMATION PROCESSING;

REINFORCEMENT LEARNING;

ADULT; ARTICLE; ASSOCIATION; DECISION MAKING; EMOTION; FEEDBACK SYSTEM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; PRIORITY JOURNAL; REINFORCEMENT; SOCIAL COGNITION; VENTRAL STRIATUM; VENTROMEDIAL PREFRONTAL CORTEX; BRAIN; BRAIN MAPPING; NEUROPSYCHOLOGICAL TEST; NUCLEAR MAGNETIC RESONANCE IMAGING; PATTERN RECOGNITION; PERCEPTION; PHYSIOLOGY; PSYCHOLOGICAL FEEDBACK; PSYCHOLOGICAL MODEL; SPEECH PERCEPTION; YOUNG ADULT;

BRAIN; BRAIN MAPPING; CUES; EMOTIONS; FEEDBACK, PSYCHOLOGICAL; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; MODELS, PSYCHOLOGICAL; NEUROPSYCHOLOGICAL TESTS; PATTERN RECOGNITION, VISUAL; REINFORCEMENT (PSYCHOLOGY); SOCIAL PERCEPTION; SPEECH PERCEPTION; YOUNG ADULT;

EID: 84982786536     PISSN: 0898929X     EISSN: 15308898     Source Type: Journal    
DOI: 10.1162/jocn_a_00978     Document Type: Article

References (84)

1. Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M. (2006). Prediction error as a linear function of reward probability is coded in human nucleus accumbens. Neuroimage, 31, 790-795.
2. Aviezer, H., Hassin, R. R., Ryan, J., Grady, C., Susskind, J., Anderson, A., et al. (2008). Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science, 19, 724-732.
3. Barch, D. M., Braver, T. S., Sabb, F. W., & Noll, D. C. (2000). Anterior cingulate and the monitoring of response conflict: Evidence from an fMRI study of overt verb generation. Journal of Cognitive Neuroscience, 12, 298-309.
4. Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497-529.
5. Behrens, T. E., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. (2008). Associative learning of social value. Nature, 456, 245-249.
6. Behrens, T. E., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10, 1214-1221.
7. Bhanji, J. P., & Beer, J. S. (2013). Dissociable neural modulation underlying lasting first impressions, changing your mind for the better, and changing it for the worse. Journal of Neuroscience, 33, 9337-9344.
8. Boorman, E. D., O’Doherty, J. P., Adolphs, R., & Rangel, A. (2013). The behavioral and neural mechanisms underlying the tracking of expertise. Neuron, 80, 1558-1571.
9. Braams, B. R., Güroğlu, B., de Water, E., Meuwese, R., Koolschijn, P. C., Peper, J. S., et al. (2013). Reward-related neural responses are dependent on the beneficiary. Social Cognitive and Affective Neuroscience, 9, 1030-1037.
10. Bromberg-Martin, E. S., & Hikosaka, O. (2009). Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron, 63, 119.
11. Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010). Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron, 68, 815-834.
12. Buckner, R. L., & Carroll, D. C. (2007). Self-projection and the brain. Trends in Cognitive Sciences, 11, 49-57.
13. Burke, C. J., Tobler, P. N., Baddeley, M., & Schultz, W. (2010). Neural mechanisms of observational learning. Proceedings of the National Academy of Sciences, U.S.A., 107, 14431-14436.
14. Carroll, J. M., & Russell, J. A. (1996). Do facial expressions signal specific emotions? Judging emotion from the face in context. Journal of Personality and Social Psychology, 70, 205-218.
15. Carter, C. S., Macdonald, A. M., Botvinick, M., Ross, L. L., Stenger, V. A., Noll, D., et al. (2000). Parsing executive processes: Strategic vs. evaluative functions of the anterior cingulate cortex. Proceedings of the National Academy of Sciences, U.S.A., 97, 1944-1948.
16. Cavanna, A. E., & Trimble, M. R. (2006). The precuneus: A review of its functional anatomy and behavioural correlates. Brain, 129, 564-583.
17. Chevallier, C., Kohls, G., Troiani, V., Brodkin, E. S., & Schultz, R. T. (2012). The social motivation theory of autism. Trends in Cognitive Sciences, 16, 231-239.
18. Cikara, M., & Van Bavel, J. J. (2014). The neuroscience of intergroup relations: An integrative review. Perspectives on Psychological Science, 9, 245-274.
19. Contreras, J. M., Banaji, M. R., & Mitchell, J. P. (2012). Dissociable neural correlates of stereotypes and other forms of semantic knowledge. Social Cognitive and Affective Neuroscience, 7, 764-770.
20. Corbetta, M., Patel, G., & Shulman, G. L. (2008). The reorienting system of the human brain: From environment to theory of mind. Neuron, 58, 306-324.
21. Dale, A. M. (1999). Optimal experimental design for event-related fMRI. Human Brain Mapping, 8, 109-114.
22. Daw, N. D. (2009). Trial-by-trial data analysis using computational models. In M. R. Delgado, E. A. Phelps, & T. W. Robbins (Eds.), Decision making, affect, and learning: Attention and performance XXIII (pp. 3-38). New York: Oxford University Press.
23. Daw, N. D., & Doya, K. (2006). The computational neurobiology of learning and reward. Current Opinion in Neurobiology, 16, 199-204.
24. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441, 876-879.
25. Dawes, C. T., Loewen, P. J., Schreiber, D., Simmons, A. N., Flagan, T., McElreath, R., et al. (2012). Neural basis of egalitarian behavior. Proceedings of the National Academy of Sciences, U.S.A., 109, 6479-6483.
26. Delgado, M. R., Miller, M. M., Inati, S., & Phelps, E. A. (2005). An fMRI study of reward-related probability learning. Neuroimage, 24, 862-873.
27. Fisher, R. (1932). Statistical methods for research workers. London: Oliver & Boyd.
28. Foerde, K., & Shohamy, D. (2011). Feedback timing modulates brain systems for learning in humans. Journal of Neuroscience, 31, 13157-13167.
29. Freeman, J. B., & Ambady, N. (2011). A dynamic interactive theory of person construal. Psychological Review, 118, 247-279.
30. Frith, C. D., & Frith, U. (2012). Mechanisms of social cognition. Annual Review of Psychology, 63, 287-313.
31. Gilbert, D. (1998). Ordinary personology. In D. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The handbook of social psychology (4th ed., pp. 89-150). New York: McGraw Hill.
32. Gläscher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66, 585-595.
33. Grabenhorst, F., & Rolls, E. T. (2011). Value, pleasure and choice in the ventral prefrontal cortex. Trends in Cognitive Sciences, 15, 56-67.
34. Gusnard, D. A., Akbudak, E., Shulman, G. L., & Raichle, M. E. (2001). Medial prefrontal cortex and self-referential mental activity: Relation to a default mode of brain function. Proceedings of the National Academy of Sciences, U.S.A., 98, 4259-4264.
35. Harbaugh, W. T., Mayr, U., & Burghart, D. R. (2007). Neural responses to taxation and voluntary giving reveal motives for charitable donations. Science, 316, 1622-1625.
36. Hare, T. A., O’Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008). Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. Journal of Neuroscience, 28, 5623-5630.
37. Izuma, K., Saito, D. N., & Sadato, N. (2008). Processing of social and monetary rewards in the human striatum. Neuron, 58, 284-294.
38. Klucharev, V., Hytonen, K., Rijpkema, M., Smidts, A., & Fernandez, G. (2009). Reinforcement learning signal predicts social conformity. Neuron, 61, 140-151.
39. Koster-Hale, J., & Saxe, R. (2013). Theory of mind: A neural prediction problem. Neuron, 79, 836-848.
40. Lin, A., Adolphs, R., & Rangel, A. (2012). Social and monetary reward learning engage overlapping neural substrates. Social Cognitive and Affective Neuroscience, 7, 274-281.
41. Macrae, C. N., Moran, J. M., Heatherton, T. F., Banfield, J. F., & Kelley, W. M. (2004). Medial prefrontal activity predicts memory for self. Cerebral Cortex, 14, 647-654.
42. Mayer, J. D., DiPaolo, M., & Salovey, P. (1990). Perceiving affective content in ambiguous visual stimuli: A component of emotional intelligence. Journal of Personality Assessment, 54, 772-781.
43. McClure, S. M., Berns, G. S., & Montague, P. R. (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron, 38, 339-346.
44. McCoy, A. N., & Platt, M. L. (2005). Risk-sensitive neurons in macaque posterior cingulate cortex. Nature Neuroscience, 8, 1220-1227.
45. Mende-Siedlecki, P., Baron, S. G., & Todorov, A. (2013). Diagnostic value underlies asymmetric updating of impressions in the morality and ability domains. Journal of Neuroscience, 33, 19406-19415.
46. Mende-Siedlecki, P., Cai, Y., & Todorov, A. (2012). The neural dynamics of updating person impressions. Social Cognitive and Affective Neuroscience, 8, 623-631.
47. Meyer, M. L., Spunt, R. P., Berkman, E. T., Taylor, S. E., & Lieberman, M. D. (2012). Evidence for social working memory from a parametric functional MRI study. Proceedings of the National Academy of Sciences, U.S.A., 109, 1883-1888.
48. Mitchell, J. P. (2009a). Inferences about mental states. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences, 364, 1309-1316.
49. Mitchell, J. P. (2009b). Social psychology as a natural kind. Trends in Cognitive Sciences, 13, 246-251.
50. Mobbs, D., Yu, R., Meyer, M., Passamonti, L., Seymour, B., Calder, A. J., et al. (2009). A key role for similarity in vicarious reward. Science, 324, 900.
51. Moran, J., & Zaki, J. (2013). Functional neuroimaging and psychology: What have you done for me lately? Journal of Cognitive Neuroscience, 25, 834-842.
52. Neisser, U. (1980). On “social knowing”. Personality and Social Psychology Bulletin, 6, 601-605.
53. Nichols, T., Brett, M., Andersson, J., Wager, T., & Poline, J. B. (2005). Valid conjunction inference with the minimum statistic. Neuroimage, 25, 653-660.
54. Nook, E. C., Lindquist, K. A., & Zaki, J. (2015). A new look at emotion perception: Concepts speed and shape facial emotion recognition. Emotion, 15, 569-578.
55. Nook, E. C., & Zaki, J. (2015). Social norms shift behavioral and neural responses to foods. Journal of Cognitive Neuroscience, 27, 1412-1426.
56. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452-454.
57. Olsson, A., & Phelps, E. A. (2007). Social learning of fear. Nature Neuroscience, 10, 1095-1102.
58. Ostrom, T. (1984). The sovereignty of social cognition. In Handbook of social cognition. Princeton, NJ: Lawrence Erlbaum.
59. Pickett, C. L., Gardner, W. L., & Knowles, M. (2004). Getting a cue: The need to belong and enhanced sensitivity to social cues. Personality & Social Psychology Bulletin, 30, 1095-1107.
60. Poldrack, R. A., & Yarkoni, T. (2015). From brain maps to cognitive ontologies: Informatics and the search for mental structure. Annual Review of Psychology, 67, 587-612.
61. Rangel, A., & Hare, T. (2010). Neural computations associated with goal-directed choice. Current Opinion in Neurobiology, 20, 262-270.
62. Rescorla, R., & Wagner, A. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. Black, & W. Prokasy (Eds.), Classical Conditioning II: Current Research and Theory (pp. 65-99). New York: Appleton-Century-Crofts.
63. Rushworth, M. F., & Behrens, T. E. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11, 389-397.
64. Russell, J. A., Bachorowski, J. A., & Fernandez-Dols, J. M. (2003). Facial and vocal expressions of emotion. Annual Review of Psychology, 54, 329-349.
65. Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., et al. (2012). Toward a second-person neuroscience. Behavioural Brain Research, 36, 393-414.
66. Schiller, D., Freeman, J. B., Mitchell, J. P., Uleman, J. S., & Phelps, E. A. (2009). A neural mechanism of first impressions. Nature Neuroscience, 12, 508-514.
67. Schönberg, T., Daw, N. D., Joel, D., & O’Doherty, J. P. (2007). Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. Journal of Neuroscience, 27, 12860-12867.
68. Slotnick, S. D., Moo, L. R., Segal, J. B., & Hart, J., Jr. (2003). Distinct prefrontal cortex activity associated with item memory and source memory for visual shapes. Brain Research. Cognitive Brain Research, 17, 75-82.
69. Spreckelmeyer, K. N., Krach, S., Kohls, G., Rademacher, L., Irmak, A., Konrad, K., et al. (2009). Anticipation of monetary and social reward differently activates mesolimbic brain structures in men and women. Social Cognitive and Affective Neuroscience, 4, 158-165.
70. Stinson, L., & Ickes, W. (1992). Empathic accuracy in the interactions of male friends versus male strangers. Journal of Personality and Social Psychology, 62, 787-797.
71. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
72. Swann, W. B., Jr. (1984). Quest for accuracy in person perception: A matter of pragmatics. Psychological Review, 91, 457-477.
73. Tricomi, E., Delgado, M. R., McCandliss, B. D., McClelland, J. L., & Fiez, J. A. (2006). Performance feedback drives caudate activation in a phonological learning task. Journal of Cognitive Neuroscience, 18, 1029-1043.
74. Tricomi, E., & Fiez, J. A. (2012). Information content and reward processing in the human striatum during performance of a declarative memory task. Cognitive, Affective & Behavioral Neuroscience, 12, 361-372.
75. Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004). Listening to speech activates motor areas involved in speech production. Nature Neuroscience, 7, 701-702.
76. Wimmer, G. E., Daw, N. D., & Shohamy, D. (2012). Generalization of value in reinforcement learning by humans. European Journal of Neuroscience, 35, 1092-1104.
77. Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8, 665-670.
78. Zaki, J. (2013). Cue integration: A common framework for physical perception and social cognition. Perspectives on Psychological Science, 8, 296-312.
79. Zaki, J. (2014). Empathy: A motivated account. Psychological Bulletin, 140, 1608-1647.
80. Zaki, J., Hennigan, K., Weber, J., & Ochsner, K. N. (2010). Social cognitive conflict resolution: Contributions of domain-general and domain-specific neural systems. Journal of Neuroscience, 30, 8481-8488.
81. Zaki, J., Lopez, G., & Mitchell, J. (2014). Activity in ventromedial prefrontal cortex covaries with revealed social preferences: Evidence for person-invariant value. Social Cognitive and Affective Neuroscience, 9, 464-469.
82. Zaki, J., & Mitchell, J. (2011). Equitable decision making is associated with neural markers of subjective value. Proceedings of the National Academy of Sciences, U.S.A., 108, 19761-19766.
83. Zaki, J., & Ochsner, K. (2009). The need for a cognitive neuroscience of naturalistic social cognition. Annals of the New York Academy of Sciences, 1167, 16-30.
84. Zaki, J., & Ochsner, K. (2011). You, me, and my brain: Self and other representations in social cognitive neuroscience. In A. Todorov, S. T. Fiske, & D. Prentice (Eds.), Social neuroscience: Toward understanding the underpinnings of the social mind (Vol. 26-48, pp. 25-52) New York: Oxford University Press.