Goal-directed EEG activity evoked by discriminative stimuli in reinforcement learning

Psychophysiology

Volumn 52, Issue 2, 2015, Pages 238-248

  Luque, David ^a   Morís, Joaquín ^a   Rushby, Jacqueline A ^b   Le Pelley, Mike E ^b  

^a Institute of Biomedical Research in Malaga and BIONAND Nanomedicine Platform   (Spain)

^b UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Cognitive neuroscience; Event related potentials; Feedback correct related positivity; Goal directed action; P2; P3; P3b; Reinforcement learning

Indexed keywords

ADOLESCENT; ADULT; BRAIN; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; EVOKED RESPONSE; FEMALE; HUMAN; LEARNING; MALE; MOTIVATION; PERCEPTIVE DISCRIMINATION; PHOTOSTIMULATION; PHYSIOLOGY; REINFORCEMENT; YOUNG ADULT;

ADOLESCENT; ADULT; BRAIN; BRAIN MAPPING; DISCRIMINATION (PSYCHOLOGY); ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS; FEMALE; GOALS; HUMANS; LEARNING; MALE; MOTIVATION; PHOTIC STIMULATION; REINFORCEMENT (PSYCHOLOGY); YOUNG ADULT;

EID: 84921539024     PISSN: 00485772     EISSN: 14698986     Source Type: Journal    
DOI: 10.1111/psyp.12302     Document Type: Article

References (57)

1. Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences of the USA, 108, 10367-10371. doi: 10.1073/pnas.1104047108
2. Baker, T. E., & Holroyd, C. B. (2009). Which way do I go? Neural activation in response to feedback and spatial processing in a virtual T-maze. Cerebral Cortex, 19, 1708-1722. doi: 10.1093/cercor/bhn223
3. Belouchrani, A., Abed-Meraim, K., Cardoso, J. F., & Moulines, E. (1997). A blind source separation technique using second-order statistics. IEEE Transactions on Signal Processing, 45, 434-444. doi: 10.1109/78.554307
4. Brown, C. A., Seymour, B., Boyle, Y., El-Deredy, W., & Jones A. K. P. (2008). Modulation of pain ratings by expectation and uncertainty: Behavioral characteristics and anticipatory neural correlates. Pain, 135, 240-250. doi: 10.1016/j.pain.2007.05.022
5. Brunia, C. H., Hackley, S. A., van Boxtel, G. J., Kotani, Y., & Ohgami, Y. (2011) Waiting to perceive: Reward or punishment? Clinical Neurophysiology, 122, 858-868. doi: 10.1016/j.clinph.2010.12.039
6. Carretié, L., Mercado, F., Tapia, M., & Hinojosa, J. A. (2001). Emotion, attention, and the "negativity bias, " studied through event-related potentials. International Journal of Psychophysiology, 41, 75-85. doi: 10.1016/S0167-8760(00)00195-1
7. Chase, H. W., Swainson, R., Durham, L., Benham, L., & Cools, R. (2011). Feedback-related negativity codes prediction error but not behavioral adjustment during probabilistic reversal learning. Journal of Cognitive Neuroscience, 23, 936-946. doi: 10.1162/jocn.2010.21456
8. Cohen, Y. E., & Andersen, R. A. (2002). A common reference frame for movement plans in the posterior parietal cortex. Nature Reviews Neuroscience, 3, 553-562. doi: 10.1038/nrn873
9. Courchesne, E., Hillyard, S. A., & Galambos, R. (1975). Stimulus novelty, task relevance and the visual evoked potential in man. Electroencephalography and Clinical Neurophysiology, 39, 131-143. doi: 10.1016/0013-4694(75)90003-6
10. Delorme, A., & Makeig, S. (2004), EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134, 9-21. doi: 10.1016/j.jneumeth.2003.10.009
11. Delplanque, S., Lavoie, M. E., Hot, P., Silvert, L., & Sequeira, H. (2004). Modulation of cognitive processing by emotional valence studied through event-related potentials in humans. Neuroscience Letters, 356, 1-4. doi: 10.1016/j.neulet.2003.10.014
12. de Wit, S., & Dickinson, A. (2009). Associative theories of goal-directed behaviour: A case for animal-human translational models. Psychological Research, 73, 463-476. doi: 10.1007/s00426-009-0230-6
13. de Wit, S., Niry, D., Wariyar, R., Aitken, M. R. F., & Dickinson, A. (2007). Stimulus-outcome interactions during instrumental discrimination learning by rats and humans. Journal of Experimental Psychology: Animal Behavior Processes, 33, 1-11
14. Dien, J. (2012). Applying principal components analysis to event-related potentials: A tutorial. Developmental Neuropsychology, 37, 497-517.
15. Donkers, F. C., Nieuwenhuis, S., & van Boxtel, G. J. (2005). Mediofrontal negativities in the absence of responding. Cognitive Brain Research, 25, 777-787. doi: 10.1016/j.cogbrainres.2005.09.007
16. Dunning, J. P., & Hajcak, G. (2007). Error-related negativities elicited by monetary loss and cues that predict loss. NeuroReport, 18, 1875-1878. doi: 10.1097/WNR.0b013e3282f0d50b
17. Evans, K. M., & Federmeier, K. D. (2007). The memory that's right and the memory that's left: Event-related potentials reveal hemispheric asymmetries in the encoding and retention of verbal information. Neuropsychologia, 45, 1777-1790. doi: 10.1016/j.neuropsychologia.2006.12.014
18. Fuentemilla, L., Cucurell, D., Marco-Pallarés, J., Guitart-Masip, M., Morís, J., & Rodríguez-Fornells, A. (2013). Electrophysiological correlates of anticipating improbable but desired events. NeuroImage, 78, 135-144. doi: 10.1016/j.neuroimage.2013.03.062
19. Frank, M. J. (2008). Schizophrenia: A computational reinforcement learning perspective. Schizophrenia Bulletin, 34, 1008-1011. doi: 10.1093/schbul/sbn123
20. Hillyard, S. A., & Anllo-Vento, L. (1998). Event-related brain potentials in the study of visual selective attention. Proceedings of the National Academy of Sciences of the USA, 95, 781-787. doi: 10.1073/pnas.95.3.781
21. Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709. doi: 10.1037/0033-295X.109.4.679
22. Holroyd, C. B., Krigolson, O. E., & Lee, S. (2011). Reward positivity elicited by predictive cues. NeuroReport, 22, 249-252. doi: 10.1097/WNR.0b013e328345441d
23. Holroyd, C. B., Pakzad-Vaezi, K., & Krigolson, O. E. (2008). The feedback correct-related positivity: Sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology, 45, 688-697. doi: 10.1111/j.1469-8986.2008.00668.x
24. Hyman, S. E. (2005). Addiction: A disease of learning and memory. American Journal of Psychiatry, 162, 1414-1422. doi: 10.1176/appi.ajp.162.8.1414
25. Hughes, G., Mathan, S., & Yeung, N. (2012). EEG indices of reward motivation and target detectability in a rapid visual detection task. NeuroImage, 64, 590-600. doi: 10.1016/j.neuroimage.2012.09.003
26. Johnson, R. (1986). A triarchic model of P300 amplitude. Psychophysiology, 23, 367-384. doi: 10.1111/j.1469-8986.1986.tb00649.x
27. Kennerley, S. W., Walton, M. E., Behrens, T. E., Buckley, M. J., & Rushworth, M. F. (2006). Optimal decision making and the anterior cingulate cortex. Nature Neuroscience, 9, 940-947. doi: 10.1038/nn1724
28. Kiss, M., Driver, J., & Eimer, M. (2009). Reward priority of visual target singletons modulates event-related potential signatures of attentional selection. Psychological Science, 20, 245-251. doi: 10.1111/j.1467-9280.2009.02281.x
29. Kotani, Y., Hiraku, S., Suda, K., & Aihara, Y. (2001). Effect of positive and negative emotion on stimulus-preceding negativity prior to feedback stimuli. Psychophysiology, 38, 873-878. doi: 10.1111/1469-8986.3860873
30. Le Pelley, M. E. (2010). Attention and human associative learning. In C. J. Mitchell & M. E. Le Pelley (Eds.), Attention and associative learning: From brain to behaviour (pp. 187-215). Oxford, UK: Oxford University Press.
31. Le Pelley, M. E., Beesley, T., & Griffiths, O. (2011). Overt attention and predictiveness in human contingency learning. Journal of Experimental Psychology: Animal Behavior Processes, 37, 220-229. doi: 10.1037/a0021384
32. Le Pelley, M. E., & McLaren, I. P. L. (2003). Learned associability and associative change in human causal learning. Quarterly Journal of Experimental Psychology: Section B, 56, 68-79. doi: 10.1080/02724990244000179
33. Le Pelley, M. E., Vadillo, M. A., & Luque, D. (2013). Learned predictiveness influences rapid attentional capture: Evidence from the dot probe task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39, 1888-1900. doi: 10.1037/a0033700.
34. Liao, Y., Gramann, K., Feng, W., Deak, G., & Li, H. (2011). This ought to be good: Brain activity accompanying positive and negative expectations and outcomes. Psychophysiology, 48, 1412-1419. doi: 10.1111/j.1469-8986.2011.01205.x
35. Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276-298. doi: 10.1037/h0076778
36. Marco-Pallares, J., Cucurell, D., Münte, T. F., Strien, N., & Rodriguez-Fornells, A. (2011). On the number of trials needed for a stable feedback-related negativity. Psychophysiology, 48, 852-860. doi: 10.1111/j.1469-8986.2010.01152.x
37. Masaki, H., Yamazaki, K., & Hackley, S. H. (2010). Stimulus-preceding negativity is modulated by action-outcome contingency. NeuroReport, 21, 277-281. doi: 10.1097/WNR.0b013e3283360bc3
38. McClure, S. M., York, M. K., & Montague, P. R. (2004). The neural substrates of reward processing in humans: The modern role of FMRI. Neuroscientist, 10, 260-268. doi: 10.1177/1073858404263526
39. Morís, J., Luque, D., & Rodríguez-Fornells, A. (2013). Learning-induced modulations of the stimulus-preceding negativity. Psychophysiology, 50, 931-939. doi: 10.1111/psyp.12073
40. Morris, R., Griffiths, O., Le Pelley, M. E., & Weickert, T. W. (2013). Attention to irrelevant cues is related to positive symptoms in schizophrenia. Schizophrenia Bulletin, 39, 575-582. doi: 10.1093/schbul/sbr192
41. Morris, S. E., Holroyd, C. B., Mann-Wrobel, M. C., & Gold, J. M. (2011). Dissociation of response and feedback negativity in schizophrenia: Electrophysiological and computational evidence for a deficit in the representation of value. Frontiers in Human Neuroscience, 5, 123. doi: 10.3389/fnhum.2011.00123
42. Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective picture processing: An integrative review of ERP findings. Biological Psychology, 77, 247-265. doi: 10.1016/j.biopsycho.2007.11.006
43. Polich, J. (2007). Updating P300: An integrative theory of P3a and P3b. Clinical Neurophysiology, 118, 2128-2148. doi: 10.1016/j.clinph.2007.04.019
44. Riesel, A., Weinberg, A., Endrass, T., Kathmann, N., & Hajcak, G. (2012). Punishment has a lasting impact on error-related brain activity. Psychophysiology, 49, 239-247. doi: 10.1111/j.1469-8986.2011.01298.x
45. Rushby, J. A., Barry, R. J., & Doherty, R. J. (2005). Separation of the components of the late positive complex in an ERP dishabituation paradigm. Clinical Neurophysiology, 116, 2363-2380. doi: 10.1016/j.clinph.2005.06.008
46. Rustemeier, M., Schwabe, L., & Bellebaum, C. (2013). On the relationship between learning strategy and feedback processing in the weather prediction task-Evidence from event-related potentials. Neuropsychologia, 51, 695-703. doi: 10.1016/j.neuropsychologia.2013.01.009
47. Santesso, D. L., Steele, K. T., Bogdan, R., Holmes, A. J., Deveney, C. M., Meites, T. M., & Pizzagalli, D. A. (2008). Enhanced negative feedback responses in remitted depression. NeuroReport, 19, 1045-1048. doi: 10.1097/WNR.0b013e3283036e73
48. Schevernels, H., Krebs, R. M., Santens, P., Woldorff, M. G., & Boehler, C. N. (2014) Task preparation processes related to reward prediction precede those related to task-difficulty expectation. NeuroImage, 84, 639-647. doi: 10.1016/j.neuroimage.2013.09.039
49. Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011). The integration of negative affect, pain and cognitive control in the cingulate cortex. Nature Reviews Neuroscience, 12, 154-167. doi: 10.1038/nrn2994
50. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
51. Treisman, A. (1996). Selection for perception for selection for action: A reply to Van der Heijden. Visual Cognition, 3, 353-357
52. van Boxtel, G. J., & Böcker, K. B. E. (2004). Cortical measures of anticipation. Journal of Psychophysiology, 18, 61-76. doi: 10.1027/0269-8803.18.2-3.61
53. Walsh, M. M., & Anderson, J. R. (2011). Modulation of the feedback-related negativity by instruction and experience. Proceedings of the National Academy of Sciences of the USA, 108, 19048-19053. doi: 10.1073/pnas.1117189108
54. Walsh, M. M., & Anderson, J. R. (2012). Learning from experience: Event-related potential correlates of reward processing, neural adaptation, and behavioral choice. Neuroscience & Biobehavioral Reviews, 36, 1870-1884. doi: 10.1016/j.neubiorev.2012.05.008
55. Waltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. M. (2007). Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biological Psychiatry, 62, 756-764. doi: 10.1016/j.biopsych.2006.09.042
56. Wills, A. J., Lavric, A., Croft, G. S., & Hodgson, T. L. (2007). Predictive learning, prediction errors, and attention: Evidence from event-related potentials and eye tracking. Journal of Cognitive Neuroscience, 19, 843-854. doi: 10.1162/jocn.2007.19.5.843
57. Wills, A. J., Lavric, A., Hemmings, Y., & Surrey, E. (2014). Attention, predictive learning, and the inverse base-rate effect: Evidence from event-related potentials. NeuroImage, 87, 61-71. doi: 10.1016/j.neuroimage.2013.10.060