A learning rule that explains how rewards teach attention

Visual Cognition

Volumn 23, Issue 1-2, 2015, Pages 179-205

  Rombouts, Jaldert O ^a   Bohte, Sander M ^a   Martinez Trujillo, Julio ^b   Roelfsema, Pieter R ^c,d,e  

^a CWI   (Netherlands)

^b MCGILL UNIVERSITY   (Canada)

^c NETHERLANDS INSTITUTE FOR NEUROSCIENCE   (Netherlands)

^d VU UNIVERSITY AMSTERDAM   (Netherlands)

^e ACADEMIC MEDICAL CENTER   (Netherlands)

Author keywords

Attention; Neural networks; Neuronal plasticity; Reinforcement learning; Top down

Indexed keywords

ANIMAL EXPERIMENT; ARTICLE; ATTENTION; COLOR; DORSOLATERAL PREFRONTAL CORTEX; FEEDBACK SYSTEM; HAPLORHINI; LEARNING; MEMORY; MOTION; NERVE CELL; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NONHUMAN; PRIORITY JOURNAL; REINFORCEMENT; REWARD; TRAINING;

EID: 84928938586     PISSN: 13506285     EISSN: 14640716     Source Type: Journal    
DOI: 10.1080/13506285.2015.1010462     Document Type: Article

References (47)

1. M.Ahissar., & S.Hochstein, (1993). Attentional control of early perceptual learning. Proceedings of the National Academy of Sciences of the United States of America, 90, 5718–5722. doi:10.1073/pnas.90.12.5718
2. B.A.Anderson., P.A.Laurent., & S.Yantis, (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences of the United States of America, 108, 10367–10371. doi:10.1073/pnas.1104047108
3. F.G.Ashby., J.M.Ennis., & B.J.Spiering, (2007). A neurobiological theory of automaticity in perceptual categorization. Psychological Review, 114, 632–656. doi:10.1037/0033-295X.114.3.632
4. E.S.Bromberg-Martin., M.Matsumoto., & O.Hikosaka, (2010). Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron, 68, 815–834. doi:10.1016/j.neuron.2010.11.022
5. C.Bundesen., T.Habekost., & S.Kyllingsbæk, (2005). A Neural theory of visual attention: Bridging cognition and neurophysiology. Psychological Review, 112, 291–328. doi:10.1037/0033-295X.112.2.291
6. L.Chelazzi., A.Perlato., E.Santandrea., & D.C.Libera, (2013). Rewards teach visual selective attention. Vision Research, 85, 58–72. doi:10.1016/j.visres.2012.12.005
7. M.Corbetta., & G.L.Shulman, (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201–215. doi:10.1038/nrn755
8. P.Dayan., & B.W.Balleine, (2002). Reward, motivation, and reinforcement learning. Neuron, 36, 285–298. doi:10.1016/S0896-6273(02)00963-7
9. P.Dayan., & A.J.Yu, (2002). ACh, uncertainty, and cortical inference (Vol. 1, p. 189). Paper Presented at the Advances in Neural Information Processing Systems, MIT Press.
10. S.Dehaene., G.Dehaene-Lambertz., & L.Cohen, (1998). Abstract representations of numbers in the animal and human brain. Trends in Neurosciences, 21, 355–361. doi:10.1016/S0166-2236(98)01263-6
11. R.Desimone., & J.Duncan, (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, 193–222. doi:10.1146/annurev.ne.18.030195.001205
12. H.Deubel., & W.X.Schneider, (1996). Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36, 1827–1837. doi:10.1016/0042-6989(95)00294-4
13. J.Duncan, (2010). The multiple-demand (MD) system of the primate brain: Mental programs for intelligent behaviour. Trends in Cognitive Sciences, 14, 172–179. doi:10.1016/j.tics.2010.01.004
14. D.J.Felleman., & D.C.Van Essen, (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1), 1–47. doi:10.1093/cercor/1.1.1
15. S.Funahashi., C.J.Bruce., & P.S.Goldman-Rakic, (1989). Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. Journal of Neurophysiology, 61, 331–349.
16. J.W.Gnadt., & R.A.Andersen, (1988). Memory related motor planning activity in posterior parietal cortex of macaque. Experimental Brain Research, 70, 216–220.
17. J.Gottlieb., & P.Balan, (2010). Attention as a decision in information space. Trends in Cognitive Sciences, 14, 240–248. doi:10.1016/j.tics.2010.03.001
18. C.Hickey., L.Chelazzi., & J.Theeuwes, (2010). Reward changes salience in human vision via the anterior cingulate. Journal of Neuroscience, 30, 11096–11103. doi:10.1523/JNEUROSCI.1026-10.2010
19. Y.Jiang., & M.M.Chun, (2001). Selective attention modulates implicit learning. The Quarterly Journal of Experimental Psychology A, 54, 1105–1124. doi:10.1080/02724980042000516
20. M.P.Kilgard., & M.M.Merzenich, (1998). Cortical map reorganization enabled by Nucleus Basalis Activity. Science, 279, 1714–1718. doi:10.1126/science.279.5357.1714
21. E.Kowler., E.Anderson., B.Dosher., & E.Blaser, (1995). The role of attention in the programming of saccades. Vision Research, 35, 1897–1916. doi:10.1016/0042-6989(94)00279-U
22. K.A.Krueger., & P.Dayan, (2009). Flexible shaping: How learning in small steps helps. Cognition, 110, 380–394. doi:10.1016/j.cognition.2008.11.014
23. T.Lennert., & J.Martinez-Trujillo, (2011). Strength of response suppression to distracter stimuli determines attentional-filtering performance in primate prefrontal neurons. Neuron, 70(1), 141–152. doi:10.1016/j.neuron.2011.02.041
24. D.C.Libera., & L.Chelazzi, (2009). Learning to attend and to ignore is a matter of gains and losses. Psychological Science, 20, 778–784. doi:10.1111/j.1467-9280.2009.02360.x
25. Z.Liu., J.Zhou., Y.Li., F.Hu., Y.Lu., M.Ma., M.Luo, (2014). Dorsal raphe neurons signal reward through 5-HT and glutamate. Neuron, 81, 1360–1374. doi:10.1016/j.neuron.2014.02.010
26. K.Louie., L.E.Grattan., & P.W.Glimcher, (2011). Reward value-based gain control: Divisive normalization in parietal cortex. Journal of Neuroscience, 31, 10627–10639. doi:10.1523/JNEUROSCI.1237-11.2011
27. T.Mao., D.Kusefoglu., B.M.Hooks., D.Huber., L.Petreanu., & K.Svoboda, (2011). Long-Range neuronal circuits underlying the interaction between sensory and motor cortex. Neuron, 72(1), 111–123. doi:10.1016/j.neuron.2011.07.029
28. J.H.R.Maunsell, (2004). Neuronal representations of cognitive state: Reward or attention? Trends in Cognitive Sciences, 8, 261–265. doi:10.1016/j.tics.2004.04.003
29. E.K.Miller., & J.D.Cohen, (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202. doi:10.1146/annurev.neuro.24.1.167
30. A.Pastor-Bernier., & P.Cisek, (2011). Neural correlates of biased competition in premotor cortex. Journal of Neuroscience, 31, 7083–7088. doi:10.1523/JNEUROSCI.5681-10.2011
31. C.J.Peck., D.C.Jangraw., M.Suzuki., R.Efem., & J.Gottlieb, (2009). Reward modulates attention independently of action value in posterior parietal cortex. Journal of Neuroscience, 29, 11182–11191. doi:10.1523/JNEUROSCI.1929-09.2009
32. M.L.Platt., & P.W.Glimcher, (1999). Neural correlates of decision variables in parietal cortex. Nature, 400, 233–238. doi:10.1038/22268
33. J.E.Raymond., & J.L.O’Brien, (2009). Selective visual attention and motivation the consequences of value learning in an attentional blink task. Psychological Science, 20, 981–988. doi:10.1111/j.1467-9280.2009.02391.x
34. J.H.Reynolds., & L.Chelazzi, (2004). Attentional modulation of visual processing. Annual Review of Neuroscience, 27, 611–647. doi:10.1146/annurev.neuro.26.041002.131039
35. P.R.Roelfsema, (2006). Cortical algorithms for perceptual grouping. Annual Review of Neuroscience, 29, 203–227. doi:10.1146/annurev.neuro.29.051605.112939
36. P.R.Roelfsema., & A.van Ooyen, (2005). Attention-gated reinforcement learning of internal representations for classification. Neural Computation, 17, 2176–2214. doi:10.1038/375484a0
37. P.R.Roelfsema., A.van Ooyen., & T.Watanabe, (2010). Perceptual learning rules based on reinforcers and attention. Trends in Cognitive Sciences, 14(2), 64–71. doi:10.1016/j.tics.2009.11.005
38. J.O.Rombouts., S.M.Bohte., & P.R.Roelfsema, (2012). Neurally plausible reinforcement learning of working memory tasks (Vol. 25, pp. 1880–1888). Paper Presented at the Advances in Neural Information Processing Systems, Lake Tahoe (Nevada, US).
39. J.O.Rombouts., S.M.Bohte., & P.R.Roelfsema, (in press). How attention can create synaptic tags for the learning of working memories in sequential tasks. PLoS Computational Biology.
40. G.A.Rummery., & M.Niranjan, (1994). On-line Q-learning using connectionist systems (No. CUED/F-INFENG/TR 166). Cambridge: Cambridge Univsersity.
41. W.Schultz, (2002). Getting formal with dopamine and reward. Neuron, 36, 241–263. doi:10.1016/S0896-6273(02)00967-4
42. J.T.Serences, (2008). Value-based modulations in human visual cortex. Neuron, 60, 1169–1181. doi:10.1016/j.neuron.2008.10.051
43. L.Stănişor., C.van der Togt., C.M.Pennartz., & P.R.Roelfsema, (2013). A unified selection signal for attention and reward in primary visual cortex. Proceedings of the National Academy of Sciences, 110, 9136–9141. doi:10.1073/pnas.1300117110/-/DCSupplemental
44. R.S.Sutton., & A.G.Barto, (1998). Reinforcement learning: An introduction. Cambridge, MA: A Bradford Book.
45. T.Trabasso., & G.H.Bower, (1968). Attention in learning: Theory and research. New York, NY: Krieger.
46. S.Treue., & J.H.Maunsell, (1996). Attentional modulation of visual motion processing in cortical areas MT and MST. Nature, 382, 539–541. doi:10.1038/382539a0
47. S.D.Whitehead., & D.H.Ballard, (1991). Learning to perceive and act by trial and error. Machine Learning, 7(1), 45–83.