Shifting attention using a temporal difference prediction error and high-dimensional input

Adaptive Behavior

Volumn 15, Issue 2, 2007, Pages 121-133

  Alexander, William H ^a  

^a OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

Author keywords

Adaptive behavior; Attention; Reinforcement learning; Representation; Temporal difference learning

Indexed keywords

PRIMATES;

EID: 34249728375     PISSN: 10597123     EISSN: 17412633     Source Type: Journal    
DOI: 10.1177/1059712307078663     Document Type: Conference Paper

References (26)

1. Alexander, W.H., Sporns, O. (2006). Temporal difference learning with learned attention shifts. In Proceedings of the Fifth International Conference on Development and Learning.
2. Aston-Jones, G., Rajkowski, J., Kubiak, P., Alexinsky, T. (1994). Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task. Journal of Neuroscience, 14, 4467 4480.
3. Chernyshev, B.V., Panasyuk, Y.A., Semikopnaya, I.I., Timofeeva, N.O. (2004). Activity of neurons in the basal magnocellular nucleus during performance of an operant task. Neuroscience and Behavioral Physiology, 34, 907 918.
4. Daw, N.D., Courville, A.C., Touretzky, D.S. (2006). Representation and timing in theories of the dopamine system. Neural Computation, 18, 1637 1677.
5. Daw, N.D., Kakade, S., Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15, 603 616.
6. Desimone, R., Duncan, J. (1995). Neural mechanics of selective visual attention. Annual Review of Neuroscience, 18, 193 222.
7. Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15, 495 506.
8. Gill, M.T., Sarter, M., Givens, B. (2000). Sustained visual attention performance-associated prefontal neuronal activity: Evidence for cholinergic modulation. Journal of Neuroscience, 20, 4745 4757.
9. Itti, L., Koch, C. (2000). A saliency-based search mechanism for over and covert shifts of visual attention. Vision Research, 40, 1489 1506.
10. Kruschke, J.K. (1992). ALCOVE: An exemplar-based connectionist model of category learning. Psychological Review, 99, 22 44.
11. Kruschke, J.K. (2001). Toward a unified model of attention in associative learning. Journal of Mathematical Psychology, 45, 812 863.
12. Lungarella, M., Pegors, T., Bulwinkle, D., Sporns, O. (2005). Methods for quantifying the informational structure of sensory and motor data. Neuroinformatics, 3, 243 262.
13. Mackintosh, N.J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276 298.
14. Mason, S.T., Iversen, S.D. (1979). Theories of the dorsal bundle extinction effect. Brain Research, 180, 107 137.
15. Montague, P.R., Dayan, P., Sejnowski, T.J. (1996). A framework for mesencephalic dopamine systems based on predictive hebbian learning. Journal of Neuroscience, 16, 1936 1947.
16. Pearce, J.M., Hall, G.H. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532 552.
17. Phillips, J.L., Noelle, D.C. (2004). Reinforcement learning of dimensional attention for categorization. In N. J. Mahwah, (Ed.), Proceedings of the Twenty-Sixth Annual Meeting of the Cognitive Science Society (pp. 11011106).
18. Rescorla, R.A., Wagner, A.R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. In A. H. Black W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64 99). New York: Appleton-Century-Crofts.
19. Sara, S.J., Vankov, A., Herv, A. (1994). Locus coeruleus-evoked responses in behaving rats: A clue to the role of noradrenaline in memory. Brain Research Bulletin, 35, 457 465.
20. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80, 1 27.
21. Schultz, W., Dayan, P., Montague, P.R. (1997). A neural substrate of prediction and reward. Science, 275 (5306). 1593 1599.
22. Suri, R.E. (2002). TD models of reward predictive responses in dopamine neurons. Neural Networks, 15, 523 533.
23. Suri, R.E., Schultz, W. (1998). Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Experimental Brain Research, 121, 350 354.
24. Suri, R.E., Schultz, W. (1999). A neural network learns a spatial delayed response task with a dopamine-like reinforcement signal. Neuroscience, 91, 871 890.
25. Sutton, R.S., Barto, A.G. (1990). Time derivative models of Pavlovian reinforcement. In M. Gabriel J. Moore (Eds.), Learning and computational neuroscience: Foundations of adaptive networks (pp. 539 602). Cambridge, MA: MIT Press.
26. Yu, A.J., Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46, 681 692.