Uncertainty and learning

IETE Journal of Research

Volumn 49, Issue 2-3, 2003, Pages 171-181

  Dayan, Peter ^a   Jyu, Angela ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ACETYLCHOLINE; ATTENTION; BEHAVIORAL LEARNING; NEUROMODULATION; NOREPINEPHRINE; UNCERTAINTY;

KALMAN FILTERING; MATHEMATICAL MODELS; NEUROPHYSIOLOGY; PHYSIOLOGICAL MODELS; PSYCHOPHYSIOLOGY; STATISTICAL METHODS;

HUMAN ENGINEERING;

EID: 0042467409     PISSN: 03772063     EISSN: 0974780X     Source Type: Journal    
DOI: 10.1080/03772063.2003.11416335     Document Type: Article

References (57)

1. A, Dickinson, Contemporary Animal Learning Theory, Cambridge, Cambridge University Press, 1980.
2. N J, Mackintosh, Conditioning and Associative Learning, Oxford:Oxford University Press, 1983.
3. M A, Gluck, E S, Reifsnider & R F, Thompson, Adaptive signal processing and the cerebellum:Models of classical conditioning and VOR adaptation. In MA, Gluck & DE, Rumelhart, editors, Neuroscience and Connectionist Theory, Developments in Connectionist Theory, 131–185, Hillsdale, NJ:Erlbaum, 1990.
4. W, Schultz, Predictive reward signal of dopamine neurons, Journal of Neurophysiology, vol 80, pp 1–27, 1998.
5. D, Marr, Vision, New York, NY:WH Freeman, 1982.
6. J R, Anderson, The Adaptive Character of Thought, Hillsdale, NJ:Erlbaum, 1990.
7. L J, Kamin, Predictability, surprise, attention, and conditioning. In:Punishment and Aversive Behavior, edited by Campbell BA, and Church RM, New York:Appleton-Century-Crofts, pp 242–259, 1969.
8. R A, Rescorla & A R, Wagner, A theory of Pavlovian conditioning:The effectiveness of reinforcement and non-reinforcement, In AH, Black & WF, Prokasy, editors, Classical Conditioning II:Current Research Theory, pp 64–69, New York, Appleton-Century-Crofts, 1972.
9. R S, Sutton & A G, Barto, Toward a modern theory of adaptive networks:Expectation and prediction, Psychological Review, vol 88, pp 135–170, 1981.
10. B, Widrow & M E, Hoff, Adaptive switching circuits, WESCON Convention Report, vol 4, pp 96–104, 1960.
11. RS, Sutton & A G, Barto, Time-derivative models of Pavlovian conditioning. In:M, Gabriel & JW, Moore, editors, Learning and Computational Neuroscience, pp 497–537. Cambridge, MA, MIT Press, 1990.
12. P R, Montague, P, Dayan & T J, Sejnowski, A framework for mesencephalic dopamine systems based on predictive Hebbian learning, Journal of Neuroscience, vol 16, pp 1936–1947, 1996.
13. W, Schultz, P, Dayan, & PR, Montague, A neural substrate of prediction and reward. Science, vol 275, pp 1593–1599, 1997
14. W, Schultz & A, Dickinson, Neuronal coding of prediction errors, Annual Review of Neuroscience, vol 23, pp 473–500, 2000.
15. J M, Pearce & G, Hall, A model for Pavlovian learning:Variation in the effectiveness of conditioned but not unconditioned stimuli, Psychological Review, vol 87, pp 532–552, 1980.
16. PC, Holland, Excitation and inhibition in unblocking. Journal of Experimental Psychology:Animal Behavior Processes, vol 14, pp 261–279, 1988.
17. P C, Holland, Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning:Implications for behavior theory, Animal Learning & Behavior, vol 25, pp 373–399, 1997.
18. P C, Holland & M, Gallagher, Amygdala circuitry in attentional and representational processes, Trends in Cognitive Sciences, vol 3, pp 65–73, 1999.
19. Q, Gu, Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity, Neuroscience, vol 111, pp 815–835, 2002.
20. K, Doya, Metalearning and neuromodulation, Neural Networks, vol 15, pp 495–506, 2002.
21. M E, Hasselmo, Neuromodulation and cortical function:modeling the physiological basis of behaviour, Behavioural Brain Research, vol 67, pp 1–27. 1995.
22. P, Dayan & A J, Yu, A, Ch, Uncertainty, and cortical inference. In TG, Dietterich, S, Becker & Z, Ghahramani, editors, Advances in Neural Information Processing Systems, 14, 2002.
23. A J, Yu & P, Dayan, Acetylcholine in cortical inference. Neural Networks, vol 15, pp 719–732, 2002.
24. J, Rajkowski, P, Kubiak & G, Aston-Jones, Locus Coeruleus activity in monkey:phasic and tonic changes are associated with altered vigilance, Brain Res Bull, vol 35, pp 607–616, 1994.
25. G, Aston-Jones, J, Rajkowski & J, Cohen, Role of locus coeruleus in attention and behavioral flexibility. Biol Psychiatry, vol 46, pp 1309–1320, 1999.
26. S J, Sara, Learning by neurons:role of attention, reinforcement and behavior, Comptes Rendus de l'Academie des Sciences Serie III-Sciences de la Vie-Life Sciences, vol 321, pp 193–198, 1998.
27. B D, Anderson & J B, Moore, Optimal Filtering, Prentice-Hall, Englewood Cliffs, NJ, 1979.
28. P, Dayan, S, Kakade, & P R, Montague, Learning and selective attention, Nature Neuroscience, vol 3, pp 1218–1223, 2000.
29. S, Kakade & P, Dayan, Acquisition and extinction in autoshaping, Psychological Review, vol 109, pp 533–544, 2002.
30. R S, Sutton, Gain adaptation beats least squares? In Proceedings of the 7th Yale Workshop on Adaptive and learning Systems, 1992.
31. N D, Daw, S, Kakade & P, Dayan, Opponent interactions between serotonin and dopamine. Neural Networks, vol 15, pp 603–616, 2002.
32. D R, Shanks, Forward and backward blocking in human contingency judgement. Quarterly Journal of Experimental Psychology:Comparative & Physiological Psychology, vol 37, pp 1–21, 1985.
33. R R, Miller & H, Matute, Biological significance in forward and backward blocking:Resolution of a discrepancy between animal conditioning and human causal judgement, Journal of Experimental Psychology:General, vol 125, pp 370–386, 1996.
34. P, Dayan & S, Kakade, Explaining away in weight space. In TK, Leen, TG, Dictterich & V, Tresp, eds, Advances in Neural Information Processing Systems, 13, 2001, pp 451–457.
35. A, Yu & P, Dayan, Expected and unexpected uncertainty:ACh and NE in the neocortex, Advances in Neural Information Processing Systems, vol 15, in press.
36. M, Gluck, P, Glauthicr & R S, Sutton, Adaptation of cue-specific learning rates in network models of human category learning, Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society, pp 540–545, Erlbaum, 1992.
37. G E, Hinton & T J, Sejnowski, Unsupervised Learning:Foundations of Neural Computation, MIT Press, Cambridge, MA, 1999.
38. D J, Bucci, P C, Holland & M, Gallagher, Removal of cholinergic input to rat posterior parietal cortex disrupts incremental processing of conditioned stimuli, Journal of Neuroscience, vol 18, no 19, pp 8038–8046, October 1998.
39. J-S, Han, P C, Holland & M, Gallagher, Disconnection of the amygdala central nucleus and substantia innominata/nucleus basalis disrupts increments in conditioned stimulus processing in rats, Behavioral Neuroscience, vol 113, pp 143–151, 1999.
40. D D, Rasmusson, The role of acetylcholine in cortical synaptic plasticity, Behavioral Brain Research, vol 115, pp 205–218, November 2000.
41. A, Dickinson & B M, Balleine, The role of learning in motivation. In CR, Gallistel (Editor), learning, Motivation & Emotion, Volume 3 of Steven's Handbook of Experimental Psychology, Third Edition, New York, John Wiley & Sons, 2002.
42. P, Dayan & B M, Balleine, BW, Reward, Motivation and reinforcement learning, Neuron, in press.
43. C R, Gallistel & J, Gibbon, Time, Rate, and Conditioning, Psychological Review, vol 107, pp 289–344, 2000.
44. P, Dayan & T, Long, Statistical models of conditioning. In MI, Jordan, M, Kearns & SA, Solla, editors, Advances in Neural Information Processing Systems. Cambridge, MA, MIT Press, pp 117–123, 1998.
45. J K, Kruschke, Toward a unified model of attention in associative learning, Journal of Mathematical Psychology, vol 45, pp 812–863, 2001.
46. M, Gluck & C, Myers, Gateway to Memory:An Introduction to Neural Network Modeling of the Hippocampus and Learning, Cambridge, MA:MIT Press, 2000.
47. Z, Gil, B M, Conners & Y, Amitai, Differential regulation of neocortical synapses by neuromodulators and activity, Neuron, vol 19, pp 679–86, 1997.
48. F, Kimura, M, Fukuada & T, Tsumoto, Acetylcholine suppresses the spread of excitation in the visual cortex revealed by optical recording:possible differential effect depending on the source of input, European Journal of Neuroscience, vol 11, pp 3597–3609, 1999.
49. C Y, Hsieh, S J, Cruikshank & R, Metherate, Differential modulation of auditory thalamocortical and intracortical synaptic transmission by cholinergic agonist. Brain Research, vol 880, pp 51–64, 2000.
50. M E, Hasselmo & M, Cekic, Suppression of synaptic transmission may allow combination of associative feedback and self-organizing feedforward connections in the neocortex, Behav Brain Res. vol 79, pp 153–161, 1996.
51. S J, Sara, A, Vankov & A, Herve, Locus coeruleus-evoked responses in behaving rats:a clue to the role of noradrenaline in memory, Brain Res Bull, vol 35, pp 457–465, 1994.
52. A, Vankov, A, Herve-Minvielle & S J, Sara, Response to novelty and its rapid habituation in locus coeruleus neurons of freely exploring rat, Eur Journal Neurosci, vol 109, pp 903–911, 1995.
53. M E, Hasselmo, C, Linster, M, Patil, D, Ma & M, Cekic, Noradrenergic suppression of synaptic transmission may influence cortical signal-to-noise ratio, Journal Neurophysiol, vol 77, pp 3326–3339, 1997.
54. M, Kobayashi, K, Imamura, T, Suai, N, Onoda, M, Yamamoto, S, Komai & Y, Watanabe, Selective suppression of horizontal propagation in rat visual cortex by norepinephrine, European Journal of Neuroscience, vol 12, pp 264–272, 2000.
55. G A, Carpenter & S, Grossberg, editors, Pattern Recognition by Self-Organizing Neural Networks. Cambridge, MA, MIT Press, 1991.
56. T W, Robbins, Cortical noradrenaline, attention and arousal, Psychological Medicine, vol 14, pp 13–21, 1984.
57. J, McGaughy & M, Sarter, Lack of effects of lesions of the dorsal noradrenergic bundle on behavioral vigilance, Behav Neurosci, vol 111, no 3, pp 646–652, 1997.