Bayesian cue integration as a developmental outcome of reward mediated learning

PLoS ONE

Volumn 6, Issue 7, 2011, Pages

  Weisswange, Thomas H ^a   Rothkopf, Constantin A ^a   Rodemann, Tobias ^b   Triesch, Jochen ^a  

^a FRANKFURT INSTITUTE FOR ADVANCED STUDIES   (Germany)

^b HONDA RESEARCH INSTITUTE EUROPE GMBH   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ASSOCIATION; BAYES THEOREM; CAUSAL MODELING; COGNITIVE DEVELOPMENT; CONTROLLED STUDY; DRIVE; LEARNING ALGORITHM; LEARNING ENVIRONMENT; MATHEMATICAL MODEL; NERVE CELL NETWORK; PREDICTION; REINFORCEMENT; REWARD; SENSORIMOTOR INTEGRATION; TASK PERFORMANCE; VISUAL DISCRIMINATION; VISUAL ORIENTATION; VISUAL STIMULATION; ALGORITHM; HUMAN; LEARNING; METHODOLOGY; PATTERN RECOGNITION; PHYSIOLOGY; PSYCHOLOGICAL MODEL; PSYCHOMETRY;

ALGORITHMS; BAYES THEOREM; CUES; HUMANS; LEARNING; MODELS, PSYCHOLOGICAL; PATTERN RECOGNITION, PHYSIOLOGICAL; PSYCHOMETRICS; REINFORCEMENT (PSYCHOLOGY); REWARD;

EID: 79959991433     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0021575     Document Type: Article

References (79)

1. Ernst MO, (2004) Merging the senses into a robust percept. Trends in Cognitive Sciences 8: 162-169.
2. Kersten D, Yuille AL, (2003) Bayesian models of object perception. Current Opinion in Neurobiology 13: 150-158.
3. Kersten D, Mamassian P, Yuille AL, (2004) Object Perception as Bayesian Inference. Annual Review of Psychology 55: 271-304.
4. Knill DC, Richards W, (1996) Perception as Bayesian inference. New York, New York, USA Cambridge University Press.
5. Knill DC, Pouget A, (2004) The Bayesian brain: the role of uncertainty in neural coding and computation. Trends in Neurosciences 27: 712-719.
6. Rothkopf CA, Weisswange TH, Triesch J, (2010) Computational modeling of multisensory object perception. New York Springer pp. 21-53 chapter 3 doi:10.1007/978-1-4419-5615-6 3. URL http://www.springerlink.com/content/u45134054ktvl422/?p=3065735194a24778bf63cf25e7bcb01b&pi=2.
7. Alais D, Burr D, (2004) The Ventriloquist E_ect Results from Near-Optimal Bimodal Integration. Current Biology 14: 257-262.
8. Battaglia PW, Jacobs RA, Aslin RN, (2003) Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A, Optics, image science, and vision 20: 1391-1397.
9. Ernst MO, Banks MS, (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415: 429-433.
10. Knill DC, Saunders JA, (2003) Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Research 43: 2539-2558.
11. Jacobs RA, (1999) Optimal integration of texture and motion cues to depth. Vision Research 39: 3621-3629.
12. Sato Y, Toyoizumi T, Aihara K, (2007) Bayesian inference explains perception of unity and ventriloquism aftere_ect: identi_cation of common sources of audiovisual stimuli. Neural Computation 19: 3335-3355.
13. Körding KP, Beierholm UR, Ma WJ, Quartz S, Tenenbaum JB, et al. (2007) Causal inference in multisensory perception. PLoS ONE 2: e943.
14. Shams L, Beierholm UR, (2010) Causal inference in perception. Trends in Cognitive Sciences 14: 425-432.
15. Wozny DR, Beierholm UR, Shams L, (2010) Probability Matching as a Computational Strategy Used in Perception. PLoS Computational Biology 6.
16. Fiser J, Berkes P, Orbán G, Lengyel M, (2010) Statistically optimal perception and learning: from behavior to neural representations. Trends in Cognitive Sciences 14: 119-130.
17. Gori M, Del Viva M, Sandini G, Burr DC, (2008) Young Children Do Not Integrate Visual and Haptic Form Information. Current Biology 18: 694-698.
18. Nardini M, Jones P, Bedford R, Braddick O, (2008) Development of cue integration in human navigation. Current Biology 18: 689-693.
19. Nardini M, Bedford R, Mareschal D, (2010) Fusion of visual cues is not mandatory in children. Proceedings of the National Academy of Sciences of the United States of America 107: 17041-17046.
20. Neil PA, Chee-Ruiter C, Scheier C, Lewkowicz DJ, Shimojo S, (2006) Development of multisensory spatial integration and perception in humans. Developmental Science 9: 454-464.
21. Beck JM, Ma WJJ, Kiani R, Hanks T, Churchland AK, et al. (2008) Probabilistic population codes for Bayesian decision making. Neuron 60: 1142-1152.
22. Ma WJ, Beck JM, Latham PE, Pouget A, (2006) Bayesian inference with probabilistic population codes. Nature Neuroscience 9: 1432-1438.
23. Putzar L, Goerendt I, Lange K, Rösler F, Röder B, (2007) Early visual deprivation impairs multisensory interactions in humans. Nature Neuroscience 10: 1243-1245.
24. Schorr EA, Fox NA, van Wassenhove V, Knudsen EI, (2005) Auditory-visual fusion in speech perception in children with cochlear implants. Proceedings of the National Academy of Sciences of the United States of America 102: 18748-50.
25. Wallace MT, Stein BE, (1997) Development of multisensory neurons and multisensory integration in cat superior colliculus. Journal of Neuroscience 17: 2429-2444.
26. Wallace MT, Stein BE, (2001) Sensory and multisensory responses in the newborn monkey superior colliculus. Journal of Neuroscience 21: 8886-8894.
27. Wallace MT, Stein BE, (2007) Early experience determines how the senses will interact. Journal of Neurophysiology 97: 921-926.
28. Knudsen EI, Brainard MS, (1991) Visual instruction of the neural map of auditory space in the developing optic tectum. Science 253: 85-87.
29. Hikosaka O, Nakamura K, Nakahara H, (2006) Basal ganglia orient eyes to reward. Journal of neurophysiology 95: 567-84.
30. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O, (2002) Modulation of saccadic eye movements by predicted reward outcome. Experimental Brain Research 142: 284-91.
31. Schultz W, (2000) Multiple reward signals in the brain. Nature reviews Neuroscience 1: 199-207.
32. Schultz W, Dayan P, Montague PR, (1997) A neural substrate of prediction and reward. Science 275: 1593-9.
33. Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H, (2006) Midbrain dopamine neurons encode decisions for future action. Nature Neuroscience 9: 1057-63.
34. D'Ardenne K, McClure SM, Nystrom LE, Cohen JD, (2008) BOLD responses reecting dopaminergic signals in the human ventral tegmental area. Science 319: 1264-7.
35. Gottfried JA, O'Doherty J, Dolan RJ, (2003) Encoding predictive reward value in human amygdale and orbitofrontal cortex. Science 301: 1104-7.
36. Haruno M, Kawato M, (2006) Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. Journal of neurophysiology 95: 948-59.
37. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, et al. (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452-4.
38. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD, (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442: 1042-5.
39. Pessiglione M, Petrovic P, Daunizeau J, Palminteri S, Dolan RJ, et al. (2008) Subliminal instrumental conditioning demonstrated in the human brain. Neuron 59: 561-7.
40. Schönberg T, Daw ND, Joel D, O'Doherty JP, (2007) Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. Journal of Neuroscience 27: 12860-7.
41. Sutton RS, Barto AG, (1998) Reinforcement Learning: An Introduction. Cambridge, UK MIT Press URL http://webdocs.cs.ualberta.ca/ sutton/book/the-book.html.
42. Samejima K, Ueda Y, Doya K, Kimura M, (2005) Representation of action-specific reward values in the striatum. Science 310: 1337-40.
43. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ, (2006) Cortical substrates for exploratory decisions in humans. Nature 441: 876-9.
44. Rangel A, Hare T, (2010) Neural computations associated with goal-directed choice. Current opinion in neurobiology 20: 262-270.
45. Rao CR, (1973) Linear Statistical Inference and its Applications. New York Wiley-Interscience pp. 656 2nd edition.
46. Hairston W, Wallace M, Vaughan J, Stein B, Norris J, et al. (2003) Visual localization ability inuences cross-modal bias. Journal of Cognitive Neuroscience 15: 20-29.
47. Knill DC, (2005) Reaching for visual cues to depth: the brain combines depth cues differently for motor control and perception. Journal of Vision 5: 103-115.
48. Weisswange TH, Rothkopf CA, Rodemann T, Triesch J, (2009) Can reinforcement learning explain the development of causal inference in multisensory integration? In:8th International Conference on Development and Learning pp. 1-7 IEEE doi:10.1109/DEVLRN.2009.5175531. URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5175531.
49. Körding KP, Fukunaga I, Howard IS, Ingram JN, Wolpert DM, (2004) A neuroeconomics approach to inferring utility functions in sensorimotor control. PLoS biology 2: e330.
50. Whiteley L, Sahani M, (2008) Implicit knowledge of visual uncertainty guides decisions with asymmetric outcomes. Journal of Vision 8: 2.1-15.
51. Daw ND, Niv Y, Dayan P, (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience 8: 1704-11.
52. Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ, (2011) Model-Based Inuences on Humans' Choices and Striatal Prediction Errors. Neuron 69: 1204-1215.
53. Keramati M, Dezfouli A, Piray P, (2011) Speed/Accuracy Trade-O_ between the Habitual and the Goal-Directed Processes. PLoS Computational Biology 7: e1002055.
54. Lengyel M, Dayan P, (2008) Hippocampal contributions to control: the third way. In: Platt J, Koller D, Singer Y, Roweis S, editors. Advances in Neural Information Processing Systems 20 MIT Press pp. 889-896.
55. Shah A, Barto AG, (2009) E_ect on movement selection of an evolving sensory representation: a multiple controller model of skill acquisition. Brain Research 1299: 55-73.
56. Sloman SA, (1996) The Empirical Case for Two Systems of Reasoning. Psychological Bulletin 119: 3-22.
57. Brandwein AB, Foxe JJ, Russo NN, Altschuler TS, Gomes H, et al. (2011) The Development of Audiovisual Multisensory Integration Across Childhood and Early Adolescence: A High-Density Electrical Mapping Study. Cerebral Cortex 21: 1042-1055.
58. Held R, Ostrovsky Y, DeGelder B, Gandhi T, Ganesh S, et al. (2011) The newly sighted fail to match seen with felt. Nature Neuroscience 14: 551-553.
59. Helbig HB, Ernst MO, (2007) Knowledge about a common source can promote visual- haptic integration. Perception 36: 1523-1533.
60. Stocker AA, Simoncelli EP, (2008) A Bayesian Model of Conditioned Perception. In: Platt JC, Koller D, Singer Y, Roweis S, editors. Advances in Neural Information Processing Systems 20 (NIPS 2007) Vancouver, Canada MIT Press pp. 1409-1416 URL http://books.nips.cc/papers/files/nips20/NIPS20071016.pdf.
61. Jacobs RA, Fine I, (1999) Experience-dependent integration of texture and motion cues to depth. Vision Research 39: 4062-4075.
62. Triesch J, Ballard DH, Jacobs RA, (2002) Fast temporal dynamics of visual cue integration. Perception 31: 421-434.
63. Young MJ, Landy MS, Maloney LT, (1993) A perturbation analysis of depth perception from combinations of texture and motion cues. Vision Research 33: 2685-2696.
64. Wozny DR, Shams L, (2011) Recalibration of Auditory Space following Milliseconds of Cross-Modal Discrepancy. Journal of Neuroscience 31: 4607-4612.
65. Bair WN, Kiemel T, Jeka JJ, Clark JE, (2007) Development of multisensory reweighting for posture control in children. Experimental Brain Research 183: 435-446.
66. Lewald J, Guski R, (2003) Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. Cognitive Brain Research 16: 468-478.
67. Fujisaki W, Shimojo S, Kashino M, Nishida S, (2004) Recalibration of audiovisual simultaneity. Nature Neuroscience 7: 773-778.
68. Crick F, (1989) The recent excitement about neural networks. Nature 337: 129-32.
69. D'Souza P, Liu SC, Hahnloser RHR, (2010) Perceptron learning rule derived from spike-frequency adaptation and spike-time-dependent plasticity. Proceedings of the National Academy of Sciences of the United States of America 107: 4722-4727.
70. Roelfsema PR, van Ooyen A, (2005) Attention-gated reinforcement learning of internal representations for classi_cation. Neural computation 17: 2176-214.
71. Tao HzW, Zhang LI, Bi Gq, Poo Mm, (2000) Selective presynaptic propagation of long-term potentiation in de_ned neural networks. Journal of Neuroscience 20: 3233-43.
72. Gori M, Sandini G, Martinoli C, Burr D, (2010) Poor Haptic Orientation Discrimination in Nonsighted Children May Reect Disruption of Cross-Sensory Calibration. Current Biology 20: 223-225.
73. Strelnikov K, Rosito M, Barone P, (2011) Effect of Audiovisual Training on Monaural Spatial Hearing in Horizontal Plane. PLoS ONE 6: e18344.
74. Thomas GJ, (1941) Experimental study of the inuence of vision on sound localization. Journal of Experimental Psychology 28: 163-177.
75. Fuzessery ZM, Wenstrup JJ, Pollak GD, (1985) A representation of horizontal sound location in the inferior colliculus of the mustache bat (Pteronotus p. parnellii). Hearing research 20: 85-9.
76. Lee C, Rohrer WH, Sparks DL, (1988) Population coding of saccadic eye movements by neurons in the superior colliculus. Nature 332: 357-60.
77. Rumelhart DE, Hinton GE, Williams RJ, (1986) Learning internal representations by error propagation. Volume 1 edition MIT Press: 318-362.
78. Grant DA, Hake HW, Hornseth JP, (1951) Acquisition and extinction of a verbal conditioned response with differing percentages of reinforcement. Journal of experimental psychology 42: 1-5.
79. Rubinstein I, (1959) Some factors in probability matching. Journal of experimental psychology 57: 413-6.