Neural correlates of reliability-based cue weighting during multisensory integration

Nature Neuroscience

Volumn 15, Issue 1, 2012, Pages 146-154

  Fetsch, Christopher R ^a   Pouget, Alexandre ^b,c   Deangelis, Gregory C ^a,b   Angelaki, Dora E ^a,d  

^a WASHINGTON UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF ROCHESTER   (United States)

^c UNIVERSITY OF GENEVA   (Switzerland)

^d BAYLOR COLLEGE OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL EXPERIMENT; ARTICLE; BEHAVIOR; MACACA MULATTA; MOVEMENT PERCEPTION; MULTISENSORY INTEGRATION; NERVE CELL NETWORK; NONHUMAN; PERCEPTIVE DISCRIMINATION; PRIORITY JOURNAL; PROBABILITY; PSYCHOPHYSIOLOGY; RELIABILITY; RELIABILITY BASED CUE WEIGHTING; STATISTICAL ANALYSIS; STIMULUS RESPONSE; TASK PERFORMANCE; TEMPORAL CORTEX; VESTIBULAR STIMULATION;

ANIMALS; BRAIN; CONFLICT (PSYCHOLOGY); CUES; HEAD MOVEMENTS; MACACA MULATTA; MODELS, NEUROLOGICAL; MOTION PERCEPTION; NEURONS; PROPRIOCEPTION; VESTIBULE, LABYRINTH; VISUAL PERCEPTION;

EID: 84655167229     PISSN: 10976256     EISSN: 15461726     Source Type: Journal    
DOI: 10.1038/nn.2983     Document Type: Article

References (50)

1. Alais, D. & Burr, D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257-262 (2004). (Pubitemid 38215356)
2. Ernst, M.O. & Banks, M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429-433 (2002). (Pubitemid 34100953)
3. Hillis, J.M., Watt, S.J., Landy, M.S. & Banks, M.S. Slant from texture and disparity cues: optimal cue combination. J. Vis. 4, 967-992 (2004). (Pubitemid 41522184)
4. Jacobs, R.A. Optimal integration of texture and motion cues to depth. Vision Res. 39, 3621-3629 (1999). (Pubitemid 29347156)
5. Knill, D.C. & Saunders, J.A. Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Res. 43, 2539-2558 (2003). (Pubitemid 38367479)
6. Landy, M.S., Maloney, L.T., Johnston, E.B. & Young, M. Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Res. 35, 389-412 (1995).
7. van Beers, R.J., Wolpert, D.M. & Haggard, P. When feeling is more important than seeing in sensorimotor adaptation. Curr. Biol. 12, 834-837 (2002). (Pubitemid 34514369)
8. Young, M.J., Landy, M.S. & Maloney, L.T. A perturbation analysis of depth perception from combinations of texture and motion cues. Vision Res. 33, 2685-2696 (1993). (Pubitemid 23354566)
9. Clark, J.J. & Yuille, A.L. Data Fusion for Sensory Information Processing Systems (Kluwer Academic, Boston, 1990).
10. Cochran, W.G. Problems arising in the analysis of a series of similar experiments. J. R. Stat. Soc. 4 (suppl.) 102-118 (1937).
11. Gu, Y., Angelaki, D.E. & DeAngelis, G.C. Neural correlates of multisensory cue integration in macaque MSTd. Nat. Neurosci. 11, 1201-1210 (2008).
12. Gu, Y., DeAngelis, G.C. & Angelaki, D.E. A functional link between area MSTd and heading perception based on vestibular signals. Nat. Neurosci. 10, 1038-1047 (2007). (Pubitemid 47185203)
13. Fetsch, C.R., Turner, A.H., DeAngelis, G.C. & Angelaki, D.E. Dynamic reweighting of visual and vestibular cues during self-motion perception. J. Neurosci. 29, 15601-15612 (2009).
14. Ma, W.J., Beck, J.M., Latham, P.E. & Pouget, A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432-1438 (2006). (Pubitemid 44646216)
15. Landy, M.S., Banks, M.S. & Knill, D.C. Ideal-observer models of cue integration. in Sensory Cue Integration (eds. Trommershäuser, J., Kording, K.P. & Landy, M.S.) 5-29 (Oxford University Press, New York, 2011).
16. Knill, D.C. & Richards, W. Perception as Bayesian Inference (Cambridge University Press, New York, 1996).
17. Gu, Y., Watkins, P.V., Angelaki, D.E. & DeAngelis, G.C. Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal area. J. Neurosci. 26, 73-85 (2006). (Pubitemid 43089521)
18. Battaglia, P.W., Jacobs, R.A. & Aslin, R.N. Bayesian integration of visual and auditory signals for spatial localization. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 1391-1397 (2003).
19. Rosas, P., Wagemans, J., Ernst, M.O. & Wichmann, F.A. Texture and haptic cues in slant discrimination: reliability-based cue weighting without statistically optimal cue combination. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 22, 801-809 (2005). (Pubitemid 40748629)
20. Duffy, C.J. MST neurons respond to optic flow and translational movement. J. Neurophysiol. 80, 1816-1827 (1998). (Pubitemid 28489113)
21. Takahashi, K. et al. Multimodal coding of three-dimensional rotation and translation in area MSTd: comparison of visual and vestibular selectivity. J. Neurosci. 27, 9742-9756 (2007). (Pubitemid 47492188)
22. Dayan, P. & Abbott, L.F. Theoretical Neuroscience (MIT press, Cambridge, Massachusetts, 2001).
23. Foldiak, P. The 'ideal homunculus': statistical inference from neural population responses. in Computation and Neural Systems (eds. F.H. Eeckman & J.M. Bower) 55-60 (Kluwer Academic Publishers, Norwell, Massachusetts, 1993).
24. Sanger, T.D. Probability density estimation for the interpretation of neural population codes. J. Neurophysiol. 76, 2790-2793 (1996). (Pubitemid 26346448)
25. Morgan, M.L., DeAngelis, G.C. & Angelaki, D.E. Multisensory integration in macaque visual cortex depends on cue reliability. Neuron 59, 662-673 (2008).
26. Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. Responses of neurons in macaque MT to stochastic motion signals. Vis. Neurosci. 10, 1157-1169 (1993). (Pubitemid 2001372)
27. Heuer, H.W. & Britten, K.H. Linear responses to stochastic motion signals in area MST. J. Neurophysiol. 98, 1115-1124 (2007). (Pubitemid 47435868)
28. Ohshiro, T., Angelaki, D.E. & DeAngelis, G.C. A normalization model of multisensory integration. Nat. Neurosci. 14, 775-782 (2011).
29. Knill, D.C. & Pouget, A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712-719 (2004). (Pubitemid 39488074)
30. Stein, B.E. & Meredith, M.A. The Merging of the Senses (MIT Press, Cambridge, Massachusetts, 1993).
31. Bremmer, F., Klam, F., Duhamel, J.R., Ben Hamed, S. & Graf, W. Visual-vestibular interactive responses in the macaque ventral intraparietal area (VIP). Eur. J. Neurosci. 16, 1569-1586 (2002). (Pubitemid 35334994)
32. Chen, A., DeAngelis, G.C. & Angelaki, D.E. Representation of vestibular and visual cues to self-motion in ventral intraparietal cortex. J. Neurosci. 31, 12036-12052 (2011).
33. Fukushima, K. Corticovestibular interactions: anatomy, electrophysiology and functional considerations. Exp. Brain Res. 117, 1-16 (1997). (Pubitemid 27448027)
34. Xiao, Q., Barborica, A. & Ferrera, V.P. Radial motion bias in macaque frontal eye field. Vis. Neurosci. 23, 49-60 (2006). (Pubitemid 43376254)
35. Lewis, J.W. & Van Essen, D.C. Corticocortical connections of visual, sensorimotor and multimodal processing areas in the parietal lobe of the macaque monkey. J. Comp. Neurol. 428, 112-137 (2000).
36. Stanton, G.B., Bruce, C.J. & Goldberg, M.E. Topography of projections to posterior cortical areas from the macaque frontal eye fields. J. Comp. Neurol. 353, 291-305 (1995).
37. Felleman, D.J. & Van Essen, D.C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1-47 (1991).
38. Maunsell, J.H. & van Essen, D.C. The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. J. Neurosci. 3, 2563-2586 (1983). (Pubitemid 14187133)
39. Lappe, M., Bremmer, F., Pekel, M., Thiele, A. & Hoffmann, K.P. Optic flow processing in monkey STS: a theoretical and experimental approach. J. Neurosci. 16, 6265-6285 (1996). (Pubitemid 26314000)
40. Fukushima, K. Extraction of visual motion and optic flow. Neural Netw. 21, 774-785 (2008).
41. Perrone, J.A. & Stone, L.S. Emulating the visual receptive-field properties of MST neurons with a template model of heading estimation. J. Neurosci. 18, 5958-5975 (1998). (Pubitemid 28341371)
42. Knill, D.C. Robust cue integration: a Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant. J. Vis. 7, 1-24 (2007).
43. Cheng, K., Shettleworth, S.J., Huttenlocher, J. & Rieser, J.J. Bayesian integration of spatial information. Psychol. Bull. 133, 625-637 (2007).
44. Körding, K.P. et al. Causal inference in multisensory perception. PLoS ONE 2, e943 (2007).
45. Komatsu, H. & Wurtz, R.H. Relation of cortical areas MT and MST to pursuit eye movements. I. Localization and visual properties of neurons. J. Neurophysiol. 60, 580-603 (1988).
46. Tanaka, K. & Saito, H. Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. J. Neurophysiol. 62, 626-641 (1989). (Pubitemid 19236118)
47. Fetsch, C.R. et al. Spatiotemporal properties of vestibular responses in area MSTd. J. Neurophysiol. 104, 1506-1522 (2010).
48. Wichmann, F.A. & Hill, N.J. The psychometric function. I. Fitting, sampling, and goodness of fit. Percept. Psychophys. 63, 1293-1313 (2001). (Pubitemid 33610257)
49. Jazayeri, M. & Movshon, J.A. Optimal representation of sensory information by neural populations. Nat. Neurosci. 9, 690-696 (2006). (Pubitemid 44012268)
50. Gu, Y., Fetsch, C.R., Adeyemo, B., DeAngelis, G.C. & Angelaki, D.E. Decoding of MSTd population activity accounts for variations in the precision of heading perception. Neuron 66, 596-609 (2010).