Dynamic weighting of multisensory stimuli shapes decisionmaking in rats and humans

Journal of Vision

Volumn 13, Issue 6, 2013, Pages

  Sheppard, John P ^a,b   Raposo, David ^a,c   Churchland, Anne K ^a  

^a Howard Hughes Medical Institute Cold Spring Harbor Laboratory Cold Spring Harbor   (United States)

^b WATSON SCHOOL OF BIOLOGICAL SCIENCES   (United States)

^c CHAMPALIMAUD FOUNDATION   (Portugal)

Author keywords

Cue weighting; Decision making; multisensory integration; Psychophysics; Rodent; Sensory reliability

Indexed keywords

ADULT; ANIMAL; ARTICLE; AUDITORY STIMULATION; CUE WEIGHTING; DECISION MAKING; DECISION-MAKING; DOG; HEARING; HUMAN; LONG EVANS RAT; MALE; METHODOLOGY; MULTISENSORY INTEGRATION; PATTERN RECOGNITION; PERCEPTIVE DISCRIMINATION; PHOTOSTIMULATION; PHYSIOLOGY; PSYCHOMETRY; PSYCHOPHYSICS; RAT; RODENT; SENSORY RELIABILITY; VISION;

CUE WEIGHTING; DECISION-MAKING; MULTISENSORY INTEGRATION; PSYCHOPHYSICS; RODENT; SENSORY RELIABILITY;

ACOUSTIC STIMULATION; ADULT; ANIMALS; AUDITORY PERCEPTION; DECISION MAKING; DISCRIMINATION (PSYCHOLOGY); DOGS; FORM PERCEPTION; HUMANS; MALE; PHOTIC STIMULATION; PSYCHOMETRICS; RATS; RATS, LONG-EVANS; VISUAL PERCEPTION;

EID: 84878360257     PISSN: None     EISSN: 15347362     Source Type: Journal    
DOI: 10.1167/13.6.4     Document Type: Article

References (63)

1. Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14(3), 257-262.
2. Alais, D., Newell, F. N., & Mamassian, P. (2010). Multisensory processing in review: From physiology to behaviour. Seeing and Perceiving, 23(1), 3- 38.
3. Battaglia, P. W., Di Luca, M., Ernst, M. O., Schrater, P. R., Machulla, T., & Kersten, D. (2010). Withinand cross-modal distance information disambiguate visual size-change perception. PLoS Computational Biology, 6(3), e1000697.
4. Beck, J. M., Ma, W. J., Kiani, R., Hanks, T., Churchland, A. K., Roitman, J., et al. (2008). Probabilistic population codes for Bayesian decision making. Neuron, 60(6), 1142-1152.
5. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B (Methodological), 57(1), 289-300.
6. Busse, L., Ayaz, A., Dhruv, N. T., Katzner, S., Saleem, A. B., Scholvinck, M. L., et al. (2011). The detection of visual contrast in the behaving mouse. The Journal of Neuroscience, 31(31), 11351-11361.
7. Carandini, M., Heeger, D. J., & Movshon, J. A. (1997). Linearity and normalization in simple cells of the macaque primary visual cortex. The Journal of Neuroscience, 17(21), 8621-8644.
8. Chalupa, L. M., & Williams, R. W. (2008). Eye, retina, and visual system of the mouse. Cambridge, MA: MIT Press.
9. Cisek, P., Puskas, G. A., & El-Murr, S. (2009). Decisions in changing conditions: The urgencygating model. The Journal of Neuroscience, 29(37), 11560-11571.
10. Deneve, S., Latham, P. E., & Pouget, A. (2001). Efficient computation and cue integration with noisy population codes. Nature Neuroscience, 4(8), 826-831.
11. Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), 429-433.
12. Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012). Neural correlates of reliability-based cue weighting during multisensory integration. Nature Neuroscience, 15(1), 146-154.
13. Fetsch, C. R., Turner, A. H., DeAngelis, G. C., & Angelaki, D. E. (2009). Dynamic reweighting of visual and vestibular cues during self-motion perception. The Journal of Neuroscience, 29(49), 15601-15612.
14. Ghazanfar, A. A., Turesson, H. K., Maier, J. X., van Dinther, R., Patterson, R. D., & Logothetis, N. K. (2007). Vocal-tract resonances as indexical cues in rhesus monkeys. Current Biology, 17(5), 425-430.
15. Gold, J. I., & Shadlen, M. N. (2007). The neural basis of decision making. Annual Review of Neuroscience, 30, 535-574.
16. Heeger, D. J. (1993). Modeling simple-cell direction selectivity with normalized, half-squared, linear operators. Journal of Neurophysiology, 70(5), 1885- 1898.
17. Hillis, J. M., Ernst, M. O., Banks, M. S., & Landy, M. S. (2002). Combining sensory information: Mandatory fusion within, but not between, senses. Science, 298(5598), 1627-1630.
18. Hillis, J. M., Watt, S. J., Landy, M. S., & Banks, M. S. (2004). Slant from texture and disparity cues: Optimal cue combination. Journal of Vision, 4(12): 1, 967-992, http://www.journalofvision.org/content/4/12/1, doi:10.1167/4.12.1. [PubMed] [Article]
19. Hirokawa, J., Bosch, M., Sakata, S., Sakurai, Y., & Yamamori, T. (2008). Functional role of the secondary visual cortex in multisensory facilitation in rats. Neuroscience, 153(4), 1402-1417.
20. Hirokawa, J., Sadakane, O., Sakata, S., Bosch, M., Sakurai, Y., & Yamamori, T. (2011). Multisensory information facilitates reaction speed by enlarging activity difference between superior colliculus hemispheres in rats. PLoS One, 6(9), e25283.
21. Howard, I. P., & Templeton, W. B. (1966). Human spatial orientation. London, UK: Wiley.
22. Huk, A. C., & Meister, M. L. (2012). Neural correlates and neural computations in posterior parietal cortex during perceptual decision-making. Frontiers in Integrative Neuroscience, 6, 86.
23. Jacobs, R. A. (1999). Optimal integration of texture and motion cues to depth. Vision Research, 39(21), 3621-3629.
24. Jiang, W., Wallace, M. T., Jiang, H., Vaughan, J. W., & Stein, B. E. (2001). Two cortical areas mediate multisensory integration in superior colliculus neurons. Journal of Neurophysiology, 85(2), 506- 522.
25. Kesten, H. (1958). Accelerated stochastic-approximation. Annals of Mathematical Statistics, 29(1), 41- 59.
26. Kiani, R., Hanks, T. D., & Shadlen, M. N. (2008). Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. The Journal of Neuroscience, 28(12), 3017-3029.
27. Knill, D. C., & Saunders, J. A. (2003). Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Research, 43(24), 2539-2558.
28. Landy, M. S., & Kojima, H. (2001). Ideal cue combination for localizing texture-defined edges. Journal of The Optical Society of America A: Optics, Image, Science, and Vision, 18(9), 2307- 2320.
29. Lovelace, C. T., Stein, B. E., & Wallace, M. T. (2003). An irrelevant light enhances auditory detection in humans: A psychophysical analysis of multisensory integration in stimulus detection. Brain Research. Cognitive Brain Research, 17(2), 447-453.
30. Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006). Bayesian inference with probabilistic population codes. Nature Neuroscience, 9(11), 1432- 1438.
31. Maier, J. X., Chandrasekaran, C., & Ghazanfar, A. A. (2008). Integration of bimodal looming signals through neuronal coherence in the temporal lobe. Current Biology, 18(13), 963-968.
32. Maier, J. X., & Ghazanfar, A. A. (2007). Looming biases in monkey auditory cortex. The Journal of Neuroscience, 27(15), 4093-4100.
33. Meredith, M. A., Nemitz, J. W., & Stein, B. E. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. The Journal of Neuroscience, 7(10), 3215-3229.
34. Nienborg, H., & Cumming, B. G. (2009). Decisionrelated activity in sensory neurons reflects more than a neuron's causal effect. Nature, 459(7243), 89-92.
35. Ohshiro, T., Angelaki, D. E., & DeAngelis, G. C. (2011). A normalization model of multisensory integration. Nature Neuroscience, 14(6), 775-782.
36. Pelli, D. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10(4), 437-442.
37. Powers, A. R., 3rd, Hillock, A. R., & Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. The Journal of Neuroscience, 29(39), 12265-12274.
38. Prusky, G. T., West, P. W., & Douglas, R. M. (2000). Behavioral assessment of visual acuity in mice and rats. Vision Research, 40(16), 2201-2209.
39. Raposo, D., Sheppard, J. P., Schrater, P. R., & Churchland, A. K. (2012). Multisensory decisionmaking in rats and humans. The Journal of Neuroscience, 32(11), 3726-3735.
40. Rieke, F. (1997). Spikes : Exploring the neural code. Cambridge, MA: MIT Press.
41. Rosas, P., Wagemans, J., Ernst, M. O., & Wichmann, F. A. (2005). Texture and haptic cues in slant discrimination: Reliability-based cue weighting without statistically optimal cue combination. Journal of the Optical Society of America A, Optics, Image, Science and Vision, 22(5), 801-809.
42. Rosas, P., Wichmann, F. A., & Wagemans, J. (2007). Texture and object motion in slant discrimination: Failure of reliability-based weighting of cues may be evidence for strong fusion. Journal of Vision, 7(6):3, 1-21, http://www.journalofvision.org/content/7/6/3, doi:10.1167/7.6.3. [PubMed] [Article]
43. Sakata, S., Yamamori, T., & Sakurai, Y. (2004). Behavioral studies of auditory-visual spatial recognition and integration in rats. Experimental Brain Research, 159(4), 409-417.
44. Salinas, E., & Abbott, L. F. (1994). Vector reconstruction from firing rates. Journal of Computational Neuroscience, 1(1-2), 89-107.
45. Sanger, T. D. (1996). Probability density estimation for the interpretation of neural population codes. Journal of Neurophysiology, 76(4), 2790-2793.
46. Sclar, G., & Freeman, R. D. (1982). Orientation selectivity in the cat's striate cortex is invariant with stimulus contrast. Experimental Brain Research, 46(3), 457-461.
47. Senkowski, D., Schneider, T. R., Foxe, J. J., & Engel, A. K. (2008). Crossmodal binding through neural coherence: Implications for multisensory processing. Trends in Neurosciences, 31(8), 401-409.
48. Serwe, S., Kording, K. P., & Trommershauser, J. (2011). Visual-haptic cue integration with spatial and temporal disparity during pointing movements. Experimental Brain Research, 210(1), 67-80.
49. Shams, L. (2012). Early integration and Bayesian causal inference in multisensory perception. In M. M. Murray & M. T. Wallace (Eds.), The neural bases of multisensory processes. Boca Raton, FL: CRC Press.
50. Shams, L., Kamitani, Y., & Shimojo, S. (2000). Illusions. What you see is what you hear. Nature, 408(6814), 788.
51. Shams, L., Kamitani, Y., & Shimojo, S. (2002). Visual illusion induced by sound. Brain Research. Cognitive Brain Research, 14(1), 147-152.
52. Shams, L., Ma, W. J., & Beierholm, U. (2005). Soundinduced flash illusion as an optimal percept. NeuroReport, 16(17), 1923-1927.
53. Stanford, T. R., Quessy, S., & Stein, B. E. (2005). Evaluating the operations underlying multisensory integration in the cat superior colliculus. The Journal of Neuroscience, 25(28), 6499-6508.
54. Sugihara, T., Diltz, M. D., Averbeck, B. B., & Romanski, L. M. (2006). Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. The Journal of Neuroscience, 26(43), 11138-11147.
55. Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004). Matching behavior and the representation of value in the parietal cortex. Science, 304(5678), 1782-1787.
56. Thomas, J. P., & Shiffrar, M. (2010). I can see you better if I can hear you coming: Action-consistent sounds facilitate the visual detection of human gait. Journal of Vision, 10(12):14, 1-11, http://www.journalofvision.org/content/10/12/14, doi:10.1167/10.12.14. [PubMed] [Article]
57. Triesch, J., Ballard, D. H., & Jacobs, R. A. (2002). Fast temporal dynamics of visual cue integration. Perception, 31(4), 421-434.
58. Trommershauser, J., Kording, K., & Landy, M. S. (2011). Sensory cue integration. New York: Oxford University Press.
59. Vicente, M., Mendonca, A., Pouget, A., & Mainen, Z. (2012). Odor detection vs. mixture categorization: Crucial differences in behavioral and learning strategies. Paper presented at The 9th Annual Computational and Systems Neuroscience (Cosyne) Meeting, Salt Lake City, UT, USA.
60. Weiss, Y., Simoncelli, E. P., & Adelson, E. H. (2002). Motion illusions as optimal percepts. Nature Neuroscience, 5(6), 598-604.
61. Wichmann, F. A., & Hill, N. J. (2001). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63(8), 1293- 1313.
62. Young, M. J., Landy, M. S., & Maloney, L. T. (1993). A perturbation analysis of depth perception from combinations of texture and motion cues. Vision Research, 33(18), 2685-2696.
63. Zalevski, A. M., Henning, G. B., & Hill, N. J. (2007). Cue combination and the effect of horizontal disparity and perspective on stereoacuity. Spatial Vision, 20(1-2), 107-138.