A Rational Analysis of the Acquisition of Multisensory Representations

Cognitive Science

Volumn 36, Issue 2, 2012, Pages 305-332

  Yildirim, Ilker ^a   Jacobs, Robert A ^a  

^a UNIVERSITY OF ROCHESTER   (United States)

Author keywords

Bayesian modeling; Learning; Multisensory perception; Rational analysis

Indexed keywords

ARTICLE; AUDITORY STIMULATION; BAYES THEOREM; HEARING; HUMAN; LEARNING; PHOTOSTIMULATION; PSYCHOLOGICAL MODEL; VISION;

ACOUSTIC STIMULATION; AUDITORY PERCEPTION; BAYES THEOREM; HUMANS; LEARNING; MODELS, PSYCHOLOGICAL; PHOTIC STIMULATION; VISUAL PERCEPTION;

EID: 84857965362     PISSN: 03640213     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1551-6709.2011.01216.x     Document Type: Article

References (69)

1. Abidi, M. A., & Gonzalez, R. C. (1992). Data fusion in robotics and machine intelligence. San Diego, CA: Academic Press.
2. Adams, R. P., Wallach, H., & Ghahramani, Z. (2010). Learning the structure of deep sparse graphical models. In Y. W. Teh & M. Titterington (Eds.), Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (pp. 1-8).
3. Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257-262.
4. Alais, D., Newell, F. N., & Mamassian, P. (2010). Multisensory processing in review: From physiology to behavior. Seeing and Perceiving, 23, 3-38.
5. Alvarado, J. C., Rowland, B. A., Stanford, T. R., & Stein, B. E. (2008). A neural network model of multisensory integration also accounts for unisensory integration in superior colliculus. Brain Research, 1242, 13-23.
6. Amedi, A., Malach, R., Hendler, T., Peled, S., & Zohary, E. (2001). Visuo-haptic object-related activation in the ventral visual pathway. Nature Neuroscience, 4, 324-330.
7. Anastasio, T. J., Patton, P. E., & Belkacem-Boussaid, K. (2000). Using Bayes' rule to model multisensory enhancement in the superior colliculus. Neural Computation, 12, 1165-1187.
8. Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Erlbaum.
9. Austerweil, J. L., & Griffiths, T. L. (2009). Analyzing human feature learning as nonparametric Bayesian inference. In D. Koller, Y. Bengio, D. Schuurmans, & L. Bottou (Eds.), Advances in neural information processing systems 21. Cambridge, MA: MIT Press.
10. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. In W. Rosenblith (Ed.), Sensory communication (pp. 217-234). Cambridge, MA: MIT Press.
11. Battaglia, P. W., Jacobs, R. A., & Aslin, R. N. (2003). Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A, 20, 1391-1397.
12. Blei, D. M., Griffiths, T. L., & Jordan, M. I. (2010). The nested Chinese restaurant process and Bayesian inference of topic hierarchies. Journal of the ACM, 57, 1-30.
13. Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., Woodruff, P. W. R., Iversen, S. D., & David, A. S. (1997). Activation of auditory cortex during silent lipreading. Science, 276, 593-596.
14. Chater, N., & Oaksford, M. (1999). Ten years of the rational analysis of cognition. Trends in Cognitive Sciences, 3, 57-65.
15. Chater, N., & Oaksford, M. (2008). The probabilistic mind. Oxford, UK: Oxford University Press.
16. Clark, J. J., & Yuille, A. L. (1990). Data fusion for sensory information processing systems. Boston: Kluwer Academic Publishers.
17. Cooke, T., Jäkel, F., Wallraven, C., & Bülthoff, H. H. (2007). Multimodal similarity and categorization of novel, three-dimensional objects. Neuropsychologia, 45, 484-495.
18. Courville, A., Eck, D., & Bengio, Y. (2009). An infinite factor model hierarchy via a noisy-or mechanism. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in neural information processing systems 22 (pp. 405-413).
19. Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. New York: Wiley.
20. Cox, R. T. (1961). The algebra of probable inference. Baltimore: Johns Hopkins University Press.
21. Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433.
22. Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1, 209-230.
23. Fetsch, C. R., DeAngelis, G. C., & Angelaki, D. E. (2010). Visual-vestibular cue integration for heading perception: Applications of optimal cue integration theory. European Journal of Neuroscience, 31, 1721-1729.
24. Geisler, W. S. (2004). Ideal observer analysis. In L. M. Chalupa & J. S. Werner (Eds.), The visual neurosciences (pp. 825-837). Cambridge, MA: MIT Press.
25. de Gelder, B., & Vroomen, J. (2000). The perception of emotions by ear and by eye. Cognition and Emotion, 14, 289-311.
26. de Gelder, B., & Vroomen, J. (2003). Multisensory integration, perception, and ecological validity. Trends in Cognitive Sciences, 7, 460-466.
27. Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (1995). Bayesian data analysis. London, UK: Chapman & Hall.
28. Gepshtein, S., & Banks, M. S. (2003). Viewing geometry determines how vision and touch combine in size perception. Current Biology, 13, 483-488.
29. Gilks, W. R., Richardson, S., & Spiegelhalter, D. J. (1996). Markov Chain Monte Carlo in practice. London, UK: Chapman & Hall.
30. Goldstone, R. L. (1998). Perceptual learning. Annual Review of Psychology, 49, 585-612.
31. Green, P., & Richardson, S. (2001). Modeling heterogeneity with and without the Dirichlet process. Scandinavian Journal of Statistics, 28, 355-377.
32. Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York: Wiley.
33. Griffiths, T. L., & Ghahramani, Z. (2005). Infinite latent feature models and the Indian buffet process. Gatsby Unit Technical Report GCNU-TR-2005-001.
34. Griffiths, T. L., & Ghahramani, Z. (2006). Infinite latent feature models and the Indian buffet process. In B. Schölkopf, J. Platt, & T. Hofmann (Eds.), Advances in neural information processing systems 18. Cambridge, MA: MIT Press.
35. Griffiths, T. L., Kemp, C., & Tenenbaum, J. B. (2008). Bayesian models of cognition. In R. Sun (Ed.), The Cambridge handbook of computational psychology. New York: Cambridge University Press.
36. Hershey, J., & Movellan, J. R. (1999). Audio-vision: Locating sounds via audio-visual synchrony. In Proceedings of the 6th symposium on neural computation (pp. 57-63). Los Angeles, CA: California Institute of Technology.
37. Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press.
38. Knill, D. C., & Richards, W. (1996). Perception as Bayesian inference. Cambridge, UK: Cambridge University Press.
39. Konkle, T., Wang, Q., Hayward, V., & Moore, C. I. (2009). Motion aftereffects transfer between touch and vision. Current Biology, 19, 745-750.
40. von Kriegstein, K., & Giraud, A.-L. (2006). Implicit multisensory associations influence voice recognition. PLoS Biology, 4, e326.
41. Lacey, S., Campbell, C., & Sathian, K. (2007). Vision and touch: Multiple or multisensory representations of objects? Perception, 36, 1513-1521.
42. Lacey, S., Hall, J., & Sathian, K. (2010). Are surface properties integrated into visuohaptic object representations? European Journal of Neuroscience, 31, 1882-1888.
43. Landy, M. S., Maloney, L. T., Johnston, E. B., & Young, M. J. (1995). Measurement and modeling of depth cue combination: In defense of weak fusion. Vision Research, 35, 389-412.
44. Lehmann, S., & Murray, M. M. (2005). The role of multisensory memories in unisensory object discrimination. Cognitive Brain Research, 24, 326-334.
45. Marr, D. (1982). Vision. New York: Freeman.
46. Massaro, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
47. Massaro, D. W., Cohen, M. M., Campbell, C. S., & Rodriguez, T. (2001). Bayes factor of model selection validates FLMP. Psychonomic Bulletin & Review, 8, 1-17.
48. Movellan J. R. (1995). Visual speech recognition with stochastic networks. In G. Tesauro, D. Toruetzky, & T. Leen (Eds.), Advances in neural information processing systems 7. Cambridge, MA: MIT Press.
49. Navarro, D. J., & Griffiths, T. L. (2008). Latent features in similarity judgments: A nonparametric Bayesian approach. Neural Computation, 20, 2597-2628.
50. Navarro, D. J., Griffiths, T. L., Steyvers, M., & Lee, M. D. (2006). Modeling individual differences using Dirichlet processes. Journal of Mathematical Psychology, 50, 101-122.
51. Neal, R. M. (2000). Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9, 249-265.
52. Neapolitan, R. E. (2004). Learning Bayesian networks. Upper Saddle River, NJ: Pearson Prentice Hall.
53. Oaksford, M., & Chater, N. (1999). Rational models of cognition. Oxford, UK: Oxford University Press.
54. Pekkola, J., Ojanen, V., Autti, T., Jääskeläinen, I. P., Möttönen, R., Tarkiainen, A., & Sams, M. (2005). Primary auditory cortex activation by visual speech: An fMRI study at 3T. Neuroreport, 16, 125-128.
55. Pouget, A., Deneve, S., & Duhamel, J. R. (2002). A computational perspective on the neural basis of multisensory spatial representations. Nature Review Neuroscience, 3, 741-747.
56. Quiroga, R. Q., Kraskov, A., Koch, C., & Fried, I. (2009). Explicit encoding of multimodal percepts by single neurons in the human brain. Current Biology, 19, 1308-1313.
57. Rasmussen, C. E. (2000). The infinite Gaussian mixture model. In S. A. Solla, T. K. Leen, & Müller, K.-R. (Eds.), Advances in neural information processing systems 12 (pp. 554-560). Cambridge, MA: MIT Press.
58. Rasmussen, C. E., & Ghahramani, Z. (2001). Occam's razor. In T. K. Leen, T. G. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems 13 (pp. 294-300). Cambridge, MA: MIT Press.
59. Sathian, K., Zangaladze, A., Hoffman, J. M., & Grafton, S. T. (1997). Feeling with the mind's eye. Neuroreport, 8, 3877-3881.
60. Seitz, A. R., Kim, R., & Shams, L. (2006). Sound facilitates visual learning. Current Biology, 16, 1422-1427.
61. Shams, L., & Seitz, A. R. (2008). Benefits of multisensory learning. Trends in Cognitive Sciences, 12, 411-417.
62. Shepard, R. N., & Arabie, P. (1979). Additive clustering: Representation of similarities as combinations of discrete overlapping properties. Psychological Review, 86, 87-123.
63. Tanabe, H. C., Honda, M., & Sadato, N. (2005). Functionally segregated neural substrates for arbitrary audiovisual paired-association learning. Journal of Neuroscience, 25, 6409-6418.
64. Todorov, E. (2004). Optimality principles in sensorimotor control. Nature Neuroscience, 7, 907-915.
65. Wallace, M. T., & Stein, B. E. (2001). Sensory and multisensory responses in the newborn monkey superior colliculus. Journal of Neuroscience, 21, 8886-8894.
66. Williamson, S., Orbanz, P., & Ghahramani, Z. (2010). Dependent India buffet processes. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (pp. 924-931).
67. Wozny, D. R., Beierholm, U. R., & Shams, L. (2008). Human trimodal perception follows optimal statistical inference. Journal of Vision, 8(3), 1-11.
68. Zangaladze, A., Epstein, C. M., Grafton, S. T., & Sathian, K. (1999). Involvement of visual cortex in tactile discrimination of orientation. Nature, 401, 587-590.
69. Zhou, Y. D., & Fuster, J. M. (2000) Visuo-tactile cross-modal associations in cortical somatosensory cells. Proceedings of the National Academy of Sciences of the United States of America, 97, 9777-9782.