Multisensory perception as an associative learning process

Frontiers in Psychology

Volumn 5, Issue SEP, 2014, Pages

  Connolly, Kevin ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Associative learning; Binding; Crossmodal integration; Crossmodal interaction; Feature binding; Multimodal interaction; Multisensory integration; Perceptual learning

Indexed keywords

EID: 84907210393     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2014.01095     Document Type: Article

References (47)

1. Altieri, N., Pisoni, D. B., & Townsend, J. T. (2011). Some behavioral and neurobiological constraints on theories of audiovisual speech integration: a review and suggestions for new directions. Seeing and perceiving, 24(6), 513-539. doi: 10.1163/187847611X595864
2. Altieri, N., & Townsend, J. T. (2011). An assessment of behavioral dynamic information processing measures in audiovisual speech perception. Frontiers in psychology, 2:238. doi: 10.3389/fpsyg.2011.00238
3. Bertelson, P. and de Gelder, B. (2004). "The psychology of multimodal perception," in Crossmodal Space and Crossmodal Attention, eds C. Spence and J. Driver (New York: Oxford University Press), 141-177.
4. Bushara, K. O., Hanakawa, T., Immisch, I., Toma, K., Kansaku, K., and Hallett, M. (2003). Neural correlates of cross-modal binding. Nature Neuroscience, 6(2):190-195. doi: 10.1038/nn993
5. Colzato, L. S., Raffone, A., & Hommel, B. (2006). What do we learn from binding features? Evidence for multilevel feature integration. Journal of Experimental Psychology: Human Perception and Performance, 32(3), 705-716. doi: 10.1037/0096-1523.32.3.705
6. Connolly, K. (2013). How to test Molyneux's question empirically. i-Perception 4(8) 508- 510. doi: 10.1068/i0623jc
7. Connolly, K. (2014). "Making sense of multiple senses," in Consciousness Inside and Out: Phenomenology, Neuroscience, and the Nature of Experience, ed. R. Brown (Dordrecht: Springer), 351-364.
8. Diamond R, Carey S. 1986. Why faces are and are not special: an effect of expertise. J. Exp. Psychol. Gen. 115:107-17. doi: 10.1037/0096-3445.115.2.107
9. Fowler, C. A. (2004). "Speech as a supramodal or amodal phenomenon," in The Handbook of Multisensory Processes, eds G. A. Calvert, C. Spence, and B. E. Stein (Cambridge, MA: MIT Press), 189-201.
10. Gauthier, I., & Tarr, M.J. (1997). Becoming a "greeble" expert: Exploring mechanisms for face recognition, Vision Res. 37: 1673-1682. doi: 10.1016/S0042-6989(96)00286-6
11. Gibson, E. J. (1963). Perceptual learning. Annu. Rev. Psychol. 14, 29-56. doi: 10.1146/annurev.ps.14.020163.000333
12. Gibson, J. J. (1950). The Perception of the Visual World. Boston: Houghton Mifflin.
13. Gibson, J. J. (1972). "A theory of direct visual perception," in The Psychology of Knowing, eds J. R. Royce and W. W. Rozeboom (New York: Gordon & Breach), 215-240.
14. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston: Houghton Mifflin.
15. Goldstone, R. L. (1998). Perceptual learning. Annu. Rev. Psychol. 49, 585-612. doi: 10.1146/annurev.psych.49.1.585
16. Goldstone, R. L. (2010). "Forward to gauthier," in Perceptualexpertise: Bridging Brain and Behavior, eds M. Tarr and D. Bub (Oxford New York: Oxford University Press).
17. Goldstone, R. L. & Byrge, Lisa A. (2014). "Perceptual learning," in The Oxford Handbook of the Philosophy of Perception, ed. M. Matthen (Oxford: Oxford University Press).
18. Held, R., Y. Ostrovsky, B. de Gelder, T. Gandhi, S. Ganesh, U. Mathur, and P. Sinha. (2011). The newly sighted fail to match seen shape with felt. Nature Neuroscience 14: 551-53. doi: 10.1038/nn.2795
19. James, W. (1890). Principles of Psychology. New York: Holt. doi: 10.1037/11059-000
20. Kubovy, M. and Schutz, M. (2010). Audio-visual objects. Review of Philosophy and Psychology, 1(1):41-61. doi: 10.1007/s13164-009-0004-5
21. Leibniz, G. W. (1765/1982). "Perception," in New Essays on Human Understanding, trans. eds P. Remnant and J. Bennett. Cambridge: Cambridge University Press.
22. Locke, J. (1690/1975). "Perception," in An Essay Concerning Human Understanding Book II, ed. P. H. Nidditch (NewYork: Oxford University Press).
23. Navarra, J., Yeung, H. H., Werker, J. F., and Soto-Faraco, S. (2012). "Multisensory interactions inspeech perception," in The New Handbook of MultisensoryProcessing, ed. B. E. Stein, (Cambridge, MA: MIT Press), 435-452.
24. O'Callaghan, C. (2014). "Not all perceptual experience is modality specific," in Perception and Its Modalities, eds D. Stokes, S. Biggs, and M. Matthen, (Oxford: Oxford University Press).
25. O'Callaghan, Casey (forthcoming). Intermodal Binding Awareness. In Sensory Integration and the Unity of Consciousness, David Bennett and Christopher Hill (eds.). MIT Press.
26. Pons, F., Lewkowicz, D. J., Soto-Faraco, S., & Sebastia´n-Galle´s, N. (2009). Narrowing of intersensory speech perception in infancy. Proceedings of the National Academy of Sciences, 106(26), 10598-10602. doi: 10.1073/pnas.0904134106
27. Pourtois, G., de Gelder, B., Vroomen, J., Rossion, B., and Crommelinck, M. (2000). The timecourse of intermodal binding between seeing and hearing affective information. Neuroreport, 11(6):1329-1333.
28. Quine, W.V.O. (1960). Word and Object. MIT Press, Cambridge, MA.
29. Schwenkler, J. (2012). On the matching of seen and felt shapes by newly sighted subjects. i-Perception 3: 186-88. doi: 10.1068/i0525ic
30. Schwenkler, J. (2013). Do things look the way they feel? Analysis 73: 86-96.
31. Shiffrin, R. M., and Lightfoot, N. (1997). Perceptual learning of alphanumeric-like characters. Psychol. Learn. Motiv. 36, 45-81. doi: 10.1016/S0079-7421(08)60281-9
32. Smith, B. C. (2013). "Taste, philosophical perspectives," in Encyclopedia of the Mind. Thousand Oaks, ed. H. E. Pashler (California: SAGE Publications, Inc.).
33. Smith, L., and Yu, C. (2007). "Infants rapidly learn words from noisy data via cross-situational statistics," in Proceedings of the 29th Annual Conference of the Cognitive Science Society (Mahwah, NJ: Lawrence Erlbaum), 27.
34. Smith, L., & Yu, C. (2008). Infants rapidly learn word-referent mappings via cross-situational statistics. Cognition, 106(3), 1558-1568. doi: 10.1016/j.cognition.2007.06.010
35. Spence, C. (2007). Audiovisual multisensory integration. Acoustical Science and Technology, 28(2):61-70.
36. Spence, C., and Bayne, T. (2014). "Is consciousness multisensory?" in Perception and its Modalities, eds S. Biggs, D. Stokes, and M. Matthen (Oxford: Oxford University Press).
37. Spence, C. and Driver, J., (ed.). (2004). Crossmodal Space and Crossmodal Attention. Oxford University Press, Oxford. doi: 10.1093/acprof:oso/9780198524861.001.0001
38. Stein, B. E. (2012). The New Handbook of Multisensory Processing. MIT Press, Cambridge, MA.
39. Stein B.E., Wallace M.T. 1996. Comparisons of cross-modality integration in midbrain and cortex. Prog. Brain Res. 112:289-99. doi: 10.1016/S0079-6123(08)63336-1
40. Townsend, J. T., & Nozawa, G. (1995). Spatio-temporal properties of elementary perception: An investigation of parallel, serial, and coactive theories. Journal of Mathematical Psychology, 39(4), 321-359. doi: 10.1006/jmps.1995.1033
41. Townsend, J. T., & Wenger, M. J. (2004). A theory of interactive parallel processing: new capacity measures and predictions for a response time inequality series. Psychological review, 111(4), 1003. doi: 10.1037/0033-295X.111.4.1003
42. Vatakis, A. and Spence, C. (2007). Crossmodal binding: Evaluating the "unity assumption" using audiovisual speech stimuli. Perception and Psychophysics, 69(5):744-756. doi: 10.3758/BF03193776
43. Wallace, M. T. (2004). The development of multisensory processes. Cognitive Processing, 5(2), 69-83. doi: 10.1007/s10339-004-0017-z
44. Yu, C., & Smith, L. B. (2006)."Statistical cross-situational learning to build word-to-worldmappings," in Proceedings of the 28th Annual Conference of the Cognitive Science Society (Mahwah, NJ: Lawrence Erlbaum).
45. Yu, C. Smith, L. (2007). Rapid word learning under uncertainty via cross-situational statistics. Psychological Science, 18, 414-420. doi: 10.1111/j.1467-9280.2007.01915.x
46. Yu, C., & Smith, L. B. (2011). What you learn is what you see: using eye movements to study infant cross-situational word learning. Developmental Science, 14(2), 165-180. doi: 10.1111/j.1467-7687.2010.00958.x
47. Yu, C., & Smith, L. B. (2012). Modeling cross-situational word-referent learning: Prior questions. Psychological review, 119(1), 21-39. doi: 10.1037/a0026182