Multistability, cross-modal binding and the additivity of conjoined grouping principles

Philosophical Transactions of the Royal Society B: Biological Sciences

Volumn 367, Issue 1591, 2012, Pages 954-964

  Kubovy, Michael ^a   Yu, Minhong ^a  

^a UNIVERSITY OF VIRGINIA   (United States)

Author keywords

Additivity; Auditory necklaces; Conjoined grouping principles; Cross modal binding; Indispensable attributes; Multistability

Indexed keywords

AUDITORY CUE; CHEMICAL BINDING; PERCEPTION; VISUAL CUE;

EID: 84857539890     PISSN: 09628436     EISSN: 14712970     Source Type: Journal    
DOI: 10.1098/rstb.2011.0365     Document Type: Review

References (63)

1. McGurk, H. & MacDonald, J. 1976 Hearing lips and seeing voices. Nature 264, 746-748. (doi:10.1038/ 264746a0)
2. Jack, C. E. & Thurlow, W. R. 1973 Effects of degree of visual association and angle of displacement on the 'ventriloquism' effect. Percept. Mot. Skills 37, 967-979. (doi:10.2466/pms.1973.37.3.967)
3. Thurlow, W. R. & Jack, C. E. 1973 Certain determinants of the 'ventriloquism effect'. Percept. Mot. Skills 36, 1171-1184. (doi:10.2466/pms.1973.36.3c.1171)
4. van Wassenhove, V., Grant, K. W. & Poeppel, D. 2007 Temporal window of integration in auditory-visual speech perceptions. Neuropsychologia 45, 598-607. (doi:10.1016/j.neuropsychologia.2006.01.001)
5. Bertelson, P., Vroomen, J. & de Gelder, B. 2003 Visual recalibration of auditory speech identification: a McGurk aftereffect. Psychol. Sci. 14, 592-597. (doi:10. 1046/j.0956-7976.2003.psci_1470.x)
6. Bushara, K. O., Grafman, J. & Hallett, M. 2001 Neural correlates of auditory-visual stimulus onset asynchrony detection. J. Neurosci. 21, 300-304.
7. Colin, C., Radeau, M., Deltenre, P. & Morais, J. 2001 Rules of intersensory integration in spatial scene analysis and speech reading. Psychol. Belg. 41, 131-144.
8. Colin, C., Radeau, M. & Deltenre, P. 2005 Top-down and bottom-up modulation of audiovisual integration in speech. Eur. J. Cogn. Psychol. 17, 541-560. (doi:10. 1080/09541440440000168)
9. Jones, J. A. & Callan, D. E. 2003 Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. Cogn. Neurosci. Neuropsychol. 14, 1129-1133.
10. Kanaya, S. & Yokosawa, K. 2011 Perceptual congruency of audio-visual speech affects ventriloquism with bilateral visual stimuli. Psychon. Bull. Rev. 18, 123-128. (doi:10. 3758/s13423-010-0027-z)
11. Morsella, E. 2005 The function of phenomenal states: supramodular interaction theory. Psychol. Rev. 112, 1000-1021. (doi:10.1037/0033-295X.112.4.1000)
12. Kö hler, W. 1929 Gestalt psychology. New York, NY: Liveridge.
13. Kö hler, W. 1947 Gestalt psychology: an introduction to new concepts in modern psychology (revised edition). New York, NY: Liveright.
14. Ramachandran, V. & Hubbard, E. M. 2001 Synaesthesia: a window into perception, thought and language. J. Conscious. Stud. 8, 3-34.
15. Maurer, D., Pathman, T. & Mondloch, C. J. 2006 The shape of boubas: sound-shape correspondences in toddlers and adults. Dev. Sci. 9, 316-322. (doi:10.1111/j. 1467-7687.2006.00495.x)
16. Parise, C. V. & Spence, C. 2009 'When birds of a feather flock together': synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS ONE 4, e5664. (doi:10.1371/journal.pone.0005664)
17. Gepshtein, S., Burge, J., Ernst, M. O. & Banks, M. S. 2005 The combination of vision and touch depends on spatial proximity. J. Vis. 5, 1013-1023. (doi:10. 1167/5.11.7)
18. Schutz, M. & Kubovy, M. 2009 Causality and crossmodal integration. J. Exp. Psychol. Hum. Percept. Perform. 35, 1791-1810. (doi:10.1037/a0016455)
19. Schutz, M. & Kubovy, M. 2009 Deconstructing a musical illusion: point-light representations capture salient properties of impact motions. Can. Acoust. 37, 23-28.
20. Armontrout, J. A., Schutz, M. & Kubovy, M. 2009 Visual determinants of a cross-modal illusion. Attent. Percept. Psychophys. 71, 1618-1627. (doi:10.3758/APP.71. 7.1618)
21. Adams, P. A. & Haire, M. 1959 The effect of orientation on the reversal of one cube inscribed in another. Am. J. Psychol. 72, 296-299. (doi:10.2307/1419384)
22. Bradley, D. R. & Petry, H. M. 1977 Organizational determinants of subjective contour: the subjective Necker cube. Am. J. Psychol. 90, 253-262. (doi:10. 2307/1422047)
23. Schumann, F. 1987 Contributions to the analysis of visual perception: first paper: some observations on the combination of visual impressions into units. In The perception of illusory contours (eds S. Petry & G. E. Meyer), pp. 21-34. New York, NY: Springer.
24. Kanizsa, G. 1955 Margini quasi-percettivi in campi con stimolazione omogenea. Riv. Psicol. 49, 7-30.
25. Kanizsa, G. 1979 Organization in vision: essays on Gestalt perception. New York, NY: Praeger.
26. Warren, R. M. & Gregory, R. L. 1958 An auditory analogue of the visual reversible figure. Am. J. Psychol. 71, 612-613. (doi:10.2307/1420267)
27. Sato, M., Basirat, A. & Schwartz, J. 2007 Visual contribution to the multistable perception of speech. Percept. Psychophys. 69, 1360. (doi:10.3758/BF03192952)
28. Strother, L. & Kubovy, M. 2003 Perceived complexity and the grouping effect in band patterns. Acta Psychol. 114, 229-244. (doi:10.1016/j.actpsy.2003.06.001)
29. Kubovy, M. 1981 Concurrent pitch segregation and the theory of indispensable attributes. In Perceptual organization (eds M. Kubovy & J. R. Pomerantz), pp. 55-98. Hillsdale, NJ: Lawrence Erlbaum.
30. Kubovy, M. 1988 Should we resist the seductiveness of the space:time::vision:audition analogy? J. Exp. Psychol. Hum. Percept. Perform. 14, 318-320. (doi:10.1037/ 0096-1523.14.2.318)
31. Kubovy, M. & Schutz, M. 2010 Audio-visual objects. Rev. Philos.Psychol. 1,41-61. (doi:10.1007/s13164-009-0004-5)
32. Kubovy, M. & Van Valkenburg, D. 2001 Auditory and visual objects. Cognition 80, 97-126. (doi:10.1016/ S0010-0277(00)00155-4)
33. Cook, L. A. & Van Valkenburg, D. L. 2009 Audio-visual organisation and the temporal ventriloquism effect between grouped sequences: evidence that unimodal grouping precedes cross-modal integration. Perception 38, 1220-1233. (doi:10.1068/p6344)
34. Pressnitzer, D. & Hupé, J.-M. 2006 Temporal dynamics of auditory and visual bistability reveal common principles of perceptual organization. Curr. Biol. 16, 1351-1357. (doi:10.1016/j.cub.2006.05.054)
35. Bregman, A. S. & Campbell, J. 1971 Primary auditory stream segregation and perception of order in rapid sequences of tones. J. Exp. Psychol. 89, 244-249. (doi:10.1037/h0031163)
36. van Noorden, L. P. A. S. 1975 Temporal coherence in the perception of tone sequences. PhD thesis, Technical University of Eindhoven, Eindhoven, The Netherlands.
37. Wallach, H. 1935 Ü ber visuell wahrgenommene Bewegungsrichtung. Psychol. Forschung 20, 325-380. (doi:10.1007/BF02409790)
38. Meng, M. & Tong, F. 2004 Can attention selectively bias bistable perception? Differences between binocular rivalry and ambiguous figures. J. Vis. 4(7), 2. (doi:10.1167/4.7.2)
39. Binda, P., Lunghi, C. & Morrone, C. 2010 Touch disambiguates rivalrous perception at early stages of visual analysis. J. Vis. 10(7), 854. (doi:10.1167/10.7.854)
40. Lunghi, C., Binda, P. & Morrone, M. C. 2010 Touch disambiguates rivalrous perception at early stages of visual analysis. Curr. Biol. 20, R143-R144. (doi:10. 1016/j.cub.2009.12.015)
41. Maruya, K., Yang, E. & Blake, R. 2007 Voluntary action influences visual competition. Psychol. Sci. 18, 1090-1098. (doi:10.1111/j.1467-9280.2007.02030.x)
42. Conrad, V., Bartels, A., Kleiner, M. & Noppeney, U. 2010 Audiovisual interactions in binocular rivalry. J. Vis. 10(10), 27. (doi:10.1167/10.10.27)
43. Kang, M.-S. & Blake, R. 2005 Perceptual synergy between seeing and hearing revealed during binocular rivalry. Psichologija 32, 7-15.
44. Wertheimer, M. 1923 Untersuchungen zur Lehre von der Gestalt, II. Psychol. Forschung 4, 301-350, 1923. [Transl. extracted from Ellis, pp. p71-88.]
45. Ash, M. G. 1998 Gestalt psychology in German culture, 1890-1967: Holism and the quest for objectivity. Cambridge, UK: Cambridge University Press.
46. Schumann, F. 1987 Beiträge zur psychologie der gesichtswahrnehmung. Z Psychol. 23, 1-32, 1900. [Transl. by Schumann.]
47. Müller, G. E. 1903 Die gesichtspunkte und die tatsachen der psychophysischen methodik. In Ergebnisse der physiologie, vol. II, part 2 (eds L. Asher & K. Spiro), pp. 267-516. Wiesbaden, Germany: J.F. Bergmann.
48. Ellis, W. D. (ed.) 1938 A source book of Gestalt psychology. New York, NY: Harcourt, Brace and Company.
49. Kubovy, M. 1994 The perceptual organization of dot lattices. Psychon. Bull. Rev. 1, 182-190. (doi:10.3758/ BF03200772)
50. Bravais, A. 1949 Crystallographic studies: on the systems formed by points regularly distributed on a plane or in space. Crystallographic Society of America, N.P. Original work published 1866. [Transl. from É tudes cristallographiques: mémoire sur les systèmes formés par des points distribués régulièrement sur un plan ou dans l'espace, Paris, France: Gauthier-Villars.]
51. Oyama, T. 1961 Perceptual grouping as a function of proximity. Percept. Mot. Skills 13, 305-306.
52. Kubovy, M. & Wagemans, J. 1995 Grouping by proximity and multistability in dot lattices: a quantitativegestalt theory. Psychol. Sci. 6, 225-234. (doi:10.1111/j. 1467-9280.1995.tb00597.x)
53. Kubovy, M., Holcombe, A. O. & Wagemans, J. 1998 On the lawfulness of grouping by proximity. Cogn. Psychol. 35, 71-98. (doi:10.1006/cogp.1997.0673)
54. Kubovy, M. & van den Berg, M. 2008 The whole is equal to the sum of its parts: a probabilistic model of grouping by proximity and similarity in regular patterns. Psychol. Rev. 115, 131-154. (doi:10.1037/0033-295X. 115.1.131)
55. Deutsch, D. 1980 The processing of structured and unstructured tonal sequences. Percept. Psychophys. 28, 381-389. (doi:10.3758/BF03204881)
56. Longuet-Higgins, H. C. & Lee, C. S. 1982 The perception of musical rhythms. Perception 11, 115-128. (doi:10. 1068/p110115)
57. Martin, J. G. 1972 Rhythmic (hierarchical) versus serial structure in speech and other behavior. Psychol. Rev. 79, 487-509. (doi:10.1037/h0033467)
58. Royer, F. L. & Garner, W. R. 1966 Response uncertainty and perceptual difficulty of auditory temporal patterns. Percept. Psychophys. 1, 41-47. (doi:10.3758/ BF03207820)
59. Garner, W. R. & Gottwald, R. L. 1968 The perception and learning of temporal patterns. Q. J. Exp. Psychol. 20(Pt. II), 97-109. (doi:10.1080/14640746808400137)
60. Royer, F. L. & Garner, W. R. 1970 Perceptual organization of nine-element auditory temporal patterns. Percept. Psychophys. 7, 115-120. (doi:10.3758/ BF03210146)
61. Preusser, D., Garner, W. R. & Gottwald, R. L. 1970 Perceptual organization of two-element temporal patterns as a function of their component one-element patterns. Am. J. Psychol. 83, 151-170. (doi:10.2307/1421321)
62. Boker, S. M. & Kubovy, M. 1998 The perception of segmentation in sequences: local information provides the building blocks for global structure. In Timing of behavior: neural, computational, and psychological perspectives, (eds D. A. Rosenbaum & C. E. Collyer), pp. 109-123. Cambridge, MA: MIT Press.
63. Gibson, J. J. 1966 The senses considered as perceptual systems. Boston, MA: Houghton Mifflin.