A spatially collocated sound thrusts a flash into awareness

Frontiers in Integrative Neuroscience

Volumn 9, Issue FEB, 2015, Pages 1-8

  Aller, Máté ^a   Giani, Anette ^b   Conrad, Verena ^b   Watanabe, Masataka ^b   Noppeney, Uta ^a,b  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

Attention; Audiovisual; Awareness; Consciousness; Multisensory integration; Perception; Perceptual illusion; Ventriloquism

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; AWARENESS; BRAIN FUNCTION; CONTINUOUS FLASH SUPPRESSION; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; MULTISENSORY INTEGRATION; NEUROMODULATION; NORMAL HUMAN; PERCEPTIVE DISCRIMINATION; SENSORY SYSTEM PARAMETERS; SOUND; SPATIAL CONGRUENCY; SPATIAL DISCRIMINATION; VISUAL FLASH; VISUAL STIMULATION; VISUAL SYSTEM PARAMETERS;

EID: 84925177573     PISSN: 16625145     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnint.2015.00016     Document Type: Article

References (74)

1. Adam, R., and Noppeney, U. (2014). A phonologically congruent sound boosts a visual target into perceptual awareness. Front. Integr. Neurosci. 8:70. doi: 10.3389/fnint.2014.00070
2. Alais, D., and Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257-262. doi: 10.1016/j.cub.2004.01.029
3. Alais, D., van Boxtel, J. J., Parker, A., and van Ee, R. (2010). Attending to auditory signals slows visual alternations in binocular rivalry. Vision Res. 50, 929-935. doi: 10.1016/j.visres.2010.03.010
4. Alsius, A., and Munhall, K. G. (2013). Detection of audiovisual speech correspondences without visual awareness.Psychol. Sci. 24, 423-431. doi: 10.1177/0956797612457378
5. Alsius, A., Navarra, J., Campbell, R., and Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Curr. Biol. 15, 839-843. doi: 10.1016/j.cub.2005.03.046
6. Arndt, P. A., and Colonius, H. (2003). Two stages in crossmodal saccadic integration: evidence from a visual-auditory focused attention task. Exp. Brain Res. 150, 417-426. doi: 10.1007/s00221-003-1424-6
7. Bertelson, P., Vroomen, J., de Gelder, B., and Driver, J. (2000). The ventriloquist effect does not depend on the direction of deliberate visual attention. Percept. Psychophys. 62, 321-332. doi: 10.3758/bf03205552
8. Bertini, C., Leo, F., and Làdavas, E. (2008). Temporo-nasal asymmetry in multisensory integration mediated by the superior colliculus. Brain Res. 1242, 37-44. doi: 10.1016/j.brainres.2008.03.087
9. Bisley, J. W., and Goldberg, M. E. (2010). Attention, intention and priority in the parietal lobe. Annu. Rev. Neurosci. 33, 1-21. doi: 10.1146/annurev-neuro-060909-152823
10. Bolognini, N., Frassinetti, F., Serino, A., and Làdavas, E. (2005). "Acoustical vision" of below threshold stimuli: interaction among spatially converging audiovisual inputs. Exp. Brain Res. 160, 273-282. doi: 10.1007/s00221-004-2005-z
11. Bonath, B., Noesselt, T., Martinez, A., Mishra, J., Schwiecker, K., Heinze, H.-J., et al. (2007). Neural basis of the ventriloquist illusion. Curr. Biol. 17, 1697-1703. doi: 10.1016/j.cub.2007.08.050
12. Brainard, D. H. (1997). The psychophysics toolbox. Spat. Vis. 10, 433-436. doi: 10.1163/156856897x00357
13. Bresciani, J.-P., Dammeier, F., and Ernst, M. O. (2006). Vision and touch are automatically integrated for the perception of sequences of events. J. Vis. 6, 554-564. doi: 10.1167/6.5.2
14. Chen, Y.-C., and Spence, C. (2011). The crossmodal facilitation of visual object representations by sound: evidence from the backward masking paradigm. J. Exp. Psychol. Hum. Percept. Perform. 37, 1784-1802. doi: 10.1037/a0025638
15. Conrad, V., Bartels, A., Kleiner, M., and Noppeney, U. (2010). Audiovisual interactions in binocular rivalry. J. Vis.10:27. doi: 10.1167/10.10.27
16. Conrad, V., Kleiner, M., Bartels, A., Hartcher O'Brien, J., Bülthoff, H. H., and Noppeney, U. (2013). Naturalistic stimulus structure determines the integration of audiovisual looming signals in binocular rivalry. PLoS One8:e70710. doi: 10.1371/journal.pone.0070710
17. Conrad, V., Vitello, M. P., and Noppeney, U. (2012). Interactions between apparent motion rivalry in vision and touch. Psychol. Sci. 23, 940-948. doi: 10.1163/187847612x646497
18. Deroy, O., Chen, Y., and Spence, C. (2014). Multisensory constraints on awareness. Philos. Trans. R. Soc. Lond. B Biol. Sci. 369:20130207. doi: 10.1098/rstb.2013.0207
19. Diederich, A., Colonius, H., Bockhorst, D., and Tabeling, S. (2003). Visual-tactile spatial interaction in saccade generation. Exp. Brain Res. 148, 328-337. doi: 10.1007/s00221-002-1302-7
20. Driver, J., and Noesselt, T. (2008). Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses and judgments. Neuron 57, 11-23. doi: 10.1016/j.neuron.2007.12.013
21. Efron, B., and Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Boston, MA: Springer US.
22. Ernst, M. O., and Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429-433. doi: 10.1038/415429a
23. Fairhall, S. L., and Macaluso, E. (2009). Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. Eur. J. Neurosci. 29, 1247-1257. doi: 10.1111/j.1460-9568.2009.06688.x
24. Faisal, A. A., Selen, L. P. J., and Wolpert, D. M. (2008). Noise in the nervous system. Nat. Rev. Neurosci. 9, 292-303. doi: 10.1038/nrn2258
25. Fetsch, C. R., DeAngelis, G. C., and Angelaki, D. E. (2013). Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons. Nat. Rev. Neurosci. 14, 429-442. doi: 10.1038/nrn3503
26. Fetsch, C. R., Pouget, A., DeAngelis, G. C., and Angelaki, D. E. (2011). Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 15, 146-154. doi: 10.1038/nn.2983
27. Fogelson, S. V., Kohler, P. J., Miller, K. J., Granger, R., and Tse, P. U. (2014). Unconscious neural processing differs with method used to render stimuli invisible. Front. Psychol. 5:601. doi: 10.3389/fpsyg.2014.00601
28. Forster, B., Cavina-Pratesi, C., Aglioti, S. M., and Berlucchi, G. (2002). Redundant target effect and intersensory facilitation from visual-tactile interactions in simple reaction time. Exp. Brain Res. 143, 480-487. doi: 10.1007/s00221-002-1017-9
29. Frassinetti, F., Bolognini, N., and Làdavas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp. Brain Res. 147, 332-343. doi: 10.1007/s00221-002-1262-y
30. Girard, S., Collignon, O., and Lepore, F. (2011). Multisensory gain within and across hemispaces in simple and choice reaction time paradigms. Exp. Brain Res. 214, 1-8. doi: 10.1007/s00221-010-2515-9
31. Gottlieb, J. P., Kusunoki, M., and Goldberg, M. E. (1998). The representation of visual salience in monkey parietal cortex. Nature 391, 481-484. doi: 10.1038/35135
32. Guzman-Martinez, E., Ortega, L., Grabowecky, M., Mossbridge, J., and Suzuki, S. (2012). Interactive coding of visual spatial frequency and auditory amplitude-modulation rate. Curr. Biol. 22, 383-388. doi: 10.1016/j.cub.2012.01.004
33. Harrington, L. K., and Peck, C. K. (1998). Spatial disparity affects visual-auditory interactions in human sensorimotor processing. Exp. Brain Res. 122, 247-252. doi: 10.1007/s002210050512
34. Hupé, J., Joffo, L., and Pressnitzer, D. (2008). Bistability for audiovisual stimuli: perceptual decision is modality specific. J. Vis. 8, 1.1-1.15. doi: 10.1167/8.7.1
35. Kayser, C., Petkov, C. I., Augath, M., and Logothetis, N. K. (2005). Integration of touch and sound in auditory cortex. Neuron 48, 373-384. doi: 10.1016/j.neuron.2005.09.018
36. Kleiner, M., Brainard, D., and Pelli, D. (2007). "What's new in Psychtoolbox-3?," in Perception (Alezzo), 36 EVCP Abstract Supplement.
37. Klink, P. C., van Wezel, R. J. A., and van Ee, R. (2012). United we sense, divided we fail: context-driven perception of ambiguous visual stimuli. Philos. Trans. R. Soc. Lond. B Biol. Sci. 367, 932-941. doi: 10.1098/rstb.2011.0358
38. Koch, C., and Tsuchiya, N. (2007). Attention and consciousness: two distinct brain processes. Trends Cogn. Sci. 11, 16-22. doi: 10.1016/j.tics.2006.10.012
39. Koch, C., and Tsuchiya, N. (2012). Attention and consciousness: related yet different. Trends Cogn. Sci. 16, 103-105. doi: 10.1016/j.tics.2011.11.012
40. Lee, H., and Noppeney, U. (2011). Physical and perceptual factors shape the neural mechanisms that integrate audiovisual signals in speech comprehension. J. Neurosci. 31, 11338-11350. doi: 10.1523/JNEUROSCI.6510-10.2011
41. Lee, H., and Noppeney, U. (2014). Temporal prediction errors in visual and auditory cortices. Curr. Biol. 24, R309-R310. doi: 10.1016/j.cub.2014.02.007
42. Lewis, R., and Noppeney, U. (2010). Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J. Neurosci. 30, 12329-12339. doi: 10.1523/JNEUROSCI.5745-09.2010
43. Lunghi, C., and Alais, D. (2013). Touch interacts with vision during binocular rivalry with a tight orientation tuning.PLoS One 8:e58754. doi: 10.1371/journal.pone.0058754
44. Lunghi, C., Binda, P., and Morrone, M. C. (2010). Touch disambiguates rivalrous perception at early stages of visual analysis. Curr. Biol. 20, R143-R144. doi: 10.1016/j.cub.2009.12.015
45. Lunghi, C., and Morrone, M. C. (2013). Early interaction between vision and touch during binocular rivalry.Multisens. Res. 26, 291-306. doi: 10.1163/22134808-00002411
46. Lunghi, C., Morrone, M. C., and Alais, D. (2014). Auditory and tactile signals combine to influence vision during binocular rivalry. J. Neurosci. 34, 784-792. doi: 10.1523/JNEUROSCI.2732-13.2014
47. Macaluso, E., and Driver, J. (2005). Multisensory spatial interactions: a window onto functional integration in the human brain. Trends Neurosci. 28, 264-271. doi: 10.1016/j.tins.2005.03.008
48. Macaluso, E., Driver, J., and Frith, C. D. (2003). Multimodal spatial representations engaged in human parietal cortex during both saccadic and manual spatial orienting. Curr. Biol. 13, 990-999. doi: 10.1016/s0960-9822(03)00377-4
49. Maruya, K., Watanabe, H., and Watanabe, M. (2008). Adaptation to invisible motion results in low-level but not high-level aftereffects. J. Vis. 8, 7.1-7.11. doi: 10.1167/8.11.7
50. Morgan, M. L., Deangelis, G. C., and Angelaki, D. E. (2008). Multisensory integration in macaque visual cortex depends on cue reliability. Neuron 59, 662-673. doi: 10.1016/j.neuron.2008.06.024
51. Munhall, K. G., ten Hove, M. W., Brammer, M., and Paré, M. (2009). Audiovisual integration of speech in a bistable illusion. Curr. Biol. 19, 735-739. doi: 10.1016/j.cub.2009.03.019
52. Olivers, C. N. L., and Van der Burg, E. (2008). Bleeping you out of the blink: sound saves vision from oblivion. Brain Res. 1242, 191-199. doi: 10.1016/j.brainres.2008.01.070
53. Onat, S., Libertus, K., and König, P. (2007). Integrating audiovisual information for the control of overt attention.J. Vis. 7, 11.1-11.16. doi: 10.1167/7.10.11
54. Palmer, T. D., and Ramsey, A. K. (2012). The function of consciousness in multisensory integration. Cognition 125, 353-364. doi: 10.1016/j.cognition.2012.08.003
55. Peremen, Z., and Lamy, D. (2014). Comparing unconscious processing during continuous flash suppression and meta-contrast masking just under the limen of consciousness. Front. Psychol. 5:969. doi: 10.3389/fpsyg.2014.00969
56. Santangelo, V., and Spence, C. (2008). Is the exogenous orienting of spatial attention truly automatic? Evidence from unimodal and multisensory studies. Conscious. Cogn. 17, 989-1015. doi: 10.1016/j.concog.2008.02.006
57. Soto-Faraco, S., and Spence, C. (2002). Modality-specific auditory and visual temporal processing deficits. Q. J. Exp. Psychol. A 55, 23-40. doi: 10.1080/02724980143000136
58. Spence, C. (2010). Crossmodal spatial attention. Ann. N Y Acad. Sci. 1191, 182-200. doi: 10.1111/j.1749-6632.2010.05440.x
59. Spence, C. (2013). Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule. Ann. N Y Acad. Sci. 1296, 31-49. doi: 10.1111/nyas.12121
60. Stanford, T. R., Quessy, S., and Stein, B. E. (2005). Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 25, 6499-6508. doi: 10.1523/jneurosci.5095-04.2005
61. Stein, B. E., and Meredith, M. A. (1993). The Merging of the Senses. (Cambridge, MA: The MIT Press), 211.
62. Stekelenburg, J. J., Vroomen, J., and de Gelder, B. (2004). Illusory sound shifts induced by the ventriloquist illusion evoke the mismatch negativity. Neurosci. Lett. 357, 163-166. doi: 10.1016/j.neulet.2003.12.085
63. Talsma, D., Doty, T. J., and Woldorff, M. G. (2007). Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? Cereb. Cortex 17, 679-690. doi: 10.1093/cercor/bhk016
64. Talsma, D., Senkowski, D., Soto-Faraco, S., and Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends Cogn. Sci. 14, 400-410. doi: 10.1016/j.tics.2010.06.008
65. Tong, F. (2003). Primary visual cortex and visual awareness. Nat. Rev. Neurosci. 4, 219-229. doi: 10.1038/nrn1055
66. Tsuchiya, N., and Koch, C. (2005). Continuous flash suppression reduces negative afterimages. Nat. Neurosci. 8, 1096-1101. doi: 10.1038/nn1500
67. van Ee, R., van Boxtel, J. J. A., Parker, A. L., and Alais, D. (2009). Multisensory congruency as a mechanism for attentional control over perceptual selection. J. Neurosci. 29, 11641-11649. doi: 10.1523/JNEUROSCI.0873-09.2009
68. Vroomen, J., Bertelson, P., and de Gelder, B. (2001). The ventriloquist effect does not depend on the direction of automatic visual attention. Percept. Psychophys. 63, 651-659. doi: 10.3758/bf03194427
69. Wallace, M. T., Roberson, G. E., Hairston, W. D., Stein, B. E., Vaughan, J. W., and Schirillo, J. A. (2004). Unifying multisensory signals across time and space. Exp. Brain Res. 158, 252-258. doi: 10.1007/s00221-004-1899-9
70. Watanabe, M., Cheng, K., Murayama, Y., Ueno, K., Asamizuya, T., Tanaka, K., et al. (2011). Attention but not awareness modulates the BOLD signal in the human V1 during binocular suppression. Science 334, 829-831. doi: 10.1126/science.1203161
71. Werner, S., and Noppeney, U. (2010). Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J. Neurosci. 30, 2662-2675. doi: 10.1523/JNEUROSCI.5091-09.2010
72. Wyart, V., and Tallon-Baudry, C. (2008). Neural dissociation between visual awareness and spatial attention. J. Neurosci. 28, 2667-2679. doi: 10.1523/JNEUROSCI.4748-07.2008
73. Yuval-Greenberg, S., and Heeger, D. J. (2013). Continuous flash suppression modulates cortical activity in early visual cortex. J. Neurosci. 33, 9635-9643. doi: 10.1523/JNEUROSCI.4612-12.2013
74. Zhou, W., Jiang, Y., He, S., and Chen, D. (2010). Olfaction modulates visual perception in binocular rivalry. Curr. Biol. 20, 1356-1358. doi: 10.1016/j.cub.2010.05.059