Cross-modal associations between vision, touch, and audition influence visual search through top-down attention, not bottom-up capture

Attention, Perception, and Psychophysics

Volumn 75, Issue 8, 2013, Pages 1892-1905

  Orchard Mills, Emily ^a   Alais, David ^a   Van der Burg, Erik ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Attention; Cross modal; Visual search; Visual selection

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; ASSOCIATION; ATTENTION; FEMALE; HEARING; HUMAN; MALE; PATTERN RECOGNITION; PHYSIOLOGY; TOUCH; VISION; YOUNG ADULT;

ADOLESCENT; ADULT; ATTENTION; AUDITORY PERCEPTION; CUES; FEMALE; HUMANS; MALE; PATTERN RECOGNITION, VISUAL; TOUCH; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84887570704     PISSN: 19433921     EISSN: 1943393X     Source Type: Journal    
DOI: 10.3758/s13414-013-0535-9     Document Type: Article

References (62)

1. Alais, D., Newell, F. N., & Mamassian, P. (2010). Multisensory processing in review: from physiology to behaviour. Seeing and perceiving, 23(1), 3-38. doi: 10. 1163/187847510X488603.
2. Alsius, A., Navarra, J., & Soto-Faraco, S. (2007). Attention to touch weakens audiovisual speech integration. Experimental Brain Research, 183(3), 399-404. doi: 10. 1007/s00221-007-1110-1.
3. Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15(9), 839-843. doi: 10. 1016/j. cub. 2005. 03. 046.
4. Amedi, A., Jacobson, G., Hendler, T., Malach, R., & Zohary, E. (2002). Convergence of visual and tactile shape processing in the human lateral occipital complex. Cerebral Cortex, 12(11), 1202-1212.
5. Amedi, A., Raz, N., Azulay, H., Malach, R., & Zohary, E. (2010). Cortical activity during tactile exploration of objects in blind and sighted humans. Restorative Neurology and Neuroscience, 28(2), 143-156. doi: 10. 3233/RNN-2010-0497.
6. Bauer, B., Jolicoeur, P., & Cowan, W. B. (1996). Visual search for colour targets that are or are not linearly separable from distractors. Vision research, 36(10), 1439-1466. doi: 10. 1016/0042-6989(95)00207-3.
7. Becker, S. I. (2010). The role of target-distractor relationships in guiding attention and the eyes in visual search. Journal of Experimental Psychology: General, 139(2), 247-265. doi: 10. 1037/a0018808.
8. Bolia, R. S., D'Angelo, W. R., & McKinley, R. L. (1999). Aurally aided visual search in three-dimensional space. Human Factors, 41(4), 664-669.
9. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10(4), 433-436. doi: 10. 1163/156856897X00357.
10. Bronkhorst, A. W. (2000). The cocktail party phenomenon: A review of research on speech intelligibility in multiple-talker conditions. Acta Acustica United with Acustica, 86(1), 117-128.
11. Cherry, E. C. (1953). Some Experiments on the Recognition of Speech, with One and with Two Ears. The Journal of the Acoustical Society of America, 25(5), 975-979. doi: 10. 1121/1. 1907229.
12. Cohen, L. G., Celnik, P., Pascual-Leone, A., Corwell, B., Falz, L., Dambrosia, J., Hallett, M. (1997). Functional relevance of cross-modal plasticity in blind humans. Nature, 389(6647), 180-183. doi: 10. 1038/38278.
13. Deibert, E., Kraut, M., Kremen, S., & Hart, J. (1999). Neural pathways in tactile object recognition. Neurology, 52(7), 1413-1417.
14. D'Zmura, M. (1991). Color in visual search. Vision research, 31(6), 951-966.
15. Fujisaki, W., Koene, A., Arnold, D., Johnston, A., & Nishida, S. (2006). Visual search for a target changing in synchrony with an auditory signal. Proc Biol Sci, 273(1588), 865-874. doi: 10. 1098/rspb. 2005. 3327.
16. Guzman-Martinez, E., Ortega, L., Grabowecky, M., Mossbridge, J., & Suzuki, S. (2012). Interactive coding of visual spatial frequency and auditory amplitude-modulation rate. Current Biology: CB, 22(5), 383-388. doi: 10. 1016/j. cub. 2012. 01. 004.
17. Iordanescu, L., Grabowecky, M., & Suzuki, S. (2011). Object-based auditory facilitation of visual search for pictures and words with frequent and rare targets. Acta Psychologica, 137(2), 252-259. doi: 10. 1016/j. actpsy. 2010. 07. 017.
18. Iordanescu, L., Grabowecky, M., Franconeri, S., Theeuwes, J., & Suzuki, S. (2010). Characteristic sounds make you look at target objects more quickly. Attention, Perception, & Psychophysics, 72(7), 1736-1741. doi: 10. 3758/APP. 72. 7. 1736.
19. Iordanescu, L., Guzman-Martinez, E., Grabowecky, M., & Suzuki, S. (2008). Characteristic sounds facilitate visual search. Psychonomic Bulletin & Review, 15(3), 548-554.
20. James, T. W., Humphrey, G. K., Gati, J. S., Servos, P., Menon, R. S., & Goodale, M. A. (2002). Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia, 40(10), 1706-1714. doi: 10. 1016/S0028-3932(02)00017-9.
21. Jones, C. M., Gray, R., Spence, C., & Tan, H. Z. (2008). Directing visual attention with spatially informative and spatially noninformative tactile cues. Experimental Brain Research, 186(4), 659-669. doi: 10. 1007/s00221-008-1277-0.
22. Loomis, J. M. (1979). An investigation of tactile hyperacuity. Sensory Processes, 3(4), 289-302.
23. Los, S. A., & van der Burg, E. (2013). Sound speeds vision through preparation, not integration. Journal of Experimental Psychology. Human Perception and Performance. doi: 10. 1037/a0032183.
24. Lunghi, C., Binda, P., & Morrone, M. C. (2010). Touch disambiguates rivalrous perception at early stages of visual analysis. Current Biology: CB, 20(4), R143-R144. doi: 10. 1016/j. cub. 2009. 12. 015.
25. Lunghi, C., & Alais, D. (2013). Touch interacts with vision during binocular rivalry with a tight orientation tuning. PLoS ONE., 8(3), e58754.
26. McCarley, J. S., Kramer, A. F., Wickens, C. D., Vidoni, E. D., & Boot, W. R. (2004). Visual skills in airport-security screening. Psychological Science, 15(5), 302-306. doi: 10. 1111/j. 0956-7976. 2004. 00673. x.
27. Merabet, L. B., Hamilton, R., Schlaug, G., Swisher, J. D., Kiriakopoulos, E. T., Pitskel, N. B., Pascual-Leone, A. (2008). Rapid and reversible recruitment of early visual cortex for touch. PLoS One, 3(8), e3046. doi: 10. 1371/journal. pone. 0003046.
28. Murray, M. M., Molholm, S., Michel, C. M., Heslenfeld, D. J., Ritter, W., Javitt, D. C., Schroeder, C. E., & Foxe, J. J. (2005). Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cerebral Cortex, 15(7), 963-974.
29. Ngo, M. K., & Spence, C. (2010). Auditory, tactile, and multisensory cues facilitate search for dynamic visual stimuli. Attention, Perception, & Psychophysics, 72(6), 1654-1665. doi: 10. 3758/APP. 72. 6. 1654.
30. Nodine, C. F., & Kundel, H. L. (1987). Using eye movements to study visual search and to improve tumor detection. Radiographics: a Review Publication of the Radiological Society of North America, Inc, 7(6), 1241-1250.
31. Olivers, C. N. L., & van der Burg, E. (2008). Bleeping you out of the blink: Sound saves vision from oblivion. Brain Research, 1242, 191-199. doi: 10. 1016/j. brainres. 2008. 01. 070.
32. Orchard-Mills, E., van der Burg, E., & Alais, D. (2013). Amplitude-modulated auditory stimuli influence selection of visual spatial frequencies. Journal of Vision, 13(3), 6. doi: 10. 1167/13. 3. 6. 1-17.
33. Pascual-Leone, A., Amedi, A., Fregni, F., & Merabet, L. B. (2005). The plastic human brain cortex. Annual review of neuroscience, 28, 377-401. doi: 10. 1146/annurev. neuro. 27. 070203. 144216.
34. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10(4), 437-442.
35. Pietrini, P., Furey, M. L., Ricciardi, E., Gobbini, M. I., Wu, W.-H. C., Cohen, L.,. Haxby, J. (2004). Beyond sensory images: Object-based representation in the human ventral pathway. Proc Natl Acad Sci U S A, 101(15), 5658-5663. doi: 10. 1073/pnas. 0400707101.
36. Plaisier, M. A., Bergmann Tiest, W. M., & Kappers, A. M. L. (2008). Haptic pop-out in a hand sweep. Acta Psychologica, 128(2), 368-377. doi: 10. 1016/j. actpsy. 2008. 03. 011.
37. Posner, M. I. (1980). Orienting of attention. The Quarterly Journal of Experimental Psychology, 32(1), 3-25.
38. Rudmann, D. S., & Strybel, T. Z. (1999). Auditory spatial facilitation of visual search performance: Effect of cue precision and distractor density. Human Factors, 41(1), 146-160. doi: 10. 1518/001872099779577354.
39. Sadato, N., Pascual-Leone, A., Grafman, J., Ibañez, V., Deiber, M. P., Dold, G., & Hallett, M. (1996). Activation of the primary visual cortex by Braille reading in blind subjects. Nature, 380(6574), 526-528. doi: 10. 1038/380526a0.
40. Santangelo, V., van der Lubbe, R. H. J., Belardinelli, M. O., & Postma, A. (2006). Spatial attention triggered by unimodal, crossmodal, and bimodal exogenous cues: A comparison of reflexive orienting mechanisms. Experimental Brain Research, 173(1), 40-48.
41. Sathian, K., & Zangaladze, A. (2002). Feeling with the mind's eye: Contribution of visual cortex to tactile perception. Behavioural Brain Research, 135(1-2), 127-132.
42. Sathian, K., Zangaladze, A., Epstein, C. M., & Grafton, S. T. (1999). Involvement of visual cortex in tactile discrimination of orientation. Nature, 401(6753), 587-590. doi: 10. 1038/44139.
43. Shams, L., Kamitani, Y., & Shimojo, S. (2000). Illusions. What you see is what you hear. Nature, 408(6814), 788. doi: 10. 1038/35048669.
44. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA, USA: MIT Press.
45. Talsma, D., Doty, T. J., & Woldorff, M. G. (2007). Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? Cereb Cortex, 17(3), 679-690. doi: 10. 1093/cercor/bhk016.
46. Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14(9), 400-410. doi: 10. 1016/j. tics. 2010. 06. 008.
47. Theeuwes, J. (2010). Top-down and bottom-up control of visual selection. Acta Psychologica, 135(2), 77-99. doi: 10. 1016/j. actpsy. 2010. 02. 006.
48. Theeuwes, J., & van der Burg, E. (2007). The role of spatial and nonspatial information in visual selection. Journal of Experimental Psychology. Human Perception and Performance, 33(6), 1335-1351. doi: 10. 1037/0096-1523. 33. 6. 1335.
49. Theeuwes, J., & van der Burg, E. (2008). The role of cueing in attentional capture. Visual Cognition, 16(2-3), 232-247. doi: 10. 1080/13506280701462525.
50. Theeuwes, J., & van der Burg, E. (2011). On the limits of top-down control of visual selection. Attention, Perception, & Psychophysics, 73(7), 2092-2103. doi: 10. 3758/s13414-011-0176-9.
51. Theeuwes, J., Reimann, B., & Mortier, K. (2006). Visual search for featural singletons: No top-down modulation, only bottom-up priming. Visual Cognition, 14(4-8), 466-489. doi: 10. 1080/13506280500195110.
52. van der Burg, E., Cass, J., Olivers, C. N., Theeuwes, J., & Alais, D. (2010). Efficient visual search from synchronized auditory signals requires transient audiovisual events. PLoS One, 5(5), e10664. doi: 10. 1371/journal. pone. 0010664.
53. van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008a). Audiovisual events capture attention: Evidence from temporal order judgments. Journal of Vision, 8(5), 2. doi: 10. 1167/8. 5. 2. 1-10.
54. van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008b). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology. Human Perception and Performance, 34(5), 1053-1065. doi: 10. 1037/0096-1523. 34. 5. 1053.
55. van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2009). Poke and pop: Tactile-visual synchrony increases visual saliency. Neuroscience Letters, 450(1), 60-64. doi: 10. 1016/j. neulet. 2008. 11. 002.
56. van Polanen, V., Tiest, W. M. B., & Kappers, A. M. L. (2012). Haptic pop-out of movable stimuli. Attention, Perception, & Psychophysics, 74(1), 204-215. doi: 10. 3758/s13414-011-0216-5.
57. Violentyev, A., Shimojo, S., & Shams, L. (2005). Touch-induced visual illusion. Neuroreport, 16(10), 1107.
58. Wang, M.-J. J., Lin, S.-C., & Drury, C. G. (1997). Training for strategy in visual search. International Journal of Industrial Ergonomics, 20(2), 101-108. doi: 10. 1016/S0169-8141(96)00043-1.
59. Wolfe, J. M., & Horowitz, T. S. (2004). What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience, 5(6), 495-501.
60. Wolfe, J. M., Friedman-Hill, S. R., Stewart, M. I., & O'Connell, K. M. (1992). The role of categorization in visual search for orientation. Journal of Experimental Psychology. Human Perception and Performance, 18(1), 34-49. doi: 10. 1037/0096-1523. 18. 1. 34.
61. Wolfe, J. M., Horowitz, T. S., Kenner, N., Hyle, M., & Vasan, N. (2004). How fast can you change your mind? The speed of top-down guidance in visual search. Vision Research, 44(12), 1411-1426. doi: 10. 1016/j. visres. 2003. 11. 024.
62. Zannoli, M., Cass, J., Mamassian, P., & Alais, D. (2012). Synchronized audio-visual transients drive efficient visual search for motion-in-depth. PLoS One, 7(5), e37190. doi: 10. 1371/journal. pone. 0037190.