Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search?

Attention, Perception, and Psychophysics

Volumn 74, Issue 6, 2012, Pages 1154-1167

  Klapetek, Anna ^a,b   Ngo, Mary Kim ^b   Spence, Charles ^b  

^a UNIVERSITY OF MUNICH   (Germany)

^b UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Brightness; Color; Lightness; Multisensory processing; Visual search

Indexed keywords

ADULT; ARTICLE; ASSOCIATION; ATTENTION; CHILD; COLOR VISION; CONTRAST SENSITIVITY; FEMALE; HEARING; HUMAN; INFANT; MALE; MIDDLE AGED; ORIENTATION; PATTERN RECOGNITION; PERCEPTIVE DISCRIMINATION; PSYCHOPHYSICS; REACTION TIME;

ADULT; ATTENTION; CHILD; COLOR PERCEPTION; CONTRAST SENSITIVITY; CUES; DISCRIMINATION (PSYCHOLOGY); FEMALE; HUMANS; INFANT; MALE; MIDDLE AGED; ORIENTATION; PATTERN RECOGNITION, VISUAL; PITCH PERCEPTION; PSYCHOPHYSICS; REACTION TIME; YOUNG ADULT;

EID: 84864555897     PISSN: 19433921     EISSN: 1943393X     Source Type: Journal    
DOI: 10.3758/s13414-012-0317-9     Document Type: Article

References (60)

1. Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under attention demands. Current Biology, 15, 839-843.
2. Baron-Cohen, S. (1996). Is there a normal phase of synaesthesia in development? Psyche, 2, retrieved on March 14, 2012, from http://daysyn. com/BaronCohen1996. pdf.
3. Ben-Artzi, E., & Marks, L. E. (1995). Visual-auditory interaction in speeded classification: Role of stimulus difference. Perception & Psychophysics, 57, 1151-1162. doi: 10. 3758/BF03208371.
4. Bernstein, I. H., & Edelstein, B. A. (1971). Effects of some variations in auditory input upon visual choice reaction time. Journal of Experimental Psychology, 87, 241-247.
5. Bien, N., ten Oever, S., Goebel, R., & Sack, A. T. (2012). The sound of size: Crossmodal binding in pitch-size synesthesia: A combined TMS, EEG and psychophysics study. NeuroImage, 59, 663-672. doi: 10. 1016/j. neuroimage. 2011. 06. 095.
6. Calvert, G., & Thesen, T. (2004). Multisensory integration: Methodological approach and emerging principles in the human brain. The Journal of Physiology, 98, 191-205.
7. Chiou, R., & Rich, A. N. (2012). Cross-modality correspondence between pitch and spatial location modulates attentional orienting. Perception, 41, 339-353.
8. Clark, H. H., & Brownell, H. H. (1976). Position, direction, and their perceptual integrality. Perception & Psychophysics, 19, 328-334. doi: 10. 3758/BF03204238.
9. Cytowic, R. E. (1989). Synesthesia: A union of the senses. New York, NY: Springer.
10. De Houwer, J. (2003). A structural analysis of indirect measures of attitudes. In J. Musch & K. C. Klauer (Eds.), The psychology of evaluation: Affective processes in cognition and emotion (pp. 219-244). Mahwah, NJ: Erlbaum.
11. Diaconescu, A. O., Alain, C., & McIntosh, A. R. (2011). The co-occurrence of multisensory facilitation and cross-modal conflict in the human brain. Journal of Neurophysiology, 106, 2896-2909.
12. Diederich, A., & Colonius, H. (2008). Crossmodal interaction in saccadic reaction time: Separating multisensory from warning effects in the time window of integration model. Experimental Brain Research, 186, 1-22.
13. Ernst, M. O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7(5), 1-14. doi: 10. 1167/7. 5. 7.
14. Evans, K. K., & Treisman, A. (2010). Natural cross-modal mappings between visual and auditory features. Journal of Vision, 10(1), 1-12. doi: 10. 1167/10. 1. 6.
15. Fairhall, S. L., & Macaluso, E. (2009). Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. European Journal of Neuroscience, 29, 1247-1257. doi: 10. 1111/j. 1460-9568. 2009. 06688. x.
16. Gallace, A., & Spence, C. (2006). Multisensory synesthetic interactions in the speeded classification of visual size. Perception & Psychophysics, 68, 1191-1203. doi: 10. 3758/BF03193720.
17. Garner, W. R. (1974). The processing of information and structure. Potomac, MD: Erlbaum.
18. Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464-1480.
19. Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and Using the Implicit Association Test: I. An Improved Scoring Algorithm. Journal of Personality and Social Psychology, 85, 197-216.
20. Grossenbacher, P. G., & Lovelace, C. T. (2001). Mechanisms of synesthesia: Cognitive and physiological constraints. Trends in Cognitive Sciences, 5, 36-41. doi: 10. 1016/S1364-6613(00)01571-0.
21. Hubbard, T. L. (1996). Synesthesia-like mappings of lightness, pitch, and melodic interval. The American Journal of Psychology, 109, 219-238.
22. Jonides, J., & Mack, R. (1984). On the cost and benefit of cost and benefit. Psychological Bulletin, 96, 29-44. doi: 10. 1037/0033-2909. 96. 1. 29.
23. Kaptein, N. A., Theeuwes, J., & van der Heijden, A. H. C. (1995). Search for a conjunctively defined target can be selectively limited to a color-defined subset of elements. Journal of Experimental Psychology. Human Perception and Performance, 21, 1053-1069.
24. Kunkler-Peck, A. J., & Turvey, M. T. (2000). Hearing shape. Journal of Experimental Psychology. Human Perception and Performance, 26, 279-294.
25. Lakatos, S., McAdams, S., & Caussé, R. (1997). The representation of auditory source characteristics: Simple geometric form. Perception & Psychophysics, 59, 1180-1190. doi: 10. 3758/BF03214206.
26. Lippert, M., Logothetis, N. K., & Kayser, C. (2007). Improvement of visual contrast detection by a simultaneous sound. Brain Research, 1173, 102-109.
27. Ludwig, V. U., Adachi, I., & Matzuzawa, T. (2011). Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humans. Proceedings of the National Academy of Sciences, 108, 20661-20665.
28. Marks, L. E. (1974). On associations of light and sound: The mediation of brightness, pitch and loudness. The American Journal of Psychology, 87, 173-188.
29. Marks, L. E. (1987). On cross-modal similarity: Auditory-visual interactions in speeded discrimination. Journal of Experimental Psychology. Human Perception and Performance, 13, 384-394. doi: 10. 1037/0096-1523. 13. 3. 384.
30. Martino, G., & Marks, L. E. (1999). Perceptual and linguistic interactions in speeded classification: Tests of the semantic coding hypothesis. Perception, 28, 903-923.
31. Maurer, D., & Mondloch, C. (2005). Neonatal synesthesia: A re-evaluation. In L. Robertson & N. Sagiv (Eds.), Synesthesia: Perspectives from cognitive neuroscience (pp. 193-213). New York, NY: Oxford University Press.
32. Melara, R. D. (1989). Dimensional interactions between color and pitch. Journal of Experimental Psychology. Human Perception and Performance, 15, 69-79.
33. Melara, R. D., & Marks, L. E. (1990). Processes underlying dimensional interactions: Correspondences between linguistic and nonlinguistic dimensions. Memory & Cognition, 18, 477-495. doi: 10. 3758/BF03198481.
34. Melara, R. D., & O'Brien, T. P. (1987). Interaction between synesthetically corresponding dimensions. Journal of Experimental Psychology. General, 116, 323-336.
35. Meredith, M. A., & Stein, B. E. (1986). Visual, auditory, and somatosensory convergence on cells in superior colliculus result in multisensory integration. Journal of Neurophysiology, 56, 640-662.
36. Miller, J. (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14, 247-279. doi: 10. 1016/0010-0285(82)90010-X.
37. Miller, J. (1991). Channel interaction and the redundant-targets effect in bimodal divided attention. Journal of Experimental Psychology. Human Perception and Performance, 17, 160-169. doi: 10. 1037/0096-1523. 17. 1. 160.
38. Mondloch, C. J., & Maurer, D. (2004). Do small white balls squeak? Pitch-object correspondences in young children. Cognitive, Affective, & Behavioral Neuroscience, 4, 133-136. doi: 10. 3758/CABN. 4. 2. 133.
39. Murray, E. A., & Mishkin, M. (1985). Amygdalectomy impairs crossmodal association in monkeys. Science, 228, 604-606.
40. Ngo, M. K., & Spence, C. (2010). Auditory, tactile, and multisensory cues facilitate search for dynamic visual stimuli. Attention, Perception, & Psychophysics, 72, 1654-1665. doi: 10. 3758/APP. 72. 6. 1654.
41. Parise, C., & Spence, C. (2008). Synaesthetic congruency modulates the temporal ventriloquism effect. Neuroscience Letters, 442, 257-261.
42. Parise, C. V., & Spence, C. (2009). "When birds of a feather flock together": Synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS One, 4, e5664.
43. Patching, G. R., & Quinlan, P. T. (2002). Garner and congruence effects in the speeded classification of bimodal signals. Journal of Experimental Psychology. Human Perception and Performance, 28, 755-775.
44. Pratt, C. C. (1930). The spatial character of high and low tones. Journal of Experimental Psychology, 13, 278-285.
45. Ramachandran, V. S., & Hubbard, E. M. (2001). Synaesthesia: A window into perception, thought and language. Journal of Consciousness Studies, 8, 30-34.
46. Roffler, S. K., & Butler, R. A. (1968). Localization of tonal stimuli in the vertical plane. Journal of the Acoustical Society of America, 43, 1260-1266.
47. Rusconi, E., Kwan, B., Giordano, B., Umiltà, C., & Butterworth, B. (2006). Spatial representation of pitch height: The SMARC effect. Cognition, 99, 113-129.
48. Sadaghiani, S., Maier, J. X., & Noppeney, U. (2009). Natural, metaphoric, and linguistic auditory direction signals have distinct influences on visual motion processing. Journal of Neuroscience, 29, 6490-6499.
49. Spence, C. (2011). Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73, 971-995. doi: 10. 3758/s13414-010-0073-7.
50. Spence, C. (2012). Multisensory perception, cognition, and behavior: Evaluating the factors modulating multisensory integration. In B. E. Stein (Ed.), The new handbook of multisensory processing (pp. 241-264). Cambridge, MA: MIT Press.
51. Spence, C., & Ngo, M. (2012). Does crossmodal attention or multisensory integration explain the crossmodal facilitation of masked visual target identification in the freezing effect? In B. E. Stein (Ed.), The new handbook of multisensory processing (pp. 345-358). Cambridge, MA: MIT Press.
52. Spence, C., Ngo, M., Lee, J.-H., & Tan, H. (2010). Solving the correspondence problem in haptic/multisensory interface design. In M. H. Zadeh (Ed.), Advances in haptics (pp. 47-74). Vukovar, Croatia: In-Tech Publishers. Retrieved on March 24, 2012, from www. sciyo. com/articles/show/title/solving-the-correspondence-problem-in-haptic-multisensory-interface-design.
53. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press.
54. Stevens, S. S. (1957). On the psychophysical law. Psychological Review, 64, 153-181. doi: 10. 1037/h0046162.
55. Talsma, D., Doty, T. J., & Woldorff, M. G. (2007). Selective attention and audiovisual integration: Is attending to both modalities a prerequisite for early integration? Cerebral Cortex, 17, 679-690. doi: 10. 1093/cercor/bhk016.
56. Trimble, O. C. (1934). Localization of sound in the anterior posterior and vertical dimensions of auditory space. British Journal of Psychology, 24, 320-334.
57. van der Burg, E., Cass, J., Olivers, C. N. L., Theeuwes, J., & Alais, D. (2010). Efficient visual search from synchronized auditory signals requires transient audiovisual events. PLoS One, 5, e10664.
58. van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology. Human Perception and Performance, 34, 1053-1065. doi: 10. 1037/0096-1523. 34. 5. 1053.
59. van der Burg, E., Talsma, D., Olivers, C. N. L., Hickey, C., & Theeuwes, J. (2011). Early multisensory interactions affect the competition among multiple visual objects. NeuroImage, 55, 1208-1218.
60. Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989). Guided search: An alternative to the feature integration model for visual search. Journal of Experimental Psychology. Human Perception and Performance, 15, 419-433. doi: 10. 1037/0096-1523. 15. 3. 419.