Audiovisual crossmodal correspondences and sound symbolism: A study using the implicit association test

Experimental Brain Research

Volumn 220, Issue 3-4, 2012, Pages 319-333

  Parise, Cesare V ^a,b,c,d   Spence, Charles ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^c Bernstein Centre for Computational Neuroscience   (Germany)

^d BIELEFELD UNIVERSITY   (Germany)

Author keywords

Audition; Crossmodal correspondences; Implicit association test; Multisensory perception; Sound symbolism; Vision

Indexed keywords

ADULT; ARTICLE; AUDIOVISUAL EQUIPMENT; AUDITORY STIMULATION; AUDITORY SYSTEM EXAMINATION; AUTOMATION; EFFECT SIZE; FEMALE; HUMAN; IMPLICIT ASSOCIATION TEST; MALE; PRIORITY JOURNAL; SCORING SYSTEM; SELECTIVE ATTENTION; SENSORY SYSTEM EXAMINATION; SOUND ANALYSIS; SPEECH; STIMULUS RESPONSE; SYMBOLISM; TASK PERFORMANCE; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; ASSOCIATION; ATTENTION; AUDITORY PERCEPTION; FEMALE; HUMANS; MALE; PHOTIC STIMULATION; REACTION TIME; SYMBOLISM; VISUAL PERCEPTION;

EID: 84864279718     PISSN: 00144819     EISSN: 14321106     Source Type: Journal    
DOI: 10.1007/s00221-012-3140-6     Document Type: Article

References (86)

1. Bernstein IH, Edelstein BA (1971) Effects of some variations in auditory input upon visual choice reaction time. J Exp Psychol 87(2):241-247
2. Bien N, ten Oever S, Goebel R, Sack AT (2012) The sound of size Crossmodal binding in pitch-size synesthesia: A combined TMS, EEG and psychophysics study. Neuroimage 59:663-672
3. Blair IV (2002) The malleability of automatic stereotypes and prejudice. Personal Soc Psychol Rev 6:242-261 (Pubitemid 37195264)
4. Bozzi P, Flores D'Arcais G (1967) Experimental research on the intermodal relationships between expressive qualities. Arch Psicol Neurol Psichiatr 28(5):377-420
5. Brang D, Rouw R, Ramachandran VS, Coulson S (2011) Similarly shaped letters evoke similar colors in grapheme-color synesthesia. Neuropsychologia 49:1355-1358
6. Bremner A, Caparos S, Davidoff J, de Fockert J, Linnell K, Spence C (submitted) Bouba and Kiki in Namibia? Western shapesymbolism does not extend to taste in a remote population. Cognition
7. Chen Y-C, Spence C (2011) Crossmodal semantic priming by naturalistic sounds and spoken words enhances visual sensitivity. J Exp Psychol Hum Percept Perform 37:1554-1568
8. Chiou R, Rich AN (2012) Cross-modality correspondence between pitch and spatial location modulates attentional orienting. Perception 41:339-353
9. Cohen J (1988) Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates, Hillsdale, NJ
10. Cohen Kadosh R, Henik A (2007) Can synaesthesia research inform cognitive science? Trends Cogn Sci 11(4):177-184 (Pubitemid 46467490)
11. Cohen Kadosh R, Cohen Kadosh K, Henik A (2007a) The neuronal correlate of bidirectional synesthesia: a combined event-related potential and functional magnetic resonance imaging study. J Cogn Neurosci 19(12):2050-2059 (Pubitemid 350220515)
12. Cohen Kadosh R, Henik A, Walsh V (2007b) Small is bright and big is dark in synaesthesia. Curr Biol 17(19):R834-R835 (Pubitemid 47503792)
13. Cowey A, Weiskrantz L (1975) Demonstration of cross-modal matching in rhesus monkeys, Macaca mulatta. Neuropsychologia 13(1):117-120
14. Crisinel AS, Spence C (2009) Implicit association between basic tastes and pitch. Neurosci Lett 464(1):39-42
15. Crisinel AS, Spence C (2010) A sweet sound? Food names reveal implicit associations between taste and pitch. Perception 39(3):417-425
16. Davis R (1961) The fitness of names to drawings: a cross-cultural study in Tanganyika. Br J Psychol 52:259-268
17. De Jong R, Liang CC, Lauber E (1994) Conditional andunconditional automaticity: a dual-process model of effects of spatial stimulusresponse correspondence. J Exp Psychol Hum Percept Perform 20(4):731-750
18. Demattè M, Sanabria D, Spence C (2006) Cross-modal associations between odors and colors. Chem Senses 31(6):531-538 (Pubitemid 44282983)
19. Demattè M, Sanabria D, Spence C (2007) Olfactory-tactile compatibility effects demonstrated using a variation of the implicit association test. Acta Psychol 124(3):332-343
20. Dixon MJ, Smilek D, Cudahy C, Merikle PM (2000) Five plus two equals yellow. Nature 406(6794):365
21. Ernst MO (2007) Learning to integrate arbitrary signals from vision and touch. J Vis 7(5):1-14
22. Evans KK, Treisman A (2010) Natural cross-modal mappings between visual and auditory features. J Vis 10(1):1-12
23. Fiedler K, Messner C, Bluemke M (2006) Unresolved problems with the "I", the "A", and the "T": a logical and psychometric critique of the implicit association test (IAT). Eur Rev Soc Psychol 17:74-147
24. Freed DJ (1990) Auditory correlates of perceived mallet hardness for a set of recorded percussive sound events. J Acoust Soc Am 87(1):311-322
25. Gallace A, Spence C (2006) Multisensory synesthetic interactions in the speeded classification of visual size. Percept Psychophys 68(7):1191-1203
26. Greenwald AG, McGhee DE, Schwartz JLK (1998) Measuring individual differences in implicit cognition: the implicit association test. J Pers Soc Psychol 74(6):1464-1480 (Pubitemid 128658112)
27. Grossenbacher PG, Lovelace CT (2001) Mechanisms of synesthesia: cognitive and physiological constraints. Trends Cogn Sci 5(1):36-41 (Pubitemid 32063337)
28. Hinton L, Nichols J, Ohala JJ (eds) (2006) Sound symbolism. Cambridge University Press, Cambridge
29. Johnson A, Jepma M, De Jong R (2007) Colours sometimes count: awareness and bidirectionality in grapheme-colour synaesthesia. Q J Exp Psychol 60(10):1406-1422
30. Klapetek A, Ngo MK, Spence C (in press) Do crossmodal correspondences enhance the facilitatory effect of auditory cues on visual search? Atten Percept Psychophys
31. Klatzky RL, Pai DK, Krotkov EP (2000) Perception of material from contact sounds. Presence Teleoper Virtual Environ 9(4):399-410
32. Klein R, Brennan M, Gilani A (1987) Covert cross-modality orienting of attention in space. Paper presented at the Annual meeting of the Psychonomic Society, Seattle, WA
33. Köhler W (1929) Gestalt psychology. Liveright, New York
34. Köhler W (1947) Gestalt psychology: an introduction to new concepts in modern psychology. Liveright Publ. Corporation, New York, NY
35. Kovic V, Plunkett K, Westermann G (2010) The shape of words in the brain. Cognition 114(1):19-28
36. Ludwig VU, Adachi I, Matzuzawa T (2011) Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humans. Proc Natl Acad Sci USA 108:20661-20665
37. Marks LE (1987) On cross-modal similarity: auditory-visual interactions in speeded discrimination. J Exp Psychol Hum Percept Perform 13(3):384-394
38. Marks LE (1989) On cross-modal similarity: the perceptual structure of pitch, loudness, and brightness. J Exp Psychol Hum Percept Perform 15(3):586-602
39. Marks LE (2004) Cross-modal interactions in speeded classification. In: Calvert GA, Spence C, Stein BE (eds) The handbook of mutisensory processes. MIT Press, Cambridge, MA, pp 85-106
40. Martino G, Marks LE (2001) Synesthesia: strong and weak. Curr Dir Psychol Sci 10(2):61-65
41. Meier B, Rothen N (2007) When conditioned responses "fire back": bidirectional cross-activation creates learning opportunities in synesthesia. Neuroscience 147(3):569-572 (Pubitemid 46990644)
42. Melara RD, O'Brien TP (1987) Interaction between synesthetically corresponding dimensions. J Exp Psychol Gen 116(4):323-336
43. Melara RD, O'Brien TP (1990) Effects of cuing on cross-modal congruity. J Mem Lang 29(6):655-686
44. Mills CB (1999) Digit synaesthesia: a case study using a Stroop-type test. Cogn Neuropsychol 16(2):181-191
45. Mulvenna CM, Walsh V (2006) Synaesthesia: supernormal integration? Trends Cogn Sci 10(8):350-352 (Pubitemid 44093325)
46. Newman S (1933) Further experiments in phonetic symbolism. Am J Psychol 45(1):53-75
47. Oberman LM, Ramachandran VS (2008) Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: the mirror neuron hypothesis. Soc Neurosci 3(3-4):348-355
48. Osgood CE (1960) The cross-cultural generality of visual-verbal synesthetic tendencies. Behav Sci 5(2):146-169
49. Osgood CE, Suci G, Tannenbaum P (1957) The measurement of meaning. University of Illinois Press, Urbana
50. Oyama T, Yamada H, Iwasawa H (1998) Synesthetic tendencies as the basis of sensory symbolism: a review of a series of experiments by means of semantic differential. Psychologia 41:203-215 (Pubitemid 128382062)
51. Parise CV, Pavani F (2011) Evidence of sound symbolism in simple vocalizations. Exp Brain Res 214(3):373-380
52. Parise CV, Spence C (2008) Synesthetic congruency modulates the temporal ventriloquism effect. Neurosci Lett 442(3):257-261
53. Parise CV, Spence C (2009) When birds of a feather flock together: synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS One 4(5):e5664
54. Parise CV, Spence C (2012) Assessing the associations between brand packaging and brand attributes using an indirect performance measure. Food Qual Prefer 24:17-23
55. Parise CV, Spence C (in press) Audiovisual crossmodal correspondences. In Simner J, Hubbard EM (eds) Oxford handbook of synaesthesia. Oxford University Press, Oxford
56. Parise CV, Spence C, Ernst MO (2012) When correlation implies causation in multisensory integration. Curr Biol 22:46-49
57. Parker A, Easton A (2004) Cross-modal memory in primates: the neural basis of learning about the multisensory properties of objects and events. In: Calvert GA, Spence C, Stein BE (eds) The handbook of multisensory processes. MIT Press, Cambridge, MA, pp 333-342
58. Poffenberger A, Barrows B (1924) The feeling value of lines. J Appl Psychol 8(2):187-205
59. Premack D, Premack AJ (2003) Original intelligence: unlocking the mystery of who we are. McGraw-Hill, New York
60. Rader C, Tellegen A (1987) An investigation of synesthesia. J Pers Soc Psychol 52(5):981-987
61. Ramachandran VS, Oberman LM (2007) Broken mirrors: a theory of autism. Sci Am Spec Ed 17(2):20-29
62. Robson D (2011) Language's missing link. New Sci 211(2821):30-33
63. Rogers SK, Ross AS (1968) A cross-cultural test of the Maluma-Takete phenomenon. Perception 4(1):105-106
64. Rousseeuw PJ, Ruts I, Tukey JW (1999) The bagplot: a bivariate boxplot. Am Stat 53(4):382-387
65. Rudmin F, Cappelli M (1983) Tone-taste synesthesia: a replication. Percept Mot Skills 56:118
66. Sapir E (1929) A study in phonetic symbolism. J Exp Psychol 12(3):225-239
67. Seo H-S, Arshamian A, Schemmer K, Scheer I, Sander T, Ritter G, Hummel T (2010) Cross-modal integration between odors and abstract symbols. Neurosci Lett 478:175-178
68. Shepherd GM (2012) Neurogastronomy: how the brain creates flavor and why it matters. Columbia University Press, New York
69. Simpson RH, Quinn M, Ausubel DP (1956) Synesthesia in children: association of colors with pure tone frequencies. J Genet Psychol Res Theory Hum Dev 89(1):95-103
70. Soto-Faraco S, Lyons J, Gazzaniga M, Spence C, Kingstone A (2002) The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. Cogn Brain Res 14(1):139-146 (Pubitemid 34626804)
71. Spence C (2011) Crossmodal correspondences: a tutorial review. Atten Percept Psychophys 73(4):1-25
72. Spence C, Deroy O (2012) Are chimpanzees really synaesthetic? i-Perception 3:316-318
73. Spence C, Squire S (2003) Multisensory integration: maintaining the perception of synchrony. Curr Biol 13(13):R519-R521 (Pubitemid 36815250)
74. Stevens JC, Marks LE (1965) Cross-modality matching of brightness and loudness. Proc Natl Acad Sci USA 54(2):407-411
75. Stumpf K (1883) Tonpsychologie. S. Hirzel, Leipzig
76. Svartdal F, Iversen T (1989) Consistency in synesthetic experience to vowels and consonants: five case studies. Scand J Psychol 30:220-227
77. Vallesi A, Mapelli D, Schiff S, Amodio P, Umiltà C (2005) Horizontal and vertical Simon effect: different underlying mechanisms? Cognition 96(1):B33-B43
78. Van den Doel K, Pai DK (1998) The sounds of physical shapes. Presence 7(4):382-395
79. Walker P, Smith S (1985) Stroop interference based on the multimodal correlates of haptic size and auditory pitch. Perception 14(6):729-736
80. Walker P, Bremner J, Mason U, Spring J, Mattock K, Slater A, Johnson S (2010) Preverbal infants' sensitivity to synaesthetic cross-modality correspondences. Psychol Sci 21(1):21-25
81. Ward J, Huckstep B, Tsakanikos E (2006) Sound-colour synaesthesia: to what extent does it use cross-modal mechanisms common to us all? Cortex 42(2):264-280
82. Watson AB, Pelli DG (1983) QUEST-a Bayesian adaptive psychometric method. Percept Psychophys 33(2):113-120
83. Weiskrantz L, Cowey A (1975) Cross-modal matching in the rhesus monkey using a single pair of stimuli. Neuropsychologia 13(3):257-261
84. Westbury C (2005) Implicit sound symbolism in lexical access: evidence from an interference task. Brain Lang 93(1):10-19 (Pubitemid 40342717)
85. Xu J, Yu L, Rowland BA, Stanford TR, Stein BE (2012) Incorporating cross-modal statistics in the development and maintenance of multisensory integration. J Neurosci 32:2287-2298
86. Zigler MJ (1930) Tone shapes: a novel type of synaesthesia. J Gen Psychol 3:276-287