Multisensory temporal integration: Task and stimulus dependencies

Experimental Brain Research

Volumn 227, Issue 2, 2013, Pages 249-261

  Stevenson, Ryan A ^a,b   Wallace, Mark T ^a,b,c  

^a VANDERBILT UNIVERSITY MEDICAL CENTER   (United States)

^b VANDERBILT UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c VANDERBILT UNIVERSITY   (United States)

Author keywords

Audiovisual; Multimodal; Multisensory integration; Psychophysics; Speech perception; Synchrony; Temporal binding window; Temporal perception

Indexed keywords

ADULT; ARTICLE; BRAIN DEPTH STIMULATION; DYSLEXIA; FEMALE; HUMAN; MALE; PRIORITY JOURNAL; SENSORY CORTEX; TASK PERFORMANCE; TEMPORAL LOBE;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; ANALYSIS OF VARIANCE; AUDITORY PERCEPTION; FEMALE; FOLLOW-UP STUDIES; FUNCTIONAL LATERALITY; HUMANS; MALE; PHOTIC STIMULATION; REACTION TIME; SENSORY THRESHOLDS; STATISTICS AS TOPIC; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84878011601     PISSN: 00144819     EISSN: 14321106     Source Type: Journal    
DOI: 10.1007/s00221-013-3507-3     Document Type: Article

References (54)

1. Beauchamp MS, Nath AR, Pasalar S (2010) fMRI-Guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J Neurosci 30:2414-2417. doi: 10.1523/JNEUROSCI.4865-09.2010
2. Boenke LT, Deliano M, Ohl FW (2009) Stimulus duration influences perceived simultaneity in audiovisual temporal-order judgment. Exp Brain Res 198:233-244. doi: 10.1007/s00221-009-1917-z
3. Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433-436
4. Conrey B, Pisoni DB (2006) Auditory-visual speech perception and synchrony detection for speech and nonspeech signals. J Acoust Soc Am 119:4065-4073
5. de Boer-Schellekens L, Eussen M, Vroomen J (2013) Diminished sensitivity of audiovisual temporal order in autism spectrum disorder. Front Integr Neurosci 7:8. doi: 10.3389/fnint.2013.00008
6. Diederich A, Colonius H (2004) Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66:1388-1404
7. Dixon NF, Spitz L (1980) The detection of auditory visual desynchrony. Perception 9:719-721
8. Foss-Feig JH, Kwakye LD, Cascio CJ, Burnette CP, Kadivar H, Stone WL, Wallace MT (2009) An extended multisensory temporal binding window in autism spectrum disorders. Exp Brain Res. doi: 10.1007/s00221-010-2240-4
9. Hairston WD, Burdette JH, Flowers DL, Wood FB, Wallace MT (2005) Altered temporal profile of visual-auditory multisensory interactions in dyslexia. Exp Brain Res 166:474-480. doi: 10.1007/s00221-005-2387-6
10. Hershenson M (1962) Reaction time as a measure of intersensory facilitation. J Exp Psychol 63:289-293
11. Hillock AR, Powers AR, Wallace MT (2011) Binding of sights and sounds: age-related changes in multisensory temporal processing. Neuropsychologia 49:461-467. doi: 10.1016/j.neuropsychologia.2010.11.041
12. Hillock-Dunn A, Wallace MT (2012) Developmental changes in the multisensory temporal binding window persist into adolescence. Dev Sci 15:688-696. doi: 10.1111/j.1467-7687.2012.01171.x
13. Hirsh IJ, Sherrick CE Jr (1961) Perceived order in different sense modalities. J Exp Psychol 62:423-432
14. James TW, Stevenson RA, Kim S (2009) Assessing multisensory integration with additive factors and functional MRI. The International Society for Psychophysics, Dublin
15. James TW, VanDerKlok RM, Stevenson RA, James KH (2011) Multisensory perception of action in posterior temporal and parietal cortices. Neuropsychologia 49:108-114. doi: 10.1016/j.neuropsychologia.2010.10.030
16. James TW, Stevenson RA, Kim S (2012) Inverse effectiveness in multisensory processing. In: Stein BE (ed) The new handbook of multisensory processes. MIT Press, Cambridge, MA
17. Keetels M, Vroomen J (2005) The role of spatial disparity and hemifields in audio-visual temporal order judgments. Exp Brain Res 167:635-640. doi: 10.1007/s00221-005-0067-1
18. Kuling IA, van Eijk RL, Juola JF, Kohlrausch A (2012) Effects of stimulus duration on audio-visual synchrony perception. Exp Brain Res 221:403-412. doi: 10.1007/s00221-012-3182-9
19. Kwakye LD, Foss-Feig JH, Cascio CJ, Stone WL, Wallace MT (2011) Altered auditory and multisensory temporal processing in autism spectrum disorders. Front Integr Neurosci 4:129. doi: 10.3389/fnint.2010.00129
20. Lovelace CT, Stein BE, Wallace MT (2003) An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Brain Res Cogn Brain Res 17:447-453. doi: 10.1016/S0926-6410(03)00160-5
21. Macaluso E, George N, Dolan R, Spence C, Driver J (2004) Spatial and temporal factors during processing of audiovisual speech: a PET study. Neuroimage 21:725-732
22. Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215-3229
23. Meredith MA, Wallace MT, Stein BE (1992) Visual, auditory and somatosensory convergence in output neurons of the cat superior colliculus: multisensory properties of the tecto-reticulo-spinal projection. Exp Brain Res 88:181-186
24. Miller LM, D'Esposito M (2005) Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J Neurosci 25:5884-5893
25. Nath AR, Beauchamp MS (2011) A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. Neuroimage. doi: 10.1016/j.neuroimage.2011.07.024
26. Nelson WT, Hettinger LJ, Cunningham JA, Brickman BJ, Haas MW, McKinley RL (1998) Effects of localized auditory information on visual target detection performance using a helmet-mounted display. Hum Factors 40:452-460
27. Pelli DG (1997) The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat Vis 10:437-442
28. Pelphrey KA, Carter EJ (2008) Charting the typical and atypical development of the social brain. Dev Psychopathol 20:1081-1102. doi: 10.1017/S0954579408000515
29. Pöppel E, Schill K, von Steinbüchel N (1990) Sensory integration within temporally neutral systems states: a hypothesis. Naturwissenschaften 77:89-91
30. Powers AR 3rd, Hillock AR, Wallace MT (2009) Perceptual training narrows the temporal window of multisensory binding. J Neurosci 29:12265-12274. doi: 10.1523/JNEUROSCI.3501-09.2009
31. Quinto L, Thompson WF, Russo FA, Trehub SE (2010) A comparison of the McGurk effect for spoken and sung syllables. Atten Percept Psychophys 72:1450-1454. doi: 10.3758/APP.72.6.1450
32. Royal DW, Carriere BN, Wallace MT (2009) Spatiotemporal architecture of cortical receptive fields and its impact on multisensory interactions. Exp Brain Res 198:127-136. doi: 10.1007/s00221-009-1772-y
33. Schall JD, Hanes DP (1993) Neural basis of saccade target selection in frontal eye field during visual search. Nature 366:467-469. doi: 10.1038/366467a0
34. Schall S, Quigley C, Onat S, Konig P (2009) Visual stimulus locking of EEG is modulated by temporal congruency of auditory stimuli. Exp Brain Res 198:137-151. doi: 10.1007/s00221-009-1867-5
35. Senkowski D, Talsma D, Grigutsch M, Herrmann CS, Woldorff MG (2007) Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia 45:561-571
36. Stein BE, Wallace MT (1996) Comparisons of cross-modality integration in midbrain and cortex. Prog Brain Res 112:289-299
37. Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44:1210-1223. doi: 10.1016/j.neuroimage.2008.09. 034
38. Stevenson RA, Geoghegan ML, James TW (2007) Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects. Exp Brain Res 179:85-95
39. Stevenson RA, Kim S, James TW (2009) An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI. Exp Brain Res 198:183-194. doi: 10.1007/s00221-009-1783-8
40. Stevenson RA, Altieri NA, Kim S, Pisoni DB, James TW (2010) Neural processing of asynchronous audiovisual speech perception. Neuroimage 49:3308-3318. doi: 10.1016/j.neuroimage.2009.12.001
41. Stevenson RA, VanDerKlok RM, Pisoni DB, James TW (2011) Discrete neural substrates underlie complementary audiovisual speech integration processes. Neuroimage 55:1339-1345. doi: 10.1016/j.neuroimage.2010.12.063
42. Stevenson RA, Wilson MM, Powers AR, Wallace MT (2013) The effects of visual training on multisensory temporal processing. Exp Brain Res. doi: 10.1007/s00221-012-3387-y
43. van Atteveldt NM, Formisano E, Blomert L, Goebel R (2007) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962-974. doi: 10.1093/cercor/bhl007
44. van Eijk RL, Kohlrausch A, Juola JF, van de Par S (2008) Audiovisual synchrony and temporal order judgments: effects of experimental method and stimulus type. Percept Psychophys 70:955-968
45. van Wassenhove V, Grant KW, Poeppel D (2007) Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45:598-607. doi: 10.1016/j.neuropsychologia.2006.01.001
46. Vatakis A, Spence C (2006) Audiovisual synchrony perception for music, speech, and object actions. Brain Res 1111:134-142. doi: 10.1016/j.brainres. 2006.05.078
47. Vatakis A, Navarra J, Soto-Faraco S, Spence C (2008) Audiovisual temporal adaptation of speech: temporal order versus simultaneity judgments. Exp Brain Res 185:521-529. doi: 10.1007/s00221-007-1168-9
48. Vroomen J, Keetels M (2010) Perception of intersensory synchrony: a tutorial review. Atten Percept Psychophys 72:871-884. doi: 10.3758/APP.72.4.871
49. Wallace MH, Murray MM (eds) (2011) Frontiers in the neural basis of multisensory processes. Taylor & Francis, London
50. Wallace MT, Roberson GE, Hairston WD, Stein BE, Vaughan JW, Schirillo JA (2004) Unifying multisensory signals across time and space. Exp Brain Res 158:252-258. doi: 10.1007/s00221-004-1899-9
51. Wilkinson LK, Meredith MA, Stein BE (1996) The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp Brain Res 112:1-10
52. Woynaroski TG, Kwakye LD, Foss-Feig JH, Stevenson RA, Stone WL, Wallace MT (2013) Multisensory speech perception in high-functioning children with autism spectrum disorders. J Autism Dev Disord. doi: JADD-D-12-00456R1
53. Zampini M, Guest S, Shore DI, Spence C (2005) Audio-visual simultaneity judgments. Percept Psychophys 67:531-544
54. Zilbovicius M, Meresse I, Chabane N, Brunelle F, Samson Y, Boddaert N (2006) Autism, the superior temporal sulcus and social perception. Trends Neurosci 29:359-366. doi: 10.1016/j.tins.2006.06.004