Integration of auditory and vibrotactile stimuli: Effects of phase and stimulus-onset asynchrony

Journal of the Acoustical Society of America

Volumn 126, Issue 4, 2009, Pages 1960-1974

  Wilson, E Courtenay ^a,b   Reed, Charlotte M ^a,b   Braida, Louis D ^a,b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b Division of Health Sciences and Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASYNCHRONY; AUDITORY STIMULI; BROADBAND NOISE; DETECTION EXPERIMENTS; DETECTION PERFORMANCE; FINE STRUCTURES; ONSET ASYNCHRONY; PHASE EFFECTS; RELATIVE PHASE; SINGLE-CHANNEL; SOUND PRESSURE LEVEL; TACTILE STIMULI; TIME CONSTANTS; TONE PULSE; UNIMODAL; VIBROTACTILE;

ACOUSTIC FIELDS;

EXPERIMENTS;

ADULT; ARTICLE; AUDITORY MASKING; AUDITORY STIMULATION; FEMALE; HEARING; HUMAN; HUMAN EXPERIMENT; MALE; PRIORITY JOURNAL; SOUND PRESSURE; TACTILE STIMULATION; VIBROTACTILE STIMULATION;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AUDITORY PERCEPTION; FEMALE; FINGERS; HUMANS; MALE; MIDDLE AGED; MODELS, PSYCHOLOGICAL; PHYSICAL STIMULATION; PSYCHOPHYSICS; SIGNAL DETECTION, PSYCHOLOGICAL; TASK PERFORMANCE AND ANALYSIS; TIME FACTORS; TOUCH PERCEPTION; VIBRATION; YOUNG ADULT;

EID: 70350510511     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.3204305     Document Type: Article

References (57)

1. Bensmaia, S. J., Hollins, M., and Yau, J. (2005). " Vibrotactile intensity and frequency information in the Pacinian system.," Percept. Psychophys. 0031-5117 67, 828-841.
2. Bhattacharya, J., Shams, L., and Shimojo, S. (2002). " Sound-induced illusory flash perception: Role of gamma band responses.," NeuroReport 0959-4965 13, 1727-1730. 10.1097/00001756-200210070-00007
3. Braida, L. D. (1991). " Crossmodal integration in the identification of consonant segments.," Q. J. Exp. Psychol. 0033-555X 43A, 647-677.
4. Bresciani, J. P., Ernst, M. O., Drewing, K., Bouyer, G., Maury, V., and Kheddar, A. (2005). " Feeling what you hear: Auditory signals can modulate tactile tap perception.," Exp. Brain Res. 0014-4819 162, 172-180. 10.1007/s00221-004-2128-2 (Pubitemid 40543828)
5. Caclin, A., Soto-Faraco, S., Kingstone, A., and Spence, C. (2002). " Tactile capture' of audition.," Percept. Psychophys. 0031-5117 64, 616-630.
6. Caetano, G., and Jousmaki, V. (2006). " Evidence of vibrotactile input to human auditory cortex.," Neuroimage 1053-8119 29, 15-28. 10.1016/j.neuroimage.2005.07.023 (Pubitemid 41828026)
7. Cappe, C., and Barone, P. (2005). " Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey.," Eur. J. Neurosci. 0953-816X 22, 2886-2902. 10.1111/j.1460-9568. 2005.04462.x (Pubitemid 41818860)
8. Craig, J. C., and Evans, P. M. (1987). " Vibrotactile masking and the persistence of tactual features.," Percept. Psychophys. 0031-5117 42, 309-317.
9. Dirks, D. D., Kamm, C., and Gilman, S. (1976). " Bone-conduction thresholds for normal listeners in force and acceleration units.," J. Speech Hear. Res. 0022-4685 19, 181-186.
10. Ernst, M. O., and Banks, M. S. (2002). " Humans integrate visual and haptic information in a statistically optimal fashion.," Nature (London) 0028-0836 415, 429-433. 10.1038/415429a (Pubitemid 34100953)
11. Fastl, H. (1977). " Subjective duration and temporal masking patterns of broadband noise impulses.," J. Acoust. Soc. Am. 0001-4966 61, 162-168. 10.1121/1.381277 (Pubitemid 8027256)
12. Foxe, J. J., Wylie, G. R., Martinez, A., Schroeder, C. E., Javitt, D. C., Guilfoyle, D., Ritter, W., and Murray, M. M. (2002). " Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study.," J. Neurophysiol. 0022-3077 8, 540-543.
13. Fu, K. M. G., Johnston, T. A., Shah, A. S., Arnold, L., Smiley, J., Hackett, T. A., Garraghty, P. E., and Schroeder, C. E. (2003). " Auditory cortical neurons respond to somatosensory stimulation.," J. Neurosci. 0270-6474 23, 7510-7515.
14. Gescheider, G. A., Barton, W. G., Bruce, M. R., Goldberg, J. H., and Greenspan, M. J. (1969). " Effects of simultaneous auditory stimulation on the detection of tactile stimuli.," J. Exp. Psychol. 0022-1015 81, 120-125. 10.1037/h0027438
15. Gescheider, G. A., Bolanowski, S. J., and Verrillo, R. T. (1989). " Vibrotactile masking: Effects of stimulus onset asynchrony and stimulus frequency.," J. Acoust. Soc. Am. 0001-4966 85, 2059-2064. 10.1121/1.397858
16. Gescheider, G. A., Hoffman, K. E., Harrison, M. A., and Travis, M. L. (1994). " The effects of masking on vibrotactile temporal summation in the detection of sinusoidal and noise signals.," J. Acoust. Soc. Am. 0001-4966 95, 1006-1016. 10.1121/1.408464
17. Gescheider, G. A., Kane, M. J., and Sager, L. C. (1974). " The effect of auditory stimulation on responses to tactile stimuli.," Bull. Psychon. Soc. 0090-5054 3, 204-206.
18. Gescheider, G. A., and Migel, N. (1995). " Some temporal parameters in vibrotactile forward masking.," J. Acoust. Soc. Am. 0001-4966 98, 3195-3199. 10.1121/1.413809
19. Gescheider, G. A., and Niblette, R. K. (1967). " Cross-modality masking for touch and hearing.," J. Exp. Psychol. 0022-1015 74, 313-320. 10.1037/h0024700
20. Gillmeister, H., and Eimer, M. (2007). " Tactile enhancement of auditory detection and perceived loudness.," Brain Res. 0006-8993 1160, 58-68. 10.1016/j.brainres.2007.03.041 (Pubitemid 47058033)
21. Green, D. M. (1958). " Detection of multiple component signals in noise.," J. Acoust. Soc. Am. 0001-4966 30, 904-911. 10.1121/1.1909400
22. Guest, S., Catmur, C., Lloyd, D., and Spence, C. (2002). " Audiotactile interactions in roughness perception.," Exp. Brain Res. 0014-4819 146, 161-171. 10.1007/s00221-002-1164-z (Pubitemid 34994858)
23. Hackett, T. A., de la Mothe, L. A., Ulbert, I., Karmos, G., Smiley, J., and Schroeder, C. E. (2007). " Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane.," J. Comp. Neurol. 0021-9967 502, 924-952. 10.1002/cne.21326 (Pubitemid 46806628)
24. Hamer, R. D., Verrillo, R. T., and Zwislocki, J. J. (1983). " Vibrotactile masking of pacinian and non-pacinian channels.," J. Acoust. Soc. Am. 0001-4966 73, 1293-1303. 10.1121/1.389278
25. Hawkins, J. E., and Stevens, S. S. (1950). " The masking of pure tones and of speech by white noise.," J. Acoust. Soc. Am. 0001-4966 22, 6-13. 10.1121/1.1906581
26. Houtsma, A. J. (2004). " Hawkins and Stevens revisited with insert earphones.," J. Acoust. Soc. Am. 0001-4966 115, 967-970. 10.1121/1.1645246 (Pubitemid 38298546)
27. Jesteadt, W., Bacon, S. P., and Lehman, J. R. (1982). " Forward masking as a function of frequency, masker level, and signal delay.," J. Acoust. Soc. Am. 0001-4966 71, 950-962. 10.1121/1.387576
28. Jousmaki, V., and Hari, R. (1998). " Parchment-skin illusion: Sound-biased touch.," Curr. Biol. 0960-9822 8, R190-191. 10.1016/S0960-9822(98)70120-4
29. Kayser, C., Petkov, C. I., Augath, M., and Logothetis, N. K. (2005). " Integration of touch and sound in auditory cortex.," Neuron 0896-6273 48, 373-384. 10.1016/j.neuron.2005.09.018 (Pubitemid 41476368)
30. Kidd, G., Jr., and Feth, L. L. (1981). " Patterns of residual masking.," Hear. Res. 0378-5955 5, 49-67. 10.1016/0378-5955(81)90026-5 (Pubitemid 11052236)
31. Lakatos, P., Chen, C. M., O'Connell, M. N., Mills, A., and Schroeder, C. E. (2007). " Neuronal oscillations and multisensory interaction in primary auditory cortex.," Neuron 0896-6273 53, 279-292. 10.1016/j.neuron.2006.12. 011 (Pubitemid 46074398)
32. Lamore, P. J., Muijser, H., and Keemijik, C. J. (1986). " Envelope detection of amplitude-modulated high-frequency sinusoidal signals by skin mechanoreceptors.," J. Acoust. Soc. Am. 0001-4966 79, 1082-1085. 10.1121/1.393380
33. Levitt, H. (1971). " Transformed up-down methods in psychoacoustics.," J. Acoust. Soc. Am. 0001-4966 49, 467-477. 10.1121/1.1912375
34. Marrill, T. (1956). " Detection theory and psychophysics.." Technical Report No. 319, Research Laboratory of Electronics, MIT, Cambridge, MA.
35. McGurk, H., and MacDonald, J. (1976). " Hearing lips and seeing voices.," Nature (London) 0028-0836 264, 746-748. 10.1038/264746a0
36. Moore, B. C. J., and Glasberg, B. R. (1983). " Growth of forward masking for sinusoidal and noise maskers as a function of signal delay; implications for suppression in noise.," J. Acoust. Soc. Am. 0001-4966 73, 1249-1259. 10.1121/1.389273
37. Moore, B. C. J., Glasberg, B. R., Plack, C. J., and Biswas, A. K. (1988). " The shape of the ear's temporal window.," J. Acoust. Soc. Am. 0001-4966 83, 1102-1116. 10.1121/1.396055
38. Neville, A. M., and Kennedy, J. B. (1964). Basic Statistical Methods for Engineers and Scientists (International Textbook, Scranton, PA), p. 133.
39. Plack, C. J., and Oxenham, A. J. (1998). " Basilar-membrane nonlinearity and the growth of forward masking.," J. Acoust. Soc. Am. 0001-4966 103, 1598-1608. 10.1121/1.421294 (Pubitemid 28115171)
40. Rabinowitz, W. M., Houtsma, A. J. M., Durlach, N. I., and Delhorne, L. A. (1987). " Multidimensional tactile displays: Identification of vibratory intensity, frequency and contactor area.," J. Acoust. Soc. Am. 0001-4966 82, 1243-1252. 10.1121/1.395260
41. Ro, T., Hsu, J., Yasar, N. E., Elmore, L. C., and Beauchamp, M. S. (2009). " Sound enhances touch perception.," Exp. Brain Res. 0014-4819 195, 135-143. 10.1007/s00221-009-1759-8
42. Robinson, C. E., and Pollack, I. (1973). " Interaction between forward and backward masking: A measure of the integrating period of the auditory system.," J. Acoust. Soc. Am. 0001-4966 53, 1313-1316. 10.1121/1.1913471
43. Schnupp, J. W. H., Dawe, K. L., and Pollack, G. (2005). " The detection of multisensory stimuli in an orthogonal sensory space.," Exp. Brain Res. 0014-4819 162, 181-190. 10.1007/s00221-004-2136-2 (Pubitemid 40543829)
44. Schroeder, C. E., Lindskey, R. W., Specht, C., Marcovici, A., Smiley, J. F., and Javitt, D. C. (2001). " Somatosensory input to auditory association cortex in the macaque monkey.," J. Neurophysiol. 0022-3077 85, 1322-1327.
45. Schurmann, M., Caetano, G., Hlushchuk, Y., Jousmaki, V., and Hari, R. (2006). " Touch activates human auditory cortex.," Neuroimage 1053-8119 30, 1325-1331. 10.1016/j.neuroimage.2005.11.020
46. Schurmann, M., Caetano, G., Jousmaki, V., and Hari, R. (2004). " Hands help hearing: Facilitatory audiotactile interaction at low sound-intensity levels.," J. Acoust. Soc. Am. 0001-4966 115, 830-832. 10.1121/1.1639909 (Pubitemid 38200669)
47. Sekuler, R., Sekuler, A. B., and Lau, R. (1997). " Sound alters visual motion perception.," Nature (London) 0028-0836 385, 308. 10.1038/385308a0 (Pubitemid 27053094)
48. Smiley, J. F., Hackett, T. A., Ulbert, I., Karmas, G., Lakatos, P., Javitt, D. C., and Schroeder, C. E. (2007). " Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys.," J. Comp. Neurol. 0021-9967 502, 894-923. 10.1002/cne.21325 (Pubitemid 46806627)
49. Soto-Faraco, S., Spence, C., and Kingstone, A. (2004). " Congruency effects between auditory and tactile motion: Extending the phenomenon of cross-modal dynamic capture.," Cogn. Affect. Behav. Neurosci. 4, 208-217. 10.3758/CABN.4.2.208 1530-7026 (Pubitemid 39167163)
50. Stein, B. E., London, N., Wilkinson, L. K., and Price, D. D. (1996). " Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis.," J. Cogn. Neurosci. 8, 497-506. 0898-929X
51. Stein, B. E., and Meredith, M. A. (1993). The Merging of the Senses (MIT, Cambridge, MA).
52. Verrillo, R. T., Gescheider, G. A., Calman, B. G., and Van Doren, C. L. (1983). " Vibrotactile masking: Effect of one- and two-site stimulation.," Percept. Psychophys. 0031-5117 33, 379-387.
53. Vogten, L. L. M. (1978). " Simultaneous pure-tone masking: The dependence of masking asymmetries on intensity.," J. Acoust. Soc. Am. 0001-4966 63, 1509-1519. 10.1121/1.381845
54. Wilson, E. C., Reed, C. M., and Braida, L. D. (2008). " Perceptual interactions between vibrotactile and auditory stimuli: Effects of frequency.," in International Multisensory Research Forum Conference, Hamburg, Germany, July.
55. Woods, T. M., and Recanzone, G. H. (2004). " Cross-modal interactions evidenced by the ventriloquism effect in humans and monkeys.," in Handbook of Multisensory Processes, edited by, G. Calvert, C. Spence, and, B. E. Stein, (MIT, Cambridge, MA), pp. 35-48.
56. Yarrow, K., Haggard, P., and Rothwell, J. C. (2008). " Vibrotactile-auditory interactions are post-perceptual.," Perception 0301-0066 37, 1114-1130. 10.1068/p5824
57. Zhou, J., and Shore, S. (2004). " Projections from the trigeminal nuclear complex to the cochlear nuclei: A retrograde and anterograde tracing study in the guinea pig.," J. Neurosci. Res. 0360-4012 78, 901-907. 10.1002/jnr.20343 (Pubitemid 39657577)