Audiotactile multisensory interactions in human information processing

Japanese Psychological Research

Volumn 48, Issue 3, 2006, Pages 158-173

  Kitagawa, Norimichi ^a,b   Spence, Charles ^a,b  

^a NTT Communication Science Laboratories   (Japan)

^b UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Crossmodal interaction; Hearing; Multisensory integration; Peripersonal space; Touch

Indexed keywords

EID: 33750570262     PISSN: 00215368     EISSN: 14685884     Source Type: Journal    
DOI: 10.1111/j.1468-5884.2006.00317.x     Document Type: Review

References (83)

1. Adelstein, B. D., Begault, D. R., Anderson, M. R., & Wenzel, E. M. (2003). Sensitivity to haptic-audio asynchrony. Proceedings of the 5th International Conference on Multimodal Interfaces, 73-76.
2. Altinsoy, M. E. (2003). Perceptual aspects of auditory-tactile asynchrony. Proceedings of the Tenth International Congress on Sound and Vibration, Stockholm, Sweden, 3831-3838.
3. Blakemore, S. J., Wolpert, D. M., & Frith, C. D. (1998). Central cancellation of self-produced tickle sensation. Nature Neuroscience, 1, 635-640.
4. Blauert, J. (1997). Spatial hearing: the psychophysics of human sound localization (Revised Ed.). Cambridge, MA: MIT Press.
5. Bresciani, J. P., Ernst, M. O., Brewing, K., Bouyer, G., Maury, V., & Kheddar, A. (2005). Feeling what you hear: Auditory signals can modulate tactile tap perception. Experimental Brain Research, 162,172-180.
6. Bronkhorst, A. W., & Houtgast, T. (1999). Auditory distance perception in rooms. Nature, 397, 517-520.
7. Brungart, D. S. (1999). Auditory localization of nearby sources. III. Stimulus effects. Journal of the Acoustical Society of America, 106,3589-3602.
8. Brungart, D. S. (2002). Near-field virtual audio displays. Presence, 11, 93-106.
9. Brungart, D. S., Durlach, N. I., & Rabinowitz, W. M. (1999). Auditory localization of nearby sources. II. Localization of a broadband source. Journal of the Acoustical Society of America, 106, 1956-1968.
10. Brungart, D. S., & Rabinowitz, W. M. (1999). Auditory localization of nearby sources. Head-related transfer functions. Journal of the Acoustical Society of America, 106, 1465-1479.
11. Buchtel, H. A., Butter, C. M., & Ayvasik, B. (1996). Effects of stimulus source and intensity on covert orientation to auditory stimuli. Neuropsychologia, 34, 979-985.
12. Caclin, A., Soto-Faraco, S., Kingstone, A., & Spence, C. (2002). Tactile "capture" of audition. Perception and Psychophysics, 64, 616-630.
13. Caetano, G., & Jousmäki, V. (2006). Evidence of vibrotactile input to human auditory cortex. Neuroimage, 29, 15-28.
14. Calvert, G. A., Spence, C., & Stein, B. E. (Eds.) (2004). The handbook of multisensory processing. Cambridge, MA: MIT Press.
15. Coleman, P. D. (1962). Failure to localize the source distance of an unfamiliar sound. Journal of the Acoustical Society of America, 34, 345-346.
16. Dehner, L. R., Keniston, L. P., Clemo, H. R., & Meredith, M. A. (2004). Cross-modal circuitry between auditory and somatosensory areas of the cat anterior ectosylvian sulcal cortex: A "new" inhibitory form of multisensory convergence. Cerebral Cortex, 14, 387-403.
17. Diederich, A., & Colonius, H. (2004). Bimodal and trimodal multisensory enhancement: Effects of stimulus onset and intensity on reaction time. Perception and Psychophysics, 66, 1388-1404.
18. Farnè, A., & Làdavas, E. (2002). Auditory peripersonal space in humans. Journal of Cognitive Neuroscience, 14, 1030-1043.
19. Forster, B., Cavina-Pratesi, C., Aglioti, S. M., & Berlucchi, G. (2002). Redundant target effect and intersensory facilitation from visual-tactile interactions in simple reaction time. Experimental Brain Research, 143, 480-487.
20. Fu, K. M., Johnston, T. A., Shah, A. S., Arnold, L., Smiley, J., Hackett, T. A., Garraghty, P. E., & Schroeder, C. E. (2003). Auditory cortical neurons respond to somatosensory stimulation. Journal of Neuroscience, 23, 7510-7515.
21. Fujisaki, W., Shimojo, S., Kashino, M., & Nishida, S. (2004). Recalibration of audiovisual simultaneity. Nature Neuroscience, 7, 773-778.
22. Gillmeister, H., & Eimer, M. (2004). Tactile enhancement of perceived loudness. Talk presented at the 5th meeting of the International Multisensory Research Forum. Sitges, Spain (June 2-5).
23. Gillmeister, H., & Eimer, M. (2005). Tactile facilitation of auditory stimulus detection. Poster presented at the 6th meeting of the International Multisensory Research Forum. Rovereto, Italy (June 5-8).
24. Gondan, M., Niederhaus, B., Rösler, F., & Röder, B. (2005). Multisensory processing in the redundant-target effect: A behavioral and event-related potential study. Perception and Psychophysics, 67, 713-726.
25. Graziano, M. S. A., & Cooke, D. F. (2006). Parietofrontal interactions, personal space, and defensive behavior. Neuropsychologia, 44, 845-859.
26. Graziano, M. S. A., Gross, C. G., Taylor, C. S. R., & Moore, T. (2004). A system of multimodal areas in the primate brain. In C. Spence & J. Driver (Eds.), Crossmodal space and crossmodat attention (pp. 51-67). Oxford: Oxford University Press.
27. Graziano, M. S. A., Reiss, L. A. J., & Gross, C. G. (1999). A neuronal representation of the location of nearby sounds. Nature, 397, 428-430.
28. Guest, S., Catmur, C, Lloyd, D., & Spence, C. (2002). Audiotactile interactions in roughness perception. Experimental Brain Research, 146, 161-171.
29. Heffner, R. S., & Heffner, H. E. (1992a). Evolution of sound localization in mammals. In D. B. Webster, R. R. Fay & A. N. Popper (Eds.), The evolutionary biology of hearing (pp. 691-715). New York: Springer-Verlag.
30. Heffner, R. S., & Heffner, H. E. (1992b). Visual factors in sound localization in mammals. Journal of Comparative Neurology, 317, 219-232.
31. Heller, M. A. (1982). Visual and tactual texture perception: Intersensory cooperation. Perception and Psychophysics, 31, 339-344.
32. Hötting, K., & Röder, B. (2004). Hearing cheats touch, but less in congenitally blind than in sighted individuals. Psychological Science, 15, 60-64.
33. Hötting, K., Rösler, F., & Röder. B. (2003). Crossmodal and intermodal attention modulates event-related brain potentials to tactile and auditory stimuli. Experimental Brain Research, 148, 26-37.
34. Jousmaki, V., & Hari, R. (1998). Parchment-skin illusion: Sound-biased touch. Current Biology, 8, R190.
35. Katz, D. (1925/1989). The world of touch. Hillsdale, NJ: Erlbaum.
36. Kayser, C., Petkov, C. I., Augath, M., & Logothetis, N. K. (2005). Integration of touch and sound in auditory cortex. Neuron, 48, 373-384.
37. Keetels, M., & Vroomen, J. (2005). The role of spatial disparity and hemifields in audio-visual temporal order judgments. Experimental Brain Research, 167, 635-640.
38. Kitagawa, N., & Igarashi, Y. (2005). Tickle sensation induced by hearing a sound. Japanese Journal of Psychonomic Science, 24, 121-122.
39. Kitagawa, N., Zampini, M., & Spence, C. (2005). Audiotactile interactions in near and far space. Experimental Brain Research, 166, 528-537.
40. Kóbor, I., Füredi, L., Kovács, G., Spence, C., & Vidnyánszky, Z. (2006). Back-to-front: Improved tactile discrimination performance in the space you cannot see. Neuroscience Letters, 400, 163-167.
41. Ladavas, E., & Farnè, A. (2004). Neuropsychological evidence for multimodal representations of space near specific body parts. In C. Spence & J. Driver (Eds.), Crossmodal space and crossmodal attention (pp. 69-98). Oxford: Oxford University Press.
42. Làdavas, E., Pavani, F., & Farnè, A. (2001). Auditory peripersonal space in humans: A case of auditorytactile extinction. Neurocase, 7, 97-103.
43. Lederman, S. J. (1979). Auditory texture perception. Perception, 8, 93-103.
44. Lederman, S. J., Klatzky, R. L., Morgan, T., & Hamilton, C. (2002). Integrating multimodal information about surface texture via a probe: Relative contributions of haptic and touch-produced sound sources. Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 97-104.
45. Lederman, S. J., Martin, A., Tong, C., & Klatzky, R. L. (2003). Relative performance using haptic and/ or touch-produced auditory cues in a remote absolute texture identification task. Proceedings of the 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 151-158.
46. Lloyd, D. M., Merat, N., McGlone, F., & Spence, C. (2003). Crossmodal links between audition and touch in covert endogenous spatial attention. Perception and Psychophysics, 65, 901-924.
47. Lundborg, G., Björkman, A., Hansson, T., Nylander, L., Nyman, T., & Rosén, B. (2005). Artificial sensibility of the hand based on cortical audiotactile interaction: A study using functional magnetic resonance imaging. Scandinavian Journal of Plastic and Reconstructive Surgery and Hand Surgery, 39, 370-372.
48. Lundborg, G., Rosén, B., & Lindberg, S. (1999). Hearing as substitution for sensation: A new principle for artificial sensibility. Journal of Hand Surgery, 24, 219-224.
49. Maravita, A., Spence, C., & Driver, J. (2003). Multisensory integration and the body schema: Close to hand and within reach. Current Biology, 13, R531-R539.
50. Menning, H., Ackermann, H., Hertrich, I., & Mathiak, K. (2005). Spatial auditory attention is modulated by tactile priming. Experimental Brain Research, 164, 41-47.
51. Merat, N., Spence, C., Lloyd, D. M., Withington, D. J., & McGlone, F. (1999). Audiotactile links in focused and divided spatial attention. Society for Neuroscience Abstracts, 25, 1417.
52. Miller, J. O. (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14, 247-279.
53. Molholm, S., Ritter, W., Murray, M. M., Javitt, D. C., Schroeder, C. E., & Foxe, J. J. (2002). Multisensory auditory-visual interactions during early sensory processing in humans: A high-density electrical mapping study. Cognitive Brain Research, 14, 115-128.
54. Murray, M. M., Molholm, S., Michel, C. M., Heslenfeld, D. J., Ritter, W., Javitt, D. C., Schroeder, C. E., & Foxe, J. J. (2005). Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cerebral Cortex, 15, 963-974.
55. Ortigue, S., Jabaudon, D., Landis, T., Michel, C. M., Maravita, A., & Blanke, O. (2005). Preattentive interference between touch and audition: A case study on multisensory alloesthesia. Neuroreport, 16, 865-868.
56. Sanabria, D., Soto-Faraco, S., & Spence, C. (2005). Spatiotemporal interactions between audition and touch depend on hand posture. Experimental Brain Research, 165, 505-514.
57. Schroeder, C. E., & Foxe, J. (2005). Multisensory contributions to low-level, "unisensory" processing. Current Opinion in Neurobiology, 15, 454-458.
58. Schümann, M., Caetano, G., Hlushchuk, Y., Jousmäki, V., & Hari, R. (2006). Touch activates human auditory cortex. Neuroimage, 30, 1325-1331.
59. Shinn-Cunningham, B. G., Kopco, N., & Martin, T. J. (2005). Localizing nearby sound sources in a classroom; Binaural room impulse responses. Journal of the Acoustical Society of America, 117, 3100-3115.
60. Spence, C., & Driver, J. (1996). Audiovisual links in endogenous covert spatial attention. Journal of Experimental Psychology: Human Perception and Performance, 22, 1005-1030.
61. Spence, C., & McGlone, F. P. (2001). Reflexive spatial orienting of tactile attention. Experimental Brain Research, 141, 324-330.
62. Spence, C., & Driver, J. (1997). Cross-modal links in attention between audition, vision, and touch: Implications for interface design. International Journal of Cognitive Ergonomics, 1, 351-373.
63. Spence, C., & Driver, J. (Eds.) (2004). Crossmodal space and crossmodal attention. Oxford: Oxford University Press.
64. Spence, C., & Zampini, M. (submitted). Signature sounds and the contribution of auditory cues to multisensory product perception. Acta Acustica.
65. Spence, C., Nicholls, M. E. R., Gillespie, N., & Driver, J. (1998). Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Perception and Psychophysics, 60, 544-557.
66. Spence, C., Pavani, F., & Driver, J. (2000). Cross-modal links between vision and touch in covert endogenous spatial attention. Journal of Experimental Psychology: Human Perception and Performance, 26, 1298-1319.
67. Spence, C., Shore, D. I., & Klein, R. M. (2001). Multi-sensory prior entry. Journal of Experimental Psychology: General, 130, 799-832.
68. Spence, C., Baddeley, R., Zampini, M., James, R., & Shore, D. I. (2003). Multisensory temporal order judgments: When two locations are better than one. Perception and Psychophysics, 65, 318-328.
69. Spence, C., Pavani, F., & Driver, J. (2004). Spatial constraints on visual-tactile cross-modal distractor congruency effects. Cognitive, Affective and Behavioral Neuroscience, 4, 148-169.
70. Spence, C. J., & Driver, J. (1994). Covert spatial orienting in audition: Exogenous and endogenous mechanisms. Journal of Experimental Psychology: Human Perception and Performance, 20, 555-574.
71. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press.
72. Tassinari, G., & Campara, D. (1996). Consequences of covert orienting to non-informative stimuli of different modalities: A unitary mechanism? Neuropsychologia, 34, 235-245.
73. Townsend, J. T., & Ashby, F. G. (1983). Stochastic modelling of elementary psychological processes. New York: Cambridge University Press.
74. Turatto, M., Galfano, G., Bridgeman, B., & Umiltà, C. (2004). Space-independent modality-driven attentional capture in auditory, tactile and visual systems. Experimental Brain Research, 155, 301-310.
75. Zahorik, P. (2002). Direct-to-reverberant energy ratio sensitivity. Journal of the Acoustical Society of America, 112, 2110-2117.
76. Zampini, M., & Spence, C. (2004). The role of auditory cues in modulating the perceived crispness and staleness of potato chips. Journal of Sensory Studies, 19, 347-363.
77. Zampini, M., & Spence, C. (2005). Modifying the multisensory perception of a carbonated beverage using auditory cues. Food Quality and Preference, 16, 632-641.
78. Zampini, M., & Spence, C. (submitted). Assessing the influence of auditory cues on the perception of the forcefulness and pleasantness of aerosol sprays. Journal of Evolutionary Psychology.
79. Zampini, M., Guest, S., & Spence, C. (2003). The role of auditory cues in modulating the perception of electric toothbrushes. Journal of Dental Research, 82, 929-932.
80. Zampini, M., Shore, D. I., & Spence, C. (2003a). Audiovisual temporal order judgments. Experimental Brain Research, 152, 198-210.
81. Zampini, M., Shore, D. I., & Spence, C. (2003b). Multisensory temporal order judgments: The role of hemispheric redundancy. International Journal of Psychophysiology, 50, 165-180.
82. Zampini, M., Brown, T., Shore, D. I., Maravita, A., Röder, B., & Spence, C. (2005). Audiotactile temporal order judgments. Acta Psychologica, 118, 277-291.
83. Zampini, M., Torresan, D., Spence, C., & Murray, M. (2006). Auditory-tactile multisensory interactions: Effects of space and posture. Poster presented at the Cognitive Neuroscience Society meeting. San Francisco, California, p. 153.