Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality

ACM Transactions on Applied Perception

Volumn 6, Issue 2, 2009, Pages

  Riecke, Bernhard E ^a,c   Väljamäe, Aleksander ^b   Schulte Pelkum, Jörg ^a  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^b CHALMERS UNIVERSITY OF TECHNOLOGY   (Sweden)

^c SIMON FRASER UNIVERSITY   (Canada)

Author keywords

Audiovisual interactions; Presence; Psychophysics; Self motion simulation; Spatial sound; Vection; Virtual reality

Indexed keywords

AUDIOVISUAL INTERACTIONS; PRESENCE; PSYCHOPHYSICS; SELF-MOTION SIMULATION; SPATIAL SOUND; VECTION;

PHYSICS; PROJECTION SCREENS; PSYCHOPHYSIOLOGY; ROTATION;

VIRTUAL REALITY;

EID: 63049105058     PISSN: 15443558     EISSN: 15443965     Source Type: Journal    
DOI: 10.1145/1498700.1498701     Document Type: Article

References (73)

1. ALGAZI, V. R., DUDA, R. O., THOMPSON, D. M., AND AVENDANO, C. 2001. The CIPIC HRTF database. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. IEEE, Los Alamitos, CA, 99-102.
2. BEST, V., VAN SCHAIK, A., JIN, C., AND CARLILE, S. 2005. Auditory spatial perception with sources overlapping in frequency and time. Acia Acustica 91, 421-428.
3. BOER, E. R., GIRSHIK, A. R., YAMAMURA, T., AND KUGE, N. 2000. Experiencing the same road twice: A driver-centred comparison between simulation and reality. In Proceedings of the Driving Simulation Conference (DSC'00). Elsevier, New York.
4. BRANDT, T., DICHGANS, J., AND HELD, R. 1973. Optokinesis affects body posture and subjective visual vertical. Pflugers Archiv-European J. Physiol. 339, 97-97.
5. BRANDT, T., DICHGANS, J., AND KOENIG, E. 1973. Differential effects of central versus peripheral vision on egocentric and exocentric motion perception. Exp. Brain Res. 16, 476-491.
6. BRUNGART, D., SIMPSON, B., AND KORDIK, A. 2005. Localization in the presence of multiple simultaneous sounds. Acta Acustica 91, 3, 471-479.
7. BURKI-COHEN, J., GO, T. H., CHUNG, W. Y., SCHROEDER, J., JACOBS, S., AND LONGRIDGE, T. 2003. Simulator fidelity requirements for airline pilot training and evaluation continued: An update on motion requirements research. In Proceedings of the 12th International Symposium on Aviation Psychology. 182-189.
8. CATER, K., CHALMERS, A., AND WARD, G. 2003. Detail to attention: Exploiting visual tasks for selective rendering. In Proceedings of the 14th Eurographics Workshop on Rendering (EGRW '03). ACM, New York, 270-280.
9. CHANCE, S. S., GAUNET, F., BEALL, A. C., AND LOOMIS, J. M. 1998. Locomotion mode affects the updating of objects encountered during travel: The contribution of vestibular and proprioceptive inputs to path integration. Presence Teleoperators Virtual Environ. 7, 2, 168-178.
10. DICHGANS, J. AND BRANDT, T. 1978. Visual-vestibular interaction: Effects on self-motion perception and postural control. In Perception. Handbook of Sensory Physiology, vol. VIII. Springer, New York, 756-804.
11. DODGE, R. 1923. Thresholds of rotation. J. Exp. Psychol. 6, 107-137.
12. DURGIN, F. H., PELAH, A., FOX, L. F., LEWIS, J. Y., KANE, R., AND WALLEY, K. A. 2005. Self-motion perception during locomotor recalibration: More than meets the eye. J. Exp. Psychol. 31, 3, 398-419.
13. DURLACH, N. I. AND MAVOR, A. S., EDS. 1995. Virtual Reality: Scientific and Technological Challenges. National Academy Press, Washington, DC.
14. ERNST, M. O. AND BÜLTHOFF, H. H. 2004. Merging the senses into a robust percept. Trends Cog. Sci. 8, 4, 162-169.
15. FISCHER, M. H. AND KORNMÜLLER, A. E. 1930. Optokinetisch ausgelöste Bewegungswahrnehmung und optokinetischer Nystagmus [Optokinetically induced motion perception and optokinetic nystagmus]. Journal für Psychologie und Neurologie, 273-308.
16. FLANAGAN, M. B., MAY, J. G., AND DOBIE, T. G. 2004. The role of vection, eye movements and postural instability in the etiology of motion sickness. J. Vestibular Res. Equilibrium Orientation 14, 4, 335-346.
17. GARDNER, W. AND MARTIN, K. 1995. Hrtf measurements of a KEMAR. J. Acoust. Soc. Am. 97, 3907-3908.
18. GEKHMAN, B. 1991. Audiokinetic nystagmus. Sensornye Sistemy 5, 2, 71-78. (In Russian.)
19. HENDRDC, C. AND BARFIELD, W. 1996. The sense of presence within auditory virtual environments. Presence Teleoperators Virtual Environ. 5, 3, 290-301.
20. HENNEBERT, P. E. 1960. Audiokinetic nystagmus. J. Auditory Res. 1, 1, 84-87.
21. HETTINGER, L. J. 2002. Illusory self-motion in virtual environments. In Handbook of Virtual Environments, K. M. Stanney, Ed. Lawrence Erlbaum, Mahwah, NJ, 471-492.
22. HOLLERBACH, J. M. 2002. Locomotion interfaces. In Handbook of Virtual Environments, K. M. Stanney, Ed. Lawrence Erlbaum, Mahwah, NJ, 239-254.
23. HOWARD, I. P. 1982. Human Visual Orientation. J. Wiley, Hoboken, NJ.
24. HOWARD, I. P. 1986. The perception of posture, self motion, and the visual vertical. In Handbook of Human Perception and Performance, vol. 1, K. R. Boff, L. Kaufman, and J. P. Thomas, Eds. Wiley, Hoboken, NJ, 18.1-18.62.
25. KAPRALOS, B., ZIKOVITZ, D., JENKIN, M., AND HARRIS, L. 2004. Auditory cues in the perception of self-motion. In Proceedings of the 116th AES Convention.
26. KLEINER, M. AND DALENBÄCK, B-I. ANDSVENSSON, P. 1993. Auralization - an overview. J. Audio Engin. Soc. 41, 11, 861-875.
27. LACKNER, J. R. 1977. Induction of illusory self-rotation and nystagmus by a rotating sound-field. Aviat. Space Environ. Med. 48, 2, 129-131.
28. LANGENDIJK, E. H. A. AND BRONKHORST, A. W. 2000. Fidelity of three-dimensional-sound reproduction using a virtual auditory display. Acoustical Soc. Am. J. 107, 528-537.
29. LARSSON, P., VÄLJAMÄE, A., VÄSTFJÄLL, D., TAJADURA-JIMÉNEZ, A., AND KLEINER, M. 2009. Auditory-induced presence in mixed presence reality environments and related technology. In The Engineering of Mixed Reality Systems. E. Dubois, L. Nigay, and P. Gray, Eds. Springer.
30. LARSSON, P., VÄSTFJÄLL, D., TAJADURA AND KLEINER, M. 2004. Perception of self-motion and presence in auditory virtual environments. In Proceedings of 7th Annual Workshop of Presence. 252-258. http://www.kyb.mpg.de/publication.html?publ=2953.
31. LEPECQ, J. C., GIANNOPULU, I., AND BAUDONNIERE, P. M. 1995. Cognitive effects on visually induced body motion in children. Perception 24, 4, 435-449.
32. LOMBARD, M. AND DITTON, T. 1999. At the heart of it all: The concept of presence. J. Comput. Mediated Comm. 3, 2.
33. LUCIANI, A. 2004. Dynamics as a common criterion to enhance the sense of presence in virtual environments. In Proceedings of 7th Annual Workshop Presence 2004. 96-103.
34. MACH, E. 1875. Grundlinien der Lehre von der Bewegungsempfindung. Engelmann, Leipzig, Germany.
35. MARMEKARELSE, A. M. AND BLES, W. 1977. Circular vection and human posture ii: Does the auditory-system play a role. Agressologie 18, 6, 329-333.
36. MCFARLAND, W. AND WEBER, B. 1969. An investigation of ocular response to various forms of sound field auditory stimulation. J. Auditory Res. 9, 3, 236-239.
37. MOECK, T., BONNEEL, N., TSINGOS, N., DRETTAKIS, G., VIAUD-DELMON, I., AND ALLOZA, D. 2007. Progressive perceptual audio rendering of complex scenes. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D '07). ACM, New York, 189-196.
38. MULDER, M., VAN PAASSEN, M. M., AND BOER, E. R. 2004. Exploring the roles of information in the control of vehicular locomotion-from kinematics and dynamics to cybernetics. Presence Teleoperators Virtual Environ. 13, 535-548.
39. OZAWA, K., CHUJO, Y., SUZUKI, Y., AND SONE, T. 2004. Psychological factors involved in auditory presence. Acoustical Sci. Tech. 24, 42-44.
40. POPE, J. AND CHALMERS, A. 1999. Multi-sensory rendering: Combining graphics and acoustics. In Proceedings of the 7th International Conference in Central Europe on Computer Graphics. 233-242.
41. RIECKE, B. E., FEUEREISSEN, D., AND RIESER, J. J. 2009. Auditory self-motion simulation is facilitated by haptic and vibrational cues suggesting the possibility of actual motion. ACM Trans. Appl. Percept. Submitted.
42. RIECKE, B. E., FEUEREISSEN, D., AND RIESER, J. J. 2008a. Auditory self-motion illusions ("circular vection") can be facilitated by vibrations and the potential for actual motion. In Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization (APGV'08). ACM, New York, 147-154.
43. RIECKE, B. E., FEUEREISSEN, D., AND RIESER, J. J. 2008b. Contribution and interaction of auditory and biomechanical cues for self-motion illusions ("circular vection"). In Proceedings of the CyberWalk Workshop.
44. RIECKE, B. E., SCHULTE-PELKUM, J., AVRAAMIDES, M. N., AND BÜLTHOFF, H. H. 2004. Enhancing the visually induced self-motion illusion (vection) under natural viewing conditions in virtual reality. In Proceedings of 7th Annual Workshop Presence 2004. 125-132. www.kyb.mpg.de/publication.html?publ=2864.
45. RIECKE, B. E., SCHULTE-PELKUM, J., AVRAAMIDES, M. N., VON DER HEYDE, M., AND BÜLTHOFF, H. H. 2006. Cognitive factors can influence self-motion perception (vection) in virtual reality. ACM Trans. Appl. Percep. 3, 3, 194-216.
46. RIECKE, B. E., SCHULTE-PELKUM, J., AND BÜLTHOFF, H. H. 2005. Perceiving simulated ego-motions in virtual reality: Comparing large screen displays with HMDs. Proc. SPIE, Vol. 5666. SPIE, Bellingham, WA, 344-355.
47. RIECKE, B. E., SCHULTE-PELKUM, J., CANIARD, F., AND BÜLTHOFF, H. H. 2005a. Influence of auditory cues on the visually-induced self-motion illusion (circular vection) in virtual reality. In Proceedings of 8th Annual Workshop Presence 2005. ACM, New York, 49-57.
48. RIECKE, B. E., SCHULTE-PELKUM, J., CANIARD, F., AND BÜLTHOFF, H. H. 2005b. Towards lean and elegant self-motion simulation in virtual reality. In Proceedings of IEEE Virtual Reality. IEEE, Los Alamitos, CA, 131-138.
49. RIECKE, B. E., VÄSTFJÄLL, D., LARSSON, P., AND SCHULTE-PELKUM, J. 2005. Top-down and multi-modal influences on self-motion perception in virtual reality. In Proceedings of the Internation Human-Computer Interaction Conference. HCI, Toronto, Ontario, 1-10.
50. RIECKE, B. E. AND WIENER, J. M. 2007. Can people not tell left from right in vr? point-to-origin studies revealed qualitative errors in visual path integration. In Proceedings of IEEE Virtual Reality 2007. IEEE, Los Alamitos, CA, 3-10.
51. RUMSEY, F. 2002. Spatial quality evaluation for reproduced sound: Terminology, meaning, and a scene-based paradigm. J. Audio Engin. Soc 50, 9, 651.
52. SAKAMOTO, S., OSADA, Y., SUZUKI, Y., AND GYOBA, J. 2004. The effects of linearly moving sound images on selfmotion perception. Acoustical Sci. Tech. 25, 100-102.
53. SCHINAUER, T., HELLMANN, A., AND HÖGER, R. 1993. Dynamic acoustical stimulation affects self-motion perception. In Contributions to Psychological Acoustics. Results of the 6th Oldenburg Symposium on Psychological Acoustics, A. Schick, Ed. BIS, Oldenburg, Germany, 373-385.
54. SCHUBERT, T., FRIEDMANN, F., AND REGENBRECHT, H. 2001. The experience of presence: Factor analytic insights. Presence Teleop. Virtual Environ. 10, 3, 266-281.
55. SCHULTE-PELKUM, J. AND RIECKE, B. E. 2009. An integrative approach to presence and self-motion perception research. http://www.sfu.ca/~ber1/web/Shuite-PelkumRiecke-HOP-chapter.pdf.
56. SCHULTE-PELKUM, J., RIECKE, B. E., AND BÜLTHOFF, H. H. 2004. Vibrational cues enhance believability of ego-motion simulation. In International Multisensory Research Forum (IMRF). http://www.kyb.mpg.de/publication.html?publ=2766.
57. SCHULTE-PELKUM, J., RIECKE, B. E., VON DER HEYDE, M., AND BÜLTHOFF, H. H. 2004. Influence of display device and screen curvature on perceiving and controlling simulated ego-rotations from optic flow. Tech. rep. 122, MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=2574.
58. SOTO-FARACO, S., KINGSTONE, A., AND SPENCE, C. 2003. Multisensory contributions to the perception of motion. Neuropsychologia 41, 13, 1847-1862.
59. SUNDSTEDT, V., DEBATTISTA, K., AND CHALMERS, A. 2004. Selective rendering using task-importance maps. In Proceedings of the 1st Symposium, on Applied Perception in Graphics and Visualization (APGV04). ACM, New York, 175.
60. TAN, D. S., GERGLE, D., SCUPELLI, P., AND PAUSCH, R. 2006. Physically large displays improve performance on spatial tasks. ACM Trans. Comput. Hum. Interact. 13, 1, 71-99.
61. TAN, D. S., GERGLE, D., SCUPELLI, P. G., AND PAUSCH, R. 2004. Physically large displays improve path integration in 3D virtual navigation tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM Press, New York, 439-446.
62. VÄLJAMÄE, A., LARSSON, P., TAJADURA-JIMÉNEZ, A., VÄSTFJÄLL, D., AND KLEINER, M. 2008. Auditory cues for multisensory enhancement of self-motion perception. IEEE Multimedia.
63. VÄLJAMÄE, A., LARSSON, P., VÄSTFJÄLL, D., AND KLEINER, M. 2004. Auditory presence, individualized head-related transfer functions, and illusory ego-motion in virtual environments. In Proceedings of 7th Annual Workshop of Presence. 141-147.
64. VÄLJAMÄE, A., LARSSON, P., VÄSTFJÄLL, D., AND KLEINER, M. 2005. Travelling without moving: Auditory scene cues for translational self-motion. In Proceedings of the 11th Meeting of International Conference on Auditory Display (ICAD 2005).
65. VÄLJAMÄE, A., LARSSON, P., VÄSTFJÄLL, D., AND KLEINER, M. 2006. Vibrotactile enhancement of auditory induced self-motion and spatial presence. J. Acoustic Engin. Soc. 54, 10, 954-963.
66. VÄLJAMÄE, A., LARSSON, P., VÄSTFJÄLL, D., AND KLEINER, M. 2008a. Auditory landmarks enhance multimodal circular vection simulations in virtual reality. J. Acoustic Engin. Soc Submitted.
67. VÄLJAMÄE, A., LARSSON, P., VÄSTFJÄLL, D., AND KLEINER, M. 2008b. Sound representing self-motion in virtual environments enhances linear vection. Presence: Teleoperators Virtual Environ. 17, 1, 43-56.
68. VÄLJAMÄE, A., VÄSTFJÄLL, D., LARSSON, P., AND KLEINER, M. 2008. Perceived sound in mediated environments. In Immersed in Media Experiences: Presence Psychology and Design (Handbook of Presence). Lawrence Erlbaum, Mahwah, NJ.
69. VAN DER STEEN, F. A. M. AND BROCKHOFF, P. T. M. 2000. Induction and impairment of saturated yaw and surge vection. Percep. Psychophys. 62, 1, 89-99.
70. VON HELMHOLTZ, H. 1896. Handbuch der physiologischen Optik. L. Voss, Hamburg-Leipzig.
71. WARREN, R. AND WERTHEIM, A. H., EDS. 1990. Perception & Control of Self-Motion. Erlbaum, Mahwah, NJ.
72. WATSON, J. E. 1968. Evaluation of audiokinetic nystagmus as as test of auditory sensitivity. J. Auditory Res. 8, 161-165.
73. WRIGHT, W. G., DIZIO, P., AND LACKNER, J. R. 2006. Perceived self-motion in two visual contexts: dissociable mechanisms underlie perception. J. Vestib. Res. 16, 1-2, 23-28.