Multisensory integration with a head-mounted display: Background visual motion and sound motion

Human Factors

Volumn 52, Issue 1, 2010, Pages 78-91

  Harrison, William J ^a   Thompson, Matthew B ^a   Sanderson, Penelope M ^a  

^a UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

auditory displays; background visual motion; displays and controls; event detection; eye movement; free field; free field speaker; head mounted displays; mobile head mounted displays; multimodal displays; multisensory integration; optokinetic nystagmus; perceptual performance; psychomotor processes; self motion; speaker motion; tracking; vection; vision; visual performance; wearable computing

Indexed keywords

AUDITORY DISPLAY; DISPLAYS AND CONTROLS; EVENT DETECTION; HEAD MOUNTED DISPLAYS; MOTION TRACKING; MULTIMODAL DISPLAYS; MULTISENSORY INTEGRATION; OPTOKINETIC NYSTAGMUS; SELF MOTION; VECTION; VISUAL MOTION; VISUAL PERFORMANCE; WEARABLE COMPUTING;

EYE MOVEMENTS; HEALTH CARE; INTEGRATION; OCCUPATIONAL DISEASES; VISUAL COMMUNICATION; WEARABLE COMPUTERS;

HELMET MOUNTED DISPLAYS;

ADOLESCENT; ADULT; ARTICLE; CLINICAL TRIAL; CONTROLLED CLINICAL TRIAL; CONTROLLED STUDY; FEMALE; HEAD; HUMAN; INFORMATION PROCESSING; MALE; MOTION; RANDOMIZED CONTROLLED TRIAL; SOUND; TASK PERFORMANCE; VISION;

ADOLESCENT; ADULT; DATA DISPLAY; FEMALE; HEAD; HUMANS; MALE; MOTION; SOUND; TASK PERFORMANCE AND ANALYSIS; VISUAL PERCEPTION; YOUNG ADULT;

EID: 77954202686     PISSN: 00187208     EISSN: 15478181     Source Type: Journal    
DOI: 10.1177/0018720810367790     Document Type: Article

References (32)

1. Behar, I., Wiley, R.W., Levine, R.R., Rash, C.E., & Walsh, D.J. (1990). Visual survey of Apache aviators (VISAA) (No. ADA230201). Fort Rucker, AL: U.S. Army Aeromedical Research Laboratory.
2. Bertelson, P., & de Gelder, B. (2004). The psychology of multimodal perception. In C. Spence & J. Driver (Eds.), Crossmodal space and crossmodal attention (pp. 141-177). New York, NY: Oxford University Press.
3. Colonius, H., & Arndt, P. (2001). A two-stage model for visual-auditory interaction in saccadic latencies. Perception & Psychophysics, 63, 126-147.
4. Eriksson, L., Berglund, A., Willen, B., Svensson, J., Petterstedt, M., Carlander, O.,... Allerbo, G. (2008). On visual, vibrotactile, and 3D audio directional cues for dismounted soldier way-point naviation. In Proceedings of the Human Factors and Ergonomics Society 52nd Annual Meeting (pp. 1282-1286). Santa Monica, CA: Human Factors and Ergonomics Society.
5. Groh, J.M., & Werner-Reiss, U. (2002). Visual and auditory integration. In V. S. Ramachandran (Ed.), Encyclopaedia of the human brain (Vol. 4, pp. 739-752). San Diego, CA: Academic Press.
6. Hutton, S.B., & Tegally, D. (2005). The effects of dividing attention on smooth pursuit eye tracking. Experimental Brain Research, 163, 306-313.
7. Jack, C.E., & Thurlow, W.R. (1973). Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. Perceptual and Motor Skills, 37, 967-979.
8. Lajoie, Y., Teasdale, N., Bard, C., & Fleury, M. (1993). Attentional demands for static and dynamic equilibrium. Experimental Brain Research, 97, 139-144.
9. Laramee, R.S., & Ware, C. (2002). Rivalry and interference with a head-mounted display. ACM Transactions on Computer-Human Interaction, 9, 238-251.
10. Leigh, R., & Zee, D. (1999). The neurology of eye movements (3 rd ed.). New York, NY: Oxford University Press.
11. Liu, D., Jenkins, S., & Sanderson, P.M. (2009). Patient monitoring with head-mounted displays. Current Opinion in Anaesthesiology, 22, 796-803.
12. Liu, D., Jenkins, S., Sanderson, P., Watson, M., Leane, T., Kruys, A., & Russell, W.J. (2009). Monitoring with head-mounted displays: Performance and safety in a full-scale simulator and part-task trainer. Anesthesia & Analgesia, 109, 1135-1146.
13. Paige, G.D., Telford, L., Seidman, S.H., & Barnes, G.R. (1998). Human vestibuloocular reflex and its interactions with vision and fixation distance during linear and angular head movement. Journal of Neurophysiology, 80, 2391-2404.
14. Pheasant, S., & Haslegrave, C.M. (2006). Bodyspace: Anthropometry, ergonomics and the design of work (3 rd ed.). London, England: Taylor and Francis.
15. Riecke, B.E., VCurrency signljamCurrency signe, A., & Schulte-Pelkum, J. (2009). Moving sounds enhance the visually-induced self-motion illustion (circular vection) in virtual reality. ACM Transactions on Applied Perception, 6 (2). 7 :1- 7: 27.
16. Sanderson, P.M., Watson, M.O., Russell, W.J., Jenkins, S., Liu, D., Green, N.,... Krupenia, S. (2008). Advanced auditory displays and head-mounted displays: Advantages and disadvantages for monitoring by the distracted anesthesiologist. Anesthesia & Analgesia, 106, 1787-1797.
17. Saville, D.J. (1990). Multiple comparison procedures: The practical solution. American Statistician, 44, 174-180.
18. Schweigart, G., Mergner, T., & Barnes, G. (1999). Eye movements during combined pursuit, optokinetic and vestibular stimulation in macaque monkey. Experimental Brain Research, 127, 54-66.
19. Sharpe, J.A., & Johnston, J.L. (1993). The vestibulo-ocular reflex: Clinical, anatomic, and physiologic correlates. In J. A. Sharpe & H. O. Barber (Eds.), The vestibulo-ocular reflex and vertigo (pp. 15-39). New York, NY: Raven Press.
20. Slutsky, D.A., & Recanzone, G.H. (2001). Temporal and spatial dependency of the ventriloquism effect. Neuroreport, 12, 7-10.
21. Sparrow, W.A., Bradshaw, E.J., Lamoureux, E., & Tirosh, O. (2002). Ageing effects on the attention demands of walking. Human Movement Science, 21, 961-972.
22. Teramoto, W., Watanabe, H., & Umemura, H. (2008). Change of temporal-order judgment of sounds during long-lasting exposure to large-field visual motion. Perception, 37, 1649-1666.
23. Teramoto, W., Watanabe, H., & Umemura, H., Matsuoka, K., & Kita, S. (2004). Judgment biases of temporal order during apparent self-motion. IEICE Transactions on Information and Systems, E87D, 1466-1476.
24. Thompson, M.B., & Sanderson, P.M. (2008). Multisensory integration with a head-mounted display: Sound delivery and self-motion. Human Factors, 50, 789-800.
25. Thompson, M.B., Tear, M.J., & Sanderson, P.M. (2010). Multisensory integration with a head-mounted display: Role of response modality. Human Factors, 52, 92-104.
26. Thurlow, W.R., & Jack, C.E. (1973). A study of certain determinants of the ventriloquism effect. Perceptual and Motor Skills, 36, 1171-1184.
27. VCurrency signljamCurrency signe, A. (2009). Auditorily-induced illusory self-motion: A review. Brain Research Reviews, 61, 240-255.
28. Veltman, J.A., Oving, A.B., & Bronkhorst, A.W. (2004). 3-D audio in the fighter cockpit improves task performance. International Journal of Aviation Psychology, 14, 239-256.
29. Wickens, C.D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3, 159-177.
30. Williams, I., Mulhall, L., Mattingley, J., Lueck, C., & Abel, L. (2006). Optokinetic nystagmus as an assessment of visual attention to divided stimuli. Journal of Clinical Neuroscience, 13, 828-833.
31. Zampini, M., Guest, S., Shore, D.I., & Spence, C. (2005). Audiovisual simultaneity judgments. Perception and Psychophysics, 67, 531-544.
32. Zotkin, D.N., Duraiswami, R., Grassi, E., & Gumerov, N.A. (2006). Fast head-related transfer function measurement via reciprocity. Acoustical Society of America, 120, 2202-2215.