Multisensory integration with a head-mounted display: Sound delivery and self-motion

Human Factors

Volumn 50, Issue 5, 2008, Pages 789-800

  Thompson, Matthew B ^a   Sanderson, Penelope M ^a  

^a UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

VISUAL COMMUNICATION;

FREE FIELDS; HUMAN OPERATORS; MOBILE WORKS; MULTIPLE MODALITIES; MULTISENSORY INTEGRATIONS; RELATIVE LOCATIONS; VISUAL INFORMATIONS; WORSE PERFORMANCES;

HELMET MOUNTED DISPLAYS;

ADOLESCENT; ADULT; ARTICLE; AUDITORY SCREENING; CONTROLLED STUDY; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; MENTAL TASK; SENSORIMOTOR INTEGRATION; SENSORY STIMULATION; SITTING; SOUND DETECTION; SPATIAL DISCRIMINATION; TASK PERFORMANCE; VISUAL INFORMATION; WALKING;

ADOLESCENT; ADULT; AUDITORY PERCEPTION; DATA DISPLAY; EQUIPMENT DESIGN; HUMANS; MULTIMEDIA; PSYCHOMOTOR PERFORMANCE; USER-COMPUTER INTERFACE; VISUAL PERCEPTION; WALKING; YOUNG ADULT;

EID: 57749206617     PISSN: 00187208     EISSN: None     Source Type: Journal    
DOI: 10.1518/001872008X312323     Document Type: Article

References (49)

1. Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257-262.
2. Barrass, S., & Kramer, G. (1999). Using sonification. Multimedia Systems, 7, 23-31.
3. Behar, I., Wiley, R. W., Levine, R. R., Rash, C. E., & Walsh, D. J. (1990). Visual survey of Apache aviators (VISAA) (No. ADA230201). Fort Rucker, AL: U.S. Army Aeromedical Research Laboratory.
4. Bolia, R. S., D'Angelo, W. R., & McKinley, R. L. (1999). Aurally aided visual search in three-dimensional space. Human Factors, 41, 664-669.
5. Brenner, E., van Beers, R. J., Rotman, G., & Smeets, J. B. J. (2006). The role of uncertainty in the systematic spatial mislocalization of moving objects. Journal of Experimental Psychology: Human Perception and Performance, 32, 811-825.
6. Brewster, S. A. (1994). Providing a structured method for integrating non-speech audio into human-computer interfaces (Doctoral dissertation, University of York, 1990). Dissertation Abstracts International, 57, 1340.
7. Calvert, G. A., Spence, C., & Stein, B. E. (2004). The handbook of multi-sensory processes. Cambridge, MA: MIT Press.
8. Driver, J., & Spence, C. (1994). Spatial synergies between auditory and visual attention. In C. Umilta & M. Moscovitch (Eds.), Attention and performance XV (pp. 311-331). Cambridge, MA: MIT Press.
9. Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433.
10. Fendrich, R., & Corballis, P. M. (2001). The temporal cross-capture of audition and vision. Perception and Psychophysics, 63, 719-725.
11. Giard, M. H., & Peronnet, F. (1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electro-physiological study. Journal of Cognitive Neuroscience, 11, 473-490.
12. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.
13. Guski, R., & Troje, N. F. (2003). Audiovisual phenomenal causality. Perception and Psychophysics, 65, 789-800.
14. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload (pp. 139-183). Amsterdam: North-Holland.
15. Jack, C. E., & Thurlow, W. R. (1973). Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. Perceptual and Motor Skills, 37, 967-979.
16. Jaekl, P. M., & Harris, L. R. (2007). Auditory-visual temporal integration measured by shifts in perceived temporal location. Neuroscience Letters, 417, 219-224.
17. Kallinen, K., & Ravaja, N. (2007). Comparing speakers versus head-phones in listening to news from a computer - Individual differences and psychophysiological responses. Computers in Human Behavior, 23, 303-317.
18. Kramer, G. (1994). An introduction to auditory display. In G. Kramer (Ed.), Auditory display: Sonification, audification, and auditory interfaces (pp. 1-78). Reading, MA: Addison Wesley.
19. Lajoie, Y., Teasdale, N., Bard, C., & Fleury, M. (1993). Attentional demands for static and dynamic equilibrium. Experimental Brain Research, 97, J39-144.
20. Laramee, R. S., & Ware, C. (2002). Rivalry and interference with a head-mounted display. ACM Transactions on Computer-Human Interaction, 9, 238-251.
21. Lewald, J., Ehrenstein, W. H., & Guski, R, (2001). Spatio-temporal constraints for auditory-visual integration. Behavioral Brain Research, 121, 69-79.
22. Lewald, J., & Guski, R. (2004). Auditory-visual temporal integration as a function of distance: No compensation for sound-transmission time in human perception. Neuroscience Letters, 357, 119-122.
23. Liu, D., Jenkins, S., Sanderson, P. M., Leane, T., Watson, M. O., & Russell, W. J. (2008). Simulator evaluation of head-mounted displays for patient monitoring. Anesthesia and Analgesia, 106(Suppl. 4), S34.
24. Michotte, A. (1963). The perception of causality (T. Miles & E. Miles, Trans.). New York: Basic Books. (Original work published 1954)
25. Morein-Zamir, S., Soto-Faraco, S., & Kingstone, A. (2003). Auditory capture of vision: Examining temporal ventriloquism. Cognitive Brain Research, 17, 154-163.
26. Nelson, T. M., & Nilsson, T. H. (1990). Comparing headphone and speaker effects on simulated driving. Accident Analysis and Prevention, 22, 523-529.
27. Neuhoff, J. G. (Ed.). (2004). Ecological psychoacoustics. Amsterdam: Elsevier Academic Press.
28. Newell, F. N. (2004). Cross-modal object recognition. In G. A. Calvert, C. Spence, & B. E. Stein (Eds.), The handbook of multisensory processes (pp. 123-138). Cambridge, MA: MIT Press.
29. Nygren, T. E. (1991). Psychometric properties of subjective workload measurement techniques: Implications for their use in the assessment of perceived mental workload. Human Factors, 33, 17-33.
30. Ockerman, J., & Pritchett, A. (1998). Preliminary investigation of wearable computers for task guidance in aircraft inspection. In Proceedings of the International Symposium on Wearable Computers (ISWC '98) (pp. 33-40). Pittsburgh, PA: CS Press.
31. Ong, S. K., Yuan, M. L., & Nee, A. Y. C. (2008). Augmented reality applications in manufacturing: A survey. International Journal of Production Research, 46, 2707-2742.
32. Ormerod, D. F., Ross, B., & Naluai-Ceechini, A. (2003). Use of an augmented reality display of patient monitoring data to enhance anesthesiologists' response to abnormal clinical events. Studies in Health Technology and Informatics, 94, 248-250.
33. Patterson, R, Winterbottom, M. D., & Pierce, B. J. (2006). Perceptual issues in the use of head-mounted visual displays. Human Factors, 48, 555-573.
34. Perrott, D. R., Cisneros, J., McKinley, R. L., & D'Angelo, W. R. (1996). Aurally aided visual search under virtual and free-field listening conditions. Human Factors, 38, 702-715.
35. Radeau, M. (1994). Auditory-visual spatial interaction and modularity. Current Psychology of Cognition, 13, 3-51.
36. Roibas, A. C., Geerts, D., Furtado, E., & Calvi, L. (2008). Implications of the socio-physical contexts when interacting with mobile media. Personal and Ubiquitous Computing, 12, 279-280.
37. Sanderson, P. M., Watson, M. O., Russell, W. J., Jenkins, S., Liu, D., Green, N., et al. (2008). Advanced auditory displays and head mounted displays: Advantages and disadvantages for monitoring by the distracted anesthesiologist. Anesthesia and Analgesia, 106, 1787-1797.
38. Slutsky, D. A., & Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. Neuroreport, 12, 7-10.
39. Soto-Faraco, S., Kingstone, A., & Spence, C. (2006). Integrating motion information across sensory modalities: The role of top-down factors. Progress in Brain Research, 155, 273-286.
40. Sparrow, W. A., Bradshaw, E. J., Lamoureux, E., & Tirosh, O. (2002). Ageing effects on the attention demands of walking. Human Movement Science, 21, 961-972.
41. Spence, C., & Driver, J. (2004). Crossmodal space and crossmodal attention. New York: Oxford University Press.
42. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
43. Thurlow, W. R., & Jack, C. E. (1973). A study of certain determinants of the ventriloquism effect. Perceptual and Motor Skills, 36, 1171-1184.
44. Viaud-Delmon, I., Warusfel, O., Seguelas, A., Rio, E., & Jouvent, R. (2006). High sensitivity to multisensory conflicts in agoraphobia exhibited by virtual reality. European Psychiatry, 21, 501-508.
45. Vroomen, J., & Keetels, M. (2006). The spatial constraint in intersensory pairing: No role in temporal ventriloquism. Journal of Esperimental Psychology: Human Perception and Performance, 32, 1063-1071.
46. Walker, B. N., & Kramer, G. (2004). Ecological psychoacoustics and auditory displays: Hearing, grouping, and meaning making. In J. G. Neuhoff (Ed.), Ecological psychoacoustics (pp. 150-174). Amsterdam: Elsevier Academic Press.
47. Watson, M. O., & Sanderson, P. (2004). Sonification supports eyes-free respiratory monitoring and task time-sharing. Human Factors, 46, 497-518.
48. Watson, M. O., & Sanderson, P. (2007). Designing for attention with sound: Challenges and extensions to ecological interface design. Human Factors, 49, 331-346.
49. Yeh, M., Merlo, J. L., Wickens, C. D., & Brandenburg, D. L. (2003). Head up versus head down: The costs of imprecision, unreliability, and visual clutter on cue effectiveness for display signaling. Human Factors, 45, 390-408.