The influence of content, task and sensory interaction on multimedia quality perception

Ergonomics

Volumn 51, Issue 2, 2008, Pages 85-97

  Rimell, A N ^a   Mansfield, N J ^a   Hands, D ^b  

^a LOUGHBOROUGH UNIVERSITY   (United Kingdom)

^b MARTLESHAM HEATH   (United Kingdom)

Author keywords

Auditory perception; Multimedia system design; Multimodal perception; Visual perception

Indexed keywords

ERROR DETECTION; HUMAN ENGINEERING; IMAGE QUALITY; MULTIMEDIA SYSTEMS; STATISTICAL METHODS;

MULTIMODAL PERCEPTION; SENSORY INTERACTION;

SENSORY PERCEPTION;

ADULT; ARTICLE; AUDITORY STIMULATION; FEMALE; HEARING; HUMAN; HUMAN EXPERIMENT; IMAGE QUALITY; MALE; MULTIMEDIA; PERCEPTION; QUALITY CONTROL; SELECTIVE ATTENTION; SENSORIMOTOR FUNCTION; SENSORY STIMULATION; STANDARD; STATISTICAL ANALYSIS; VISION; VISUAL STIMULATION;

ADULT; ATTENTION; AUDITORY PERCEPTION; EQUIPMENT DESIGN; FEMALE; HEARING; HUMANS; MALE; MIDDLE AGED; MODELS, THEORETICAL; MULTIMEDIA; TAPE RECORDING; TASK PERFORMANCE AND ANALYSIS; VIDEOTAPE RECORDING; VISION; VISUAL PERCEPTION;

EID: 38649105488     PISSN: 00140139     EISSN: 13665847     Source Type: Journal    
DOI: 10.1080/00140130701526432     Document Type: Article

References (41)

1. Beerends, J. G. (1999) The influence of video quality on perceived audio quality and vice versa. Journal of the Audio Engineering Society, 47, pp. 355-362.
2. Chen, T. (2001) Audiovisual speech processing - Lip reading and lip synchronization. IEEE Signal Processing Magazine, 18, pp. 9-21.
3. Chen, T. and Rao, R. (1998) Audio-visual integration in multimodal communication. Proceedings of the IEEE, 86, pp. 837-852.
4. Coen, M. H. (2001) Multimodal integration - A biological view. Proceedings of the International Joint Conference on Artificial Intelligence Kaufmann, pp. 1417-1424. San Francisco, CA
5. Driver, J. (1996) Enhancement of selective listening by illusory mislocation of speech sounds due to lip reading. Nature, 381, pp. 66-68.
6. Duncan, J., Martens, S. and Ward, R. (1997) Restricted attentional capacity within but not between sensory modalities. Nature, 387, pp. 808-810.
7. Ernst, M. O. and Bülthoff, H. H. (2004) Merging the senses into a robust percept. Trends in Cognitive Sciences, 8, pp. 162-169.
8. Frater, M. R., Arnold, J. F. and Vahedian, A. (2001) Impact of audio on subjective assessment of video quality in videoconferencing applications. IEEE Transactions on Circuits and Systems for Video Technology, 11, pp. 1059-1062.
9. Fuster, J. M., Bodner, M. and Kroger, J. K. (2000) Cross-modal and cross-temporal association in neurons of frontal cortex. Nature, 405, pp. 347-351.
10. Ghanbari, M. (2003) Standard Codecs: Image Compression to Advanced Video Coding, IEE, London
11. Hekstra, A. P., Beerends, J. G., Ledermann, D., De Caluwe, F. E., Kohler, S., Koenen, R. H., Rihs, S., Ehrsam, M. and Schlauss, D. (2002) PVQM - A perceptual video quality measure. Signal Processing: Image Communication, 17, pp. 781-798.
12. Hollier, M. P., Rimell, A. N., Hands, D. S. and Voelcker, R. (1999) Multimodal perception. BT Technology Journal, 17, pp. 35-46.
13. Hollier, M. P. and Voelcker, R. (1997) Objective performance assessment: Video quality as an influence on audio perception. Presented at the 103rd Audio Engineering Society Convention The Audio Engineering Society, New York
14. Hollier, M. P. and Voelcker, R. (1997) Towards a multimodal perceptual model. BT Technology Journal, 15, pp. 162-171.
15. International Organization for Standardization (1993) MPEG-1: Coding of Moving Pictures and Associated Audio for Digital Storage Media at Up To About 1.5 Mbit/s, ISO, Geneva
16. International Organization for Standardization (2000) MPEG-2: Generic Coding of Moving Pictures and Associated Audio Information: Systems, ISO, Geneva
17. International Organization for Standardization (2001) MPEG-4: Coding of Audio-Visual Objects, ISO, Geneva
18. International Telecommunications Union (1995) Studio Encoding Parameters of Digital Television for Standard 4:3 and Wide-Screen 16:9 Aspect Ratios, ITU, Geneva
19. International Telecommunications Union (1996) Principles of a Reference Impairment System for Video, ITU, Geneva
20. International Telecommunications Union (1996) Subjective Video Quality Assessment Methods for Multimedia Applications, ITU, Geneva
21. International Telecommunications Union (2002) Methodology for the Subjective Assessment of the Quality of Television Pictures, ITU, Geneva
22. Jordan, T. R., McCotter, M. V. and Thomas, S. M. (2000) Visual and audiovisual speech perception with color and gray-scale facial images. Perception and Psychophysics, 62, pp. 1394-1404.
23. Kanwisher, N., Dermott, J. M. and Chunn, M. (1997) The fusiform face area: A module in human extrastriate cortex specialised for face perception. Journal of Neuroscience, 11, pp. 4302-4311.
24. King, A. J. and Calvert, G. A. (2001) Perceptual grouping by eye and ear. Current Biology, 11, pp. R322-R325.
25. Lansing, C. R. and McConkie, G. W. (2003) Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Perception and Psychophysics, 65, pp. 536-552.
26. Larsen, A., McIlhagga, W., Baert, J. and Bundesen, C. (2003) Seeing or hearing? Perceptual independence, modality confusions, and crossmodal congruity effects with focused and divided attention. Perception and Psychophysics, 65, pp. 568-574.
27. McGurk, H. and MacDonald, J. (1976) Hearing lips and seeing voices. Nature, 264, pp. 746-748.
28. Patel, D. and Turner, L. F. (2000) Effects of ATM network impairments on audio-visual broadcast applications. IEE Proceedings - Vision, Image and Signal Processing, 147, pp. 436-444.
29. Rimell, A. N. and Hollier, M. (1998) The influence of cross-modal interaction on audio-visual speech quality perception. Presented at the 105th Audio Engineering Society Convention The Audio Engineering Society, New York
30. Rimell, A. N. and Hollier, M. (1999) The significance of cross-modal interaction in audio-visual quality perception. Proceedings of the 1999 IEEE International Workshop on Multimedia Processing IEEE, pp. 508-514. New York
31. Rimell, A. N. and Owen, A. (2000) The effect of focused attention on audio-visual quality perception with applications in multimodal codec design. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing 2000 IEEE, pp. 2377-2380. New York
32. Rix, A. W., Bourret, A. and Hollier, M. P. (1999) Models of human perception. BT Technology Journal, 17, pp. 24-34.
33. Sharma, R., Pavlovic, V. I. and Huang, T. S. (1998) Toward a multimodal human-computer interface. Proceedings of the IEEE, 86, pp. 853-869.
34. Spence, C. and Ranson, J. (2000) Cross-modal selective attention: On the difficulty of ignoring sounds at the locus of visual attention. Perception and Psychophysics, 62, pp. 410-424.
35. Stein, B. E. and Meredith, M. A. (1993) The Merging of the Senses, MIT, Cambridge, MA
36. Sumby, W. H. and Pollack, I. (1954) Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, pp. 212-215.
37. Voelcker, R., Hands, D., Davis, A., Bourret, A. and Bayart, D. (2003) Video quality measurement - Predicting subjective quality using perceptual models. Journal of the Communications Network, 1, pp. 40-45.
38. Ward, L. M., Mcdonald, J. J. and Lin, D. (2000) On asymmetries in cross-modal spatial attention orienting. Perception and Psychophysics, 62, pp. 1258-1264.
39. Whittaker, S. and Walker, M. (1991) Toward a theory of multimodal interaction. AAAI Workshop on Multimodal Interaction AAAI, pp. 78-85. Menlo Park, CA
40. Wuerger, S. M., Hofbauer, M. and Meyer, G. F. (2003) The integration of auditory and visual motion signals at threshold. Perception and Psychophysics, 65, pp. 1188-1196.
41. Yarbus, A. L. (1973) Eye Movements and Vision, Plenum, New York