Objective assessment of speech and audio quality - Technology and applications

IEEE Transactions on Audio, Speech and Language Processing

Volumn 14, Issue 6, 2006, Pages 1890-1901

  Rix, Anthony W ^a   Beerends, John G ^b   Kim, Doh Suk ^c   Kroon, Peter ^d   Ghitza, Oded ^e  

^a Technology Partnership (TTP)   (United Kingdom)

^b TNO   (Netherlands)

^c NOKIA RESEARCH CENTER   (United States)

^d AGERE SYSTEMS   (United States)

^e SENSIMETRICS CORPORATION   (United States)

Author keywords

Audio quality; Intrusive and nonintrusive testing; Objective models; Quality assessment; Speech quality

Indexed keywords

AUDIO QUALITY; INTRUSIVE AND NONINTRUSIVE TESTING; OBJECTIVE MODELS; QUALITY ASSESSMENT; SPEECH QUALITY;

LARGE SCALE SYSTEMS; TELEPHONE; TELEPHONE CIRCUITS;

AUDIO EQUIPMENT;

EID: 34748832326     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2006.883260     Document Type: Article

References (90)

1. B. C. J. Moore, An Introduction to the Psychology of Hearing, 4th ed. Norwell, MA: Academic, 1997.
2. M. Karjalainen, "A new auditory model for the evaluation of sound quality of audio system," in Proc. IEEE ICASSP, 1985, pp. 608-611.
3. K. Brandenburg, "Evaluation of quality for audio encoding at low bit rates," in Proc. 82nd Audio Eng. Soc. Conv., 1987, preprint 2433.
4. S. R. Quackenbush, T. P. Barnwell, III, and M. A. Clements, Objective Measures of Speech Quality. Englewood Cliffs, NJ: Prentice-Hall, 1988.
5. quot;Objective quality measurement of telephone-band (300-3400 Hz) speech codecs," 1998, ITU-T P.861.
6. quot;Method for objective measurements of perceived audio quality," 1999, ITU-R BS.1387.
7. quot;Perceptual evaluation of speech quality (PESQ), an objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs," 2001, ITU-T P.862.
8. J. Liang and R. Kubichek, "Output-based objective speech quality," in Proc. IEEE Vehicular Technol. Conf., Stockholm, Sweden, 1994, pp. 1719-1723.
9. P. Gray, M. P. Hollier, and R. E. Massara, "Non-intrusive speech quality assessment using vocal tract models," Inst. Elect. Eng. Proc. Vis. Image Sig. Process., vol. 147, no. 6, pp. 493-501, 2000.
10. D.-S. Kim, "ANIQUE: An auditory model for single-ended speech quality estimation," IEEE Trans. Speech Audio Process., vol. 13, no. 5, pp. 821-831, Sep. 2005.
11. quot;Single-ended method for objective speech quality assessment in narrow-band telephony applications," 2004, ITU-T P.563.
12. quot;Analysis and interpretation of INMD voice-service measurements," 2000, ITU-T P.562.
13. A. Clark, "Description of VQMON algorithm," 2003, ITU-T del. cont. COM12-D105.
14. S. Broom, "VoIP quality assessment: Taking account of the edge-device," IEEE Trans. Audio, Speech, Language Process., vol. 14, no. 6, pp. 1977-1983, No. 2006.
15. M. Werner, T. Junge, and P. Vary, "Quality control for AMR speech channels in GSM networks," in Proc. IEEE ICASSP, 2004, pp. 1076-1079.
16. quot;The E-model, a computational model for use in transmission planning," 2002, ITU-T G.107.
17. quot;Application guide for objective quality measurement based on recommendations, P.862, P.862.1 and P.862.2," 2005, ITU-T P.862.3.
18. quot;Methods for subjective determination of transmission quality," 1996, ITU-T P.800.
19. quot;Subjective performance assessment of telephone-band and wideband digital codecs," 1996, ITU-T P.830.
20. quot;Subjective test methodology for evaluating speech communication systems that include noise suppression algorithm," 2003, ITU-T P.835.
21. J. Hall, "Application of multidimensional scaling to subjective evaluation of coded speech," in Proc. IEEE Workshop Speech Coding, 2000, pp. 20-22.
22. V.-V. Mattila, "Perceptual analysis of speech quality in mobile communications," Ph.D. dissertation, Tampere Univ. Technol., Tampere, Finland, 2001.
23. quot;Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems," 1998, ITU-R BS.1116.
24. quot;Wideband extension to recommendation for the assessment of wideband telephone networks and speech codecs," 2005, ITU-T P 862.2.
25. quot;Method for the subjective assessment of intermediate audio quality," 2003, ITU-R BS.1534-1.
26. S. R. Appel and J. G. Beerends, "On the quality of hearing one's own voice," J. Audio Eng. Soc., vol. 50, pp. 237-248, 2002.
27. N. Kitawaki, "Pure delay effects on speech quality in telecommunication," IEEE J. Sel. Areas Commun., vol. 9, no. 4, pp. 586-593, May 1991.
28. J. G. Beerends, "A subjective/objective test protocol for determining the conversational quality of a voice link," 2003, ITU-T cont. COM 12-55.
29. quot;Mean opinion score (MOS) terminology," 2003, ITU-T P.800.1.
30. A. W. Rix, "Comparison between subjective listening quality and P.862 PESQ score," in Proc. Meas. Speech Qual. Net. (MESAQIN), 2003, pp. 17-25.
31. S. Wang, A. Sekey, and A. Gersho, "An objective measure for predicting subjective quality of speech coders," IEEE J. Sel. Areas Commun., vol. 10, no. 5, pp. 819-829, Jun. 1992.
32. S. Voran, "Objective estimation of perceived speech quality-Part I: Development of the measuring normalizing block technique," IEEE Trans. Speech Audio Process., vol. 7, no. 4, pp. 371-382, 1999.
33. J. G. Beerends, A. P. Hekstra, A. W. Rix, and M. P. Hollier, "Perceptual evaluation of speech quality (PESQ), the new ITU standard for end-to-end speech quality assessment, Part II-Psychoacoustic model," J. Audio Eng. Soc., vol. 50, no. 10, pp. 765-778, 2002.
34. quot;ITU-T coded-speech database," 1998, Supp. 23 to P series rec., ITU-T.
35. SQAM [Online]. Available: http://sound.media.mit.edu/mpeg4/audio/ sqam
36. W. Yang, M. Dixon, and R. Yantorno, "A modified Bark spectral distortion measure which uses noise masking threshold," in Proc. IEEE Work. Speech Coding Telecom., 1997, pp. 55-56.
37. S. W. Park, S. K. Ryu, Y. C. Park, and D. H. Youn, "A Bark coherence function for perceived speech quality estimation," in Proc. Int. Conf. Spoken Lang. Process. (ICSLP), 2000, vol. 2, pp. 218-221.
38. T. H. Falk, Q. Xu, and W.-Y. Chan, "Non-intrusive GMM-based speech quality measurement," in Proc. IEEE ICASSP, Philadelphia, PA, 2005, pp. 125-128.
39. T. H. Falk and W.-Y. Chan, "Single-ended speech quality measurement using machine learning methods," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 6, pp. 1935-1947, Nov. 2006.
40. M. R. Schroeder, B. S. Atal, and J. L. Hall, "Optimizing digital speech coders by exploiting masking properties of the human ear," J. Acoust. Soc. Amer., vol. 66, no. 6, pp. 1647-1652, 1979.
41. J. G. Beerends and J. A. Stemerdink, "A perceptual audio quality measure based on a psychoacoustic sound representation," J. Audio Eng. Soc., vol. 40, no. 12, pp. 963-974, 1992.
42. T. Dau, B. Kollmeier, and A. Kohlrausch, "Modeling auditory processing of amplitude modulation. II. Spectral and temporal integration," J. Acoust. Soc. Am., vol. 102, no. 5, pp. 2906-2919, 1997.
43. M. Hansen and B. Kollmeier, "Using a quantitative psycho-acoustical signal representation for objective speech quality measurement," in Proc. ICASSP, 1997, pp. 1387-1390.
44. M. Hauenstein, "Application of Meddis' inner hair-cell model to the prediction of subjective speech quality," in Proc. IEEE ICASSP, 1998, pp. 545-548.
45. B. C. J. Moore, C.-T. Tan, N. Zacharov, and V.-V. Mattila, "Measuring and predicting the perceived quality of music and speech subjected to combined linear and nonlinear distortion," J. Audio Eng. Soc., vol. 52, no. 12, pp. 1228-1244, 2004.
46. R. Huber and B. Kollmeier, "PEMO-Q-A new method for objective audio quality assessment using a model of auditory perception," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 6, pp. 1902 -1911, Nov. 2006.
47. J. G. Beerends and J. A. Stemerdink, "The optimal time-frequency smearing and amplitude compression in measuring the quality of audio devices," in Proc. 94th Audio Eng. Soc. Conv., preprint 3604.
48. O. Ghitza, "Auditory models and human performance in tasks related to speech coding and speech recognition," IEEE Trans. Speech Audio Process., vol. 2, no. 1, pp. 115-132, Jan. 1994.
49. J. G. Beerends and J. A. Stemerdink, "Aperceptual speech quality measure based on a psychoacoustic sound representation," J. Audio Eng. Soc., vol. 42, no. 3, pp. 115-123, 1994.
50. J. G. Beerends, "Measuring the quality of speech and music codecs, an integrated psychoacoustic approach," in Proc. 98th Audio Eng. Soc. Conv., 1995, preprint 3945.
51. B. Paillard, P. Mabilleau, S. Morisette, and J. Soumagne, "PERCEVAL: Perceptual evaluation of the quality of audio signals," J. Audio Eng. Soc., vol. 40, no. 1/2, pp. 21-31, 1992.
52. C. Colomes, M. Lever, J. B. Rault, and Y. F. Dehery, "A perceptual model applied to audio bit-rate reduction," J. Audio Eng. Soc., vol. 43, no. 4, pp. 233-240, 1995.
53. T. Thiede and E. Kabot, "A new perceptual quality measure for bit rate reduced audio," in Proc. 100th Audio Eng. Soc. Conv., 1996, preprint 4280.
54. T. Sporer, "Objective audio signal evaluation-Applied psychoacoustics for modelling the perceived quality of digital audio," in Proc. 103rd Audio Eng. Soc. Conv., 1997, preprint 4512.
55. T. Thiede, W. C. Treurniet, R. Bitto, C. Schmidmer, T. Sporer, J. G. Beerends, C. Colomes, M. Keyhl, G. Stoll, K. Brandenburg, and B. Feiten, "PEAQ-The ITU standard for objective measurement of perceived audio quality," J. Audio Eng. Soc., vol. 48, no. 1/2, pp. 3-29, 2000.
56. S. George, S. Zielinski, and F. Rumsey, "Feature extraction for the prediction of multichannel spatial audio fidelity," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 6, pp. 1994-2005, Nov. 2006.
57. J. Barbedo and A. Lopes, "A new cognitive model for objective assessment of audio quality," J. Audio Eng. Soc., vol. 53, no. 1/2, pp. 22-31, 2005.
58. S. Zielinski, F. Rumsey, R. Kassier, and S. Bech, "Development and initial validation of a multichannel audio quality expert system," J. Audio Eng. Soc., vol. 53, no. 1/2, pp. 4-21, 2005.
59. A. De and P. Kabal, "Auditory distortion measure for speech coder evaluation-Discrimination information approach," Speech Commun., vol. 14, pp. 205-229, 1994.
60. , "Auditory distortion measure for speech coder evaluation- Hidden Markovian approach," Speech Commun., vol. 17, pp. 39-57, 1995.
61. M. P. Hollier, M. O. Hawksford, and D. R. Guard, "Error activity and error entropy as a measure of psychoacoustic significance in the perceptual domain," Inst. Elect. Eng. Proc. Vision, Image, Signal Process., vol. 141, no. 3, pp. 203-208, 1994.
62. A. W. Rix and M. P. Hollier, "The perceptual analysis measurement system for robust end-to-end speech quality assessment," in Proc. IEEE ICASSP, 2000, vol. 3, pp. 1515-1518.
63. A. W. Rix, M. P. Hollier, A. P. Hekstra, and J. G. Beerends, "Perceptual evaluation of speech quality (PESQ), the new ITU standard for end-to-end speech quality assessment, Part I-Time-delay compensation," J. Audio Eng. Soc., vol. 50, no. 10, pp. 755-764, 2002.
64. B. Novorita, "Incorporation of temporal masking effects into Bark spectral distortion measure," in Proc. IEEE ICASSP, 1999, vol. 2, pp. 665-668.
65. quot;Mapping function for transforming P.862 raw result scores to MOSLQO," 2003, ITU-T P.862.1.
66. J. Holub, M. D. Street, and R. Smid, "Intrusive speech transmission quality measurements for low bit-rate coded audio signals," in Proc. 115th Audio Eng. Soc. Conv., 2003, preprint 5954.
67. V.-V. Mattila and A. Kurittu, "Practical issues in objective speech quality assessment with ITU-T P.862," in Proc. 117th Audio Eng. Soc. Conv., 2004, preprint 6209.
68. P. Kozlowski and A. Dobrucki, "Proposed changes to the methods of objective, perceptual based evaluation of compressed speech and audio signals," in Proc. 116th Audio Eng. Soc. Conv., 2004, preprint 6085.
69. J. Salmela and V.-V. Mattila, "New intrusive method for the objective quality evaluation of acoustic noise suppression in mobile communications," in Proc. 116th Audio Eng. Soc. Conv., 2004, preprint 6145.
70. V. Gautier-Turbin and N. Le Faucheur, "A perceptual objective measure for noise reduction systems," in Proc. Meas. Speech Qual. Net. (MESAQIN), 2005, pp. 81-84.
71. quot;Head and Torso Simulator for Telephonometry," 1996, ITU-T P.58.
72. T. Goldstein, H. Klaus, J. G. Beerends, and C. Schmidmer, "Draft recommendation P.AAM-An objective method for end-to-end speech quality assessment of narrow-band telephone networks including acoustic terminal(s)," 2003, ITU-T cont. COM12-C64.
73. T. Goldstein and A. W. Rix, "Perceptual speech quality assessment in acoustic and binaural applications," in Proc. IEEE ICASSP, 2004, vol. 3, pp. 1064-1067.
74. H. Hermansky, "Perceptual linear prediction (PLP) analysis of speech," J. Acoust. Soc. Amer., vol. 87, pp. 1738-1752, 1990.
75. C. Jin and R. Kubichek, "Vector quantization techniques for outputbased objective speech quality," in Proc. IEEE ICASSP, Atlanta, GA, 1996, pp. 491-494.
76. G. Talwar and R. Kubichek, "Output based speech quality measurement using hidden Markov models," in Proc. Int. Signal Process. Conf., Dallas, TX, 2003.
77. J. G. Beerends, P. Gray, A. P. Hekstra, and M. P. Hollier, "Call for proposals for a single-ended speech quality measurement method for non-intrusive measurements on live voice traffic," 2000, ITU-T cont. COM12-C11.
78. D.-S. Kim, "A cue for objective speech quality estimation in temporal envelope representations," IEEE Signal Process. Lett., vol. 11, pp. 849-852, 2004.
79. L. Malfait, J. Berger, and M. Kastner, "P.563-The ITU-T standard for single-ended speech quality assessment," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 6, pp. 1924-1934, Nov. 2006.
80. quot;Methodology for the derivation of equipment impairment factors from instrumental models," 2002, ITU-T P.834.
81. A. Raake, "Short- and long-term packet loss behavior: Towards speech quality prediction for arbitrary loss distributions," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 6, pp. 1957-1968, Nov. 2006.
82. S. Moller, A. Raake, N. Kitawaki, and A. Takahashi, "Impairment factor framework for wideband speech codecs," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 6, pp. 1969-1976, Nov. 2006.
83. A. Karlsson, G. Heikkila, T. B. Minde, M. Nordlund, B. Timus, and N. Wiren, "Radio link parameter based speech quality index-SQI," in Proc. IEEE Int. Conf. Elect., Circuits Syst., 1999, vol. 3, pp. 1569-1572.
84. P. Gray, R. E. Massara, and M. P. Hollier, "An experimental investigation of the accumulation of perceived error in time-varying speech distortions," in Proc. 103rd Audio Eng. Soc. Conv., 1997, preprint 4588.
85. J. H. Rosenbluth, "Testing the quality of connections having time varying impairments," 1998, ITU-T del. cont. COM12-D64.
86. N. Chateau, "Continuous assessment of time-varying subjective vocal quality and its relationship with overall subjective quality," 1999, ITU-T cont. COM12-94.
87. L. Gros, N. Chateau, and S. Busson, "A comparison of speech quality judgments in laboratory and in real environment," in Proc. 114th Audio Eng. Soc. Conv., 2003, preprint 5738.
88. D. Sen, "Predicting foreground SH, SL and BNH DAMscores for multidimensional objective measure of speech quality," in ICASSP, 2004, pp. 493-496.
89. J. G. Beerends, P. C. H. Oudshoorn, and J. M. van Vugt, "Speech degradation decomposition using a P.862 PESQ based approach," 2004, ITU-T cont. COM 12-C4.
90. A. W. Rix, A. Bourret, and M. P. Hollier, "Modelling human perception," BT Tech. J., vol. 17, no. 1, pp. 24-34, 1999