Quality of experience of VoIP service: A survey of assessment approaches and open issues

IEEE Communications Surveys and Tutorials

Volumn 14, Issue 2, 2012, Pages 491-513

  Jelassi, Sofiene ^a   Rubino, Gerardo ^a   Melvin, Hugh ^b   Youssef, Habib ^c   Pujolle, Guy ^d  

^a INRIA   (France)

^b NATIONAL UNIVERSITY OF IRELAND   (Ireland)

^c Higher Institute of Computer Science and Communication Technologies (ISITC)   (Tunisia)

^d UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

Author keywords

Mobile heterogeneous networks; Parametric SQA algorithms; Quality of users' Experience (QoE); Speech quality assessment (SQA); VoIP

Indexed keywords

ASSESSMENT APPROACHES; AUTOMATIC ASSESSMENT; BUILDING BLOCKES; BURSTINESS; HAND OVER; IP NETWORKS; MOBILE DATA NETWORKS; MOBILE HETEROGENEOUS NETWORKS; OBJECTIVE SPEECH QUALITY; PRACTICAL PROCEDURES; QUALITY DEGRADATION; QUALITY OF EXPERIENCE (QOE); SINGLE-ENDED; SPEECH QUALITY ASSESSMENT; TELECOM NETWORKS; TIME VARYING; TRANSIENT LOSS; TRANSPORT NETWORKS; VOICE CONVERSATION; VOIP; VOIP APPLICATIONS; VOIP SERVICE;

ALGORITHMS; DEGRADATION; HETEROGENEOUS NETWORKS; INTERNET TELEPHONY; MATHEMATICAL MODELS; PARAMETER ESTIMATION; QUALITY OF SERVICE; RATING; SURVEYS;

VOICE/DATA COMMUNICATION SYSTEMS;

EID: 84860883368     PISSN: None     EISSN: 1553877X     Source Type: Journal    
DOI: 10.1109/SURV.2011.120811.00063     Document Type: Review

References (78)

1. J. Hassan, S. K. Das, M. Hassan, C. Bisdikian, and D. Soldani, "Improving quality of experience for network services," IEEE Network, vol.24, pp.4-6, Mar.2010.
2. H. Koumaras, F. Liberal, and L. Sun, "Quality of experience issues in multimedia provision," Telecommunication Systems, pp.1-3, Jun.2010.
3. P. Brooks and B. Hestnes, "User measures of quality of experience: why being objective and quantitative is important," IEEE Network, vol.24, no.2, pp.8-13, Mar.2010.
4. K.-T. Chen, C.-J. Chang, C.-C. Wu, Y.-C. Chang, and C.-L. Lei, "Quadrant of euphoria: a crowdsourcing platform for qoe assessment," IEEE Network, vol.24, no.2, pp.28-35, Mar.2010.
5. A. Raake, Speech Quality of VoIP: Assessment and Prediction. John Wiley & Sons,2006.
6. ITU-T. (2010) Itu-t study group12 (study period2009-2012). [Online]. Available: http://www.itu.int/ITU-T/studygroubs/com12/index.asp
7. G. Scheets, M. Parperis, and R. Singh, "Voice over the internet: A tutorial discussing problems and solutions associated with alternative transport," IEEE Commun. Surveys Tutorials, vol.6, no.1-4, pp.22-31,2004.
8. I. Telegeography. (2010, Mar.) International carriers' traffic grows despite skype popularity. [Online]. Available: www.telegeography.com
9. J. Ellis and C. P. J. Rahman, Voice, video, and data network convergence: architecture and design, from VoIP to wireless. Academic Press,2003.
10. Y. Pan, "Digital audio compression," Digital Technical Journal, vol. 5, no.2, pp.1-14,1993.
11. H. Schulzrinne and E. Wedlund, "Application-layer mobility using sip," SIGMOBILE Mob. Comput. Commun. Rev., vol.4, pp.47-57, Jul.2000.
12. B. Li, M. Hamdi, D. Jiang, and X.-R. Cao, "Qos-enabled voice support in the next-generation internet: Issues, existing approaches, and challenges," IEEE Commun. Mag., vol.38, no.4, pp. 54-61, Apr.2000.
13. V. Srivastava, , and M. Motani, "Cross-layer design: a survey and the road ahead," IEEE Commun. Mag., vol.43, no.12, pp.112-119, Dec.2005.
14. L. Roychoudhuri, "Proactive rate and error control for packet multimedia transmissions based on loss prediction," Ph.D. dissertation, School of Computer Science, Telecommunications and Information Systems, DePaul University, May2008.
15. Z. Li, L. Sun, Z. Qiao, and E. Ifeachor, "Perceived speech quality driven retransmission mechanism for wireless voip," in Proc. IEEE Fourth International Conference on3G Mobile Communication Technologies, London, UK, Jun.25-27,2003, pp.395-399.
16. A. W. Rix, J. G. Beerends, D.-S. Kim, P. Kroon, and O. Ghitza, "Objective assessment of speech and audio quality-technology and applications," IEEE Trans. Audio Speech Language Processing, vol.14, no.6, pp.1890-1901,2006.
17. Methods for Subjective Determination of Transmission Quality, ITU-T Std. Recommendation P.800, May1996.
18. S. Quackenbush, T. Burnwell, and M. Clements, Objective Measures of Speech Quality. Prentice Hall,1988.
19. Continuous Evaluation of Time-Varying Speech Quality, ITU-T Std. Recommendation P.880,2005.
20. Study the Relationship between Instantaneous and Overall Subjective Speech Quality for Time-Varying Speech Sequence: Influence of a Recency Effect, ITU-T Std. ITU Study Group12, Contribution D.139 (France Telecom),2000.
21. One-Way Transmission Delay, ITU-T Std. Recommendation G.114,2003.
22. N. Kitawaki and K. Itoh, "Pure delay effects on speech quality in telecommunications," IEEE J. Sel. Areas Commun., vol.9, no.4, pp. 586-593, May1991. (Pubitemid 21645616)
23. L. Sun and E. Ifeachor, "New methods for voice quality evaluation for ip networks," in Proc.18-th International Teletraffic Congress (ITC18), Berlin, Germany, Aug.31 Sep 052003, pp.395-399.
24. K.-T. Chen, C.-C. Tu, and W.-C. Xiao, "Oneclick: A framework for measuring network quality of experience," in Proc. of INFOCOM2009, Rio de Janeiro, Brazil, Apr.19-25,2009, pp.702-710.
25. L. Sun and E. C. Ifeachor, "Voice quality prediction models and their application in voip networks," IEEE Trans. Multimedia, vol.8, no.4, pp.809-820,2006. (Pubitemid 44144164)
26. S. Voran, "Objective estimation of perceived speech quality, part ii: Evaluation of the measuring normalizing block technique," IEEE Trans. Speech Audio Processing, vol.7, no.4, pp.383-390, Jul.1999.
27. N. Kitawaki, H. Nagabuchi, and K. Itoh, "Objective evaluation for lowbit-rate coding systems," IEEE J. Sel. Areas Commun., vol.6, no.2, pp.242-248, Feb.1988. (Pubitemid 18596866)
28. S. Rein, F. H. P. Fitzek, and M. Reisslein, "Voice quality evaluation in wireless packet communication systems: a tutorial and perfomance results for rohc," IEEE Wireless Commun., vol.12, no.1, pp.60-67, Feb.2005. (Pubitemid 40449471)
29. Single-ended method for objective speech quality assessment in narrowband telephony applications, ITU-T Std. Recommendation P.563,2004.
30. The E-Model, A Computational Model for Use in Transmission Planning, ITU-T Std. Recommendation G.107,2000.
31. N. Kitawaki, "Perspectives on multimedia quality prediction methodologies for advanced mobile and ip-based telephony," in Proc. Workshop on Wideband Speech Quality in Terminals and Networks: Assessment and Prediction, Mainz, Germany, Jun.8-9,2004.
32. A. Takahashi, "Opinion model for estimating conversational quality of voip," in Proc.32nd IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'04), Honolulu, Hawaii, USA, Apr.15-20,2004.
33. C. Hoene, "Internet telephony over wireless links," Ph.D. dissertation, Technical University of Berlin, Germany, Dec.2005.
34. S. Moller, A. Raake, N. Kitawaki, A. Takahashi, and M. Waltermann, "Impairment factor framework for wide-band speech codecs," IEEE Trans. Audio, Speech, and Language Processing, vol.14, no.6, pp.1969-1976, Nov.2006.
35. A. Takahashi, A. Kurashima, and H. Yoshino, "Objective quality methodology for estimating conversational quality in voip," IEEE Trans. Audio, Speech, and Language Processing, vol.14, no.6, pp.1984-1993,2006.
36. L. Ding, Z. Lin, A. Radwan, M. S. El-Hennawey, and R. A. Goubran, "Non-intrusive single-ended speech quality assessment in voip," Speech Communication, Elsevier, vol.49, no.6, pp.477-489,2007. (Pubitemid 46891625)
37. S. Jelassi, H. Youssef, C. Hoene, and G. Pujolle, "Voicing-aware parametric speech quality models over voip networks," in Proc.2nd IEEE Global Information Infrastructure Symposium (GIIS2009), Hammamet, Tunisia, Jun.23-26,2009.
38. G. Corporation. (2010, Mar.23) Vquad: Voice quality testing across all networks. [Online]. Available: http://www.gl.com/vquad.html
39. A. Rix. (2010, Sep. 03) Personal homepage of dr. antony rix. [Online]. Available: http://www.rixcam.com/
40. A. E. Conway and Y. Zhu, "A simulation-based methodology and tool for automating the modeling and analysis of voice-over-ip perceptual quality," Performance Evaluation, vol. 54, no.2, pp.129-147,2003.
41. J. Turunen, P. Loula, and T. Lipping, "Assessment of objective voice quality over best-effort networks," Computer Networks, vol.28, no. 5, pp. 582-588, Mar.2005. (Pubitemid 40401824)
42. A. Clark, "Modeling the effects of burst packet loss and recency on subjective voice quality," in Proc.2nd IP-Telephony Workshop (IPTel'2001), Columbia University, New York City, USA, Apr.2-3,2001.
43. 2S. Jelassi and H. Youssef, "Evom: A software based platform for voice transmission and quality assessment over wireless ad-hoc networks," in Proc.10th IEEE/ACM Annual International Symposium on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM2007), Chania, Crete Island, Greece, Oct.22-26,2007.
44. F. Kettler, N. Rohrer, and S. Poschen, "Voip speech quality test event," in Proc. 5th ETSI Speech Quality Test Event, Herzogenrath, Germany, Jan.Dec2006.
45. One-View Visualization of Speech Quality Measurement Results, ITU-T Std. Recommendation P.505,2005.
46. T. L. Gall. (2009, Mar.10) One view visualization computer program. [Online]. Available: http://www.mathworks.com/matlabcentral/fileex change/25299-itu-t-p505-one-view-visualization-diagrams-gui
47. S. Jelassi, H. Youssef, L. Sun, and G. Pujolle, "Nida: A parametric vocal quality assessment algorithm over transient connections," in Proc.12th IFIP/IEEE International Conference on Management of Multimedia and Mobile Networks and Services (MMNS2009), Venice, Italy, Oct.26-302009.
48. R. G. Cole and J. H. Rosenbluth, "Voice over ip performance monitoring," Computer Communication Review, ACM SIGCOMM, vol.31, no.2, pp. 09-24,2001.
49. B. Sat and B. W. Wah, "Analyzing voice quality in popular voip applications," IEEE Multimedia Mag., vol.16, no.1, pp.46-59, Jan./Mar.2009.
50. Provisional Planning Values for the Equipment Impairment Factor Ie, ITU-T Std. Recommendation G.113-Appendix I,2001.
51. L. Sun and E. Ifeachor, "New models for perceived voice quality prediction new models for perceived voice quality prediction optimization for voip networks," in Proc. IEEE International Conference on Communications (ICC2004), Paris, France, Jan.20-242004, pp.1478-1483.
52. L. Roychoudhuri and E. Al-Shaer, "Real-time audio quality evaluation for adaptive multimedia protocols," in Proc. Multimedia Networks and Services (MMNS2005), Spain, Oct.24-262005.
53. A. Raake, "Short-and long-term packet loss behavior: Towards speech quality prediction for arbitrary loss distributions," IEEE Trans. Audio, Speech, and Language Processing, vol.14, no.6, pp.1957-1968, Nov.2006.
54. H. Zhang, L. Xie, J. Byun, P. Flynn, and C. Shim, "Packet loss burstiness and enhancement to the e-model," in Proc. The6th IEEE International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, Towson, Maryland, USA, May23-252005.
55. S. Jelassi and G. Rubino, "A comparison study of speech quality assessment algorithms sensitive to packet loss burstiness," in Proc. IEEE Consumer Communications and Networking Conference, Las Vegas, Nevada, USA, Jan.09-122011.
56. S. R. Broom, "Voip quality assessment: Taking account of the edgedevice," IEEE Trans. Audio, Speech, And Language Processing, vol.14, no.6, pp.1977-1983, Nov.2006.
57. M. Masuda and T. Hayashi, "Non-intrusive quality monitoring method of voip speech based on network performance metrics," IEICE Trans. Communication, vol. E89-B, no.2, pp.304-312, Feb.2006.
58. S. Mohamed, F. C.-Prez, and H. Afifi, "Audio quality assessment in packet networks: an inter-subjective neural network model," in Proc.15th International Conference on Information Networking (ICOIN'01), Beppu City, Oita, Japan, Jan.31 Feb. 022001.
59. G. Rubino, P. Tirilly, and M. Varela, "Evaluating users' satisfaction in packet networks using random neural networks," in Proc.16-th International Conference on Artificial Neural Networks (ICANN2006), Athens, Greece, Sep.10-142006.
60. M. Narbutt, A. Kelly, L. Murphy, and P. Perry, "Adaptive voip playout scheduling: Assessing user satisfaction," IEEE Internet Computing, vol.9, no.4, pp.28-34, Jul./Aug.2005. (Pubitemid 41245023)
61. End-User Multimedia QoS Categories, ITU-T Std. Recommendation G.1010,2001.
62. L. Sun, "Speech quality prediction for voice over internet protocol networks," Ph.D. dissertation, School of Computing, Communications and Electronics Faculty of Technology, University of Plymouth, Dec.2004.
63. S. Jelassi, H. Youssef, C. Hoene, and G. Pujolle, "Single-ended parametric voicing-aware models for live assessment of packetized voip conversations," Telecommunications Systems, Special Issue on the Quality of Experience issues in Multimedia Provision, Jun.2010.
64. S. Jelassi and H. Youssef, "Connectivity aware instrumental approach for measuring vocal transmission quality over a wireless ad hoc network," in Proc.2nd International IEEE/IFIP Conference on New Technologies, Mobility and Security (NTMS2008), Tangier, Morocco, Nov.10-142008.
65. X. Wang, A. Patil, and W. Wang, "Voip over wireless mesh networks: Challenges and approaches," in Proc.2nd Annual ACM International Wireless Internet Conference (WICON'06), Boston, MA, USA, Aug.2-52006.
66. K. Kim and S. Hong, "Vomesh: Voice over wireless mesh networks," in Proc. IEEE Wireless Communications and Networking Conference (WCNC2006), Las Vegas, Nevada, USA, Apr.03-062006.
67. I. F. Akyildiz, W. Y. Lee, and K. Chowdhury, "Spectrum management in cognitive radio ad hoc networks," IEEE Network, vol.23, no.4, pp. 06-12, Jul./Aug.2009.
68. S. Ganguly, V. Navda, K. Kim, A. Kashyap, D. Niculescu, R. Izmailov, S. Hong, and S. Das, "Performance optimizations for deploying voip services in mesh networks," IEEE J. Sel. Areas Commun., vol.24, no.11, pp.2147-2158, Oct.2006. (Pubitemid 44711718)
69. P. C. Ng, S. C. Liew, and C. Lin, "Voice over wireless lan via ieee802.16 wireless man and ieee802.11 wireless distribution system," in Proc. International Conference on Wireless Networks, Communications and Mobile Computing (WIRELESSCOM'05), Maui, Hawaii, USA, Jun.13-162005.
70. S. Madhani, S. Shah, and A. Gutierrez, "Optimized adaptive jitter buffer design for wireless internet telephony," in Proc. IEEE GLOBECOM'07, Washington, DC, USA, Nov.25-302007.
71. P. Vidales, "A talk on user perception of mobility in ngn," in Proc. DTAG Workshop QoS and QoE monitoring, Berlin, Germany,2007.
72. S. Mller, M. Wltermann, B. Lewcio, N. Kirschnick, and P. Vidales, "Speech quality while roaming in next generation networks," in Proc. IEEE International Conference on Communications (ICC'09), Dresden, Germany, Jun.14-182009.
73. H. Melvin and L. Murphy, "Exploring the extent and impact of playout adjustments within voip applications on mos scores," in Proc. Measurement of Speech and Audio Quality in Networks (MESAQIN'05), Prague, Czech Republic, Jun.09-102005.
74. S. Voran, "Perception of temporal discontinuity impairments in coded speech-a proposal for objective estimators and some subjective test results," in Proc.2nd International Conference on Measurement of Speech and Audio Quality in Networks, Prague, Czech Republic, Jun.17-182003.
75. F. Hammer, P. Reichl, and A. Raake, "Elements of interactivity in telephone conversations," in Proc.8th International Conference on Spoken Language Processing (ICSLP /INTERSPEECH2004), Jeju Island, Korea, Oct.04-082004.
76. K. Chen, C.-Y. Huang, P. Huang, and C.-L. Lei, "Qunatifying skype user satisfaction," in Proc. ACM SIGCOMM'06, Pisa, Italy, Sep.11-152006.
77. Perceptual Evaluation of Speech Quality (PESQ): An Objective Method for End-to-end Speech Quality Assessment of Narrow-Band Telephone Networks and Speech Codecs, ITU-T Std. Recommendation P.862,2001.
78. B. Sat and B. W.Wah, "Analysis and evaluation of the skype and googletalk voip systems," in Proc.2006 IEEE International Conference on Multimedia and Expo (ICME2006), Toronto, Ontario, Canada, Jul.09-12,2006, pp.2153-2156.