Design of a Q-learning-based client quality selection algorithm for HTTP adaptive video streaming

AAMAS 2013 Workshop on Adaptive and Learning Agents, ALA 2013

Volumn , Issue , 2013, Pages

  Claeys, Maxim ^a   Latré, Steven ^a   Famaey, Jeroen ^a   Wu, Tingyao ^b   Van Leekwijck, Werner ^b   De Turck, Filip ^a  

^a GHENT UNIVERSITY   (Belgium)

^b LUCENT TECHNOLOGIES   (United States)

Author keywords

Agent systems; HTTP adaptive streaming; Reinforcement learning

Indexed keywords

HTTP; INTELLIGENT AGENTS; MULTIMEDIA SERVICES; QUALITY OF SERVICE; VIDEO STREAMING;

ADAPTIVE STREAMING; ADAPTIVE VIDEO STREAMING; AGENT SYSTEMS; DETERMINISTIC HEURISTICS; NETWORK CONFIGURATION; NETWORK ENVIRONMENTS; NETWORKING ENVIRONMENT; QUALITY OF EXPERIENCE (QOE);

REINFORCEMENT LEARNING;

EID: 84942646620     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Adobe. HTTP Dynamic Streaming: Flexible delivery of on-demand and live video streaming. Online. http://www.adobe.com/products/hds-dynamic-streaming.html, Last accessed: Dec. 2012.
2. V. Adzic, H. Kalva, and B. Furht. Optimized adaptive HTTP streaming for mobile devices. Applications of Digital Image Processing XXXIV, page 81350T, 2011.
3. R. Bagnasco and J. Serrat. Multi-agent Reinforcement Learning in Network Management. In Scalability of Networks and Services, Volume 5637 of Lecture Notes in Computer Science, pages 199-202. Springer Berlin Heidelberg, 2009.
4. N. Bouten, J. Famaey, S. Latre, R. Huysegems, B. De Vleeschauwer, W. Van Leekwijck, and F. De Turck. QoE optimization through in-network quality adaptation for HTTP Adaptive Streaming. In Network and Service Management (CNSM), 2012 8th International Conference on, pages 336-342. IEEE, 2012.
5. J. Cao. Using reinforcement learning for agent-based network fault diagnosis system. In Information and Automation (ICIA), 2011 IEEE International Conference on, pages 750-754, June 2011.
6. D. Jarnikov and T. Özçelebi. Client intelligence for adaptive streaming solutions. In Multimedia and Expo (ICME), 2010 IEEE International Conference on, pages 1499-1504, July 2010.
7. L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 4: 237-285, 1996.
8. J. Klaue, B. Rathke, and A. Wolisz. EvalVid - A Framework for Video Transmission and Quality Evaluation. In Proc. of the 13th International Conference on Modelling Techniques and Tools for Computer Performance Evaluation, pages 255-272, 2003.
9. M. Lee, D. Marconett, X. Ye, and S. Yoo. Cognitive Network Management with Reinforcement Learning for Wireless Mesh Networks. In IP Operations and Management, Volume 4786 of Lecture Notes in Computer Science, pages 168-179. Springer Berlin Heidelberg, 2007.
10. C. Liu, I. Bouazizi, and M. Gabbouj. Parallel Adaptive HTTP Media Streaming. In Computer Communications and Networks (ICCCN), 2011 Proceedings of 20th International Conference on, pages 1-6, Aug. 2011.
11. T. Lohmar, T. Einarsson, P. Frojdh, F. Gabin, and M. Kampmann. Dynamic adaptive HTTP streaming of live content. In World of Wireless, Mobile and Multimedia Networks (WoWMoM), 2011 IEEE International Symposium on a, pages 1-8, June 2011.
12. Microsoft. Smooth Streaming: The Official Microsoft IIS Site. Online. http://www.iis.net/downloads/microsoft/smooth-streaming, Last accessed: Dec. 2012.
13. R. Mok, E. Chan, and R. Chang. Measuring the Quality of Experience of HTTP video streaming. In Integrated Network Management (IM), 2011 IFIP/IEEE International Symposium on, pages 485-492, May 2011.
14. O. Nemethova, M. Ries, M. Zavodsky, and M. Rupp. PSNR-Based Estimation of Subjective Time-Variant Video Quality for Mobiles. In Proc. of MESAQIN 2006, 2006.
15. D. Niyato and E. Hossain. A Radio Resource Management Framework for IEEE 802.16-Based OFDM/TDD Wireless Mesh Networks. In Communications, 2006. ICC '06. IEEE International Conference on, Volume 9, pages 3911-3916, June 2006.
16. ns 3. The Network Simulator ns-3. Online. http://www.nsnam.org, Last accessed: Dec. 2012.
17. N. Ouferhat and A. Mellouk. A QoS Scheduler Packets for Wireless Sensor Networks. In Computer Systems and Applications, 2007. AICCSA '07. IEEE/ACS International Conference on, pages 211-216, May 2007.
18. R. Pantos and W. May. HTTP Live Streaming. Online. http://tools.ietf.org/html/draft-pantos-http-live-streaming-10, Last accessed: Dec. 2012.
19. H. Riiser, P. Vigmostad, C. Griwodz, and P. Halvorsen. Bitrate and video quality planning for mobile streaming scenarios using a GPS-based bandwidth lookup service. In Multimedia and Expo (ICME), 2011 IEEE International Conference on, pages 1-6, July 2011.
20. T. Stockhammer. Dynamic adaptive streaming over HTTP: standards and design principles. In Proceedings of the second annual ACM conference on Multimedia systems, MMSys '11, pages 133-144, 2011.
21. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). The MIT Press, Mar. 1998.
22. G. Tesauro, N. K. Jong, R. Das, and M. N. Bennani. On the use of hybrid reinforcement learning for autonomic resource allocation. Cluster Computing, 10(3): 287-299, Sept. 2007.
23. M. Tokic. Adaptive e-greedy exploration in reinforcement learning based on value differences. In Proceedings of the 33rd annual German conference on Advances in artificial intelligence, KI'10, pages 203-210. Springer-Verlag, 2010.
24. M. Tokic and G. Palm. Value-difference based exploration: adaptive control between epsilon-greedy and softmax. In Proceedings of the 34th Annual German conference on Advances in artificial intelligence, KI'11, pages 335-346. Springer-Verlag, 2011.
25. R. Trestian, A.-N. Moldovan, O. Ormond, and G.-M. Muntean. Energy consumption analysis of video streaming to Android mobile devices. In Network Operations and Management Symposium (NOMS), 2012 IEEE, pages 444-452, Apr. 2012.
26. D. Vengerov. A Reinforcement Learning Approach to Dynamic Resource Allocation. Technical report, Sun Microsystems Laboratories, 2005.
27. C. Watkins. Learning from Delayed Rewards. PhD thesis, University of Cambridge, England, 1989.