Online learning based congestion control for adaptive multimedia transmission

IEEE Transactions on Signal Processing

Volumn 61, Issue 6, 2013, Pages 1460-1469

  Habachi, Oussama ^a   Shiang, Hsien Po ^b   Van Der Schaar, Mihaela ^c   Hayel, Yezekael ^a  

^a UNIVERSITY OF AVIGNON   (France)

^b CISCO SYSTEMS   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Congestion control; learning technology; multimedia communication

Indexed keywords

ADAPTIVE MULTIMEDIA TRANSMISSION; ADDITIVE INCREASE MULTIPLICATIVE DECREASE; LEARNING TECHNOLOGY; MULTI-MEDIA COMMUNICATIONS; MULTIMEDIA APPLICATIONS; MULTIMEDIA TRANSMISSIONS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; TCP CONGESTION CONTROL;

CONGESTION CONTROL (COMMUNICATION); E-LEARNING; MULTIMEDIA SYSTEMS; OPTIMIZATION; REINFORCEMENT LEARNING;

TRANSMISSION CONTROL PROTOCOL;

EID: 84875012268     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2237171     Document Type: Article

References (28)

1. B. Wang, J. Kurose, P. Shenoy, and D. Towsley, "Multimedia streaming via TCP: An analytic performance study," ACM Trans. Multimedia Comput., Commun., Appl., vol. 4, pp. 1-22,May 2008.
2. A. Balk, M. Gerla, D. Maggiorini, and M. Sanadidi, "Adaptive video streaming: Pre-encoded MPEG-4 with bandwidth scaling," Comput. Netw., vol. 44, pp. 415-439, Mar. 2004.
3. L. Cai, X. Shen, J. Pan, and J.W.Mark, "Performance analysis of TCPfriendly AIMD algorithm for multimedia applications," IEEE Trans. Multimedia, vol. 7, pp. 339-355, Apr. 2005.
4. D. Bansal and H. Balakrishnan, "Binomial congestion control algorithm," in Proc. IEEE INFOCOM, 2001, pp. 631-640.
5. A. Cassandra, M. Littmann, and N. Zhang, "Incremental pruning: A simple, fast, exact method for partially observable Markov decision process," in Proc. UAI, 1997, pp. 54-61.
6. M. L. Littman, "The Witness Algorithm: Solving partially observable Markov decision processes," Comput. Sci. Dept., Brown Univ., Providence, RI, Tech. Rep. CS-94-40, 1994.
7. L. Chrisman, "Reinforcement learning with perceptual aliasing: The perceptual distinctions approach," in Proc. 10th Nat. Conf. Artif. Intell., 1992, pp. 183-188.
8. R. A. McCallum, "First results with utile distinction memory for reinforcement learning," presented at the Mach. Conf., Rochester, NY, USA, 1993.
9. C. J. C. H. Watkins, "Learning from delayed rewards," Ph.D. dissertation, Univ. of Cambridge, King's College, Cambridge, U.K., 1989.
10. D. E. Rumelhart, "Learning internal representations by error backpropagation," Parallel Distrib. Process., pp. 318-362, 1986.
11. H.-P. Shiang and M. van der Schaar, "A quality-centric TCP-friendly congestion control for multimedia transmission," IEEE Trans. Multimedia, vol. 14, no. 3-2, pp. 896-909, 2012.
12. R. Rejaie, M. Handley, and D. Estrin, "Rap: An end-to-end rate-based congestion control mechanisme for real-time stream in internet," in Proc. INFOCOM, 1999, pp. 1337-1345.
13. F. Fu and M. van der Schaar, "Structural solutions for dynamic scheduling in wireless multimedia transmission," IEEE Trans. Circuits Syst. Video Technol., vol. 22, p. 22, May 2012.
14. H. Shiang and M. van der Schaar, "Multi-user video streaming over multi-hop wireless networks: A distributed cross-layer approach based on priority queuing," in IEEE J. Sel. Areas Commun., May 2007, vol. 25, pp. 770-785.
15. P. Chou and Z. Miao, "Rate-distortion optimized streaming of packetized media," IEEE Trans. Multimedia, vol. 8, pp. 390-404, Apr. 2006.
16. M. van der Schaar and D. Turaga, "Cross-layer packetization and retransmission strategies for delay-sensitive wireless multimedia transmission," IEEE Trans. Multimedia, vol. 9, pp. 185-197, Jan. 2007.
17. S. Floyd and K. Fall, "Promoting the use of end-to-end congestion control in the internet," IEEE/ACM Trans. Netw., vol. 7, pp. 458-472, Aug. 1999.
18. J. Padhye, V. Firoiu, D. F. Towsley, and J. F. Kurose, "A model based TCP-friendly rate control protocol," in Proc. NOSSDAV, 1999.
19. S. Floyd, M. Handley, J. Padhye, and J.Widmer, "Equation-based congestion control for unicast applications," in ACM SIGCOMM Comput. Commun. Rev., Oct. 2000, vol. 30, pp. 43-56.
20. M. L. Putterman, Markov Decision Process Discrete Stochastic Dynamic Programming, ser. Wiley Series in Probability and Statistique. Hoboken, NJ, USA: Wiley, 2005.
21. R. D. Smallwood and E. J. Sondik, "The optimal control of partially observable Markov decision processes over a finite horizon," Oper. Res., vol. 21, pp. 1071-1088, 1973.
22. E. Sondik, "The optimal control of partially observable Markov processes over the infinite horizon: Discounted cost," Oper. Res., vol. 26, pp. 282-304, 1978.
23. N. L. Zhang and W. Zhang, "Speeding up the convergence of value iteration in partially observable Markov decision process," J. Artif. Intell. Res., vol. 14, pp. 29-51, 2001.
24. R. S. Sutton, "Generalization in reinforcement learning: Successful examples using sparse coarse coding," Adv. Neural Inf. Process. Syst., pp. 1038-1045, 1996.
25. J. Pineau, G. Gordon, and S. Thrun, "Point-based value iteration: An anytime algorithm for POMDPS," in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2003, pp. 1025-1030.
26. K. W. Lee, R. Puri, T. Kim, K. Ramchandran, and V. Bharghavan, "An integrated source coding and congestion control framework for video streaming in the internet," in Proc. IEEE INFOCOM, Mar. 2000, pp. 747-756.
27. D. P. Bertsekas and D. A. Castanon, "Adaptive aggregation methods for infinite horizon dynamic programming," IEEE Trans. Autom. Control, vol. 34, no. 6, pp. 589-598, 1989.
28. S. Singh, T. Jaakkola,M. L. Littman, and C. Szepesvari, "Convergence results for single-step on-policy reinforcement-learning algorithms," Mach. Learn., vol. 38, no. 3, pp. 287-308, 2000.