Structure-aware stochastic control for transmission scheduling

IEEE Transactions on Vehicular Technology

Volumn 61, Issue 9, 2012, Pages 3931-3945

  Fu, Fangwen ^a   Van Der Schaar, Mihaela ^b  

^a INTEL CORPORATION   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Delay sensitive communications; energy efficient data transmission; Markov decision processes (MDPs); scheduling; stochastic control

Indexed keywords

CHANNEL STATISTICS; COMPUTATION COMPLEXITY; DELAY SENSITIVE; ENERGY EFFICIENT; ENERGY TRADEOFF; LOW-COMPLEXITY; LOW-STORAGE; MARKOV DECISION PROCESSES; MONOTONICITY; ONLINE LEARNING; ONLINE LEARNING ALGORITHMS; ONLINE METHODS; ONLINE OPTIMIZATION; OPTIMAL SCHEDULING; OPTIMAL SOLUTIONS; PIECE-WISE LINEAR FUNCTIONS; PRIORI KNOWLEDGE; REAL-TIME TRANSMISSIONS; STATE-VALUE FUNCTIONS; STOCHASTIC CONTROL; TIME VARYING CHANNEL; TRANSMISSION SCHEDULING;

DATA COMMUNICATION SYSTEMS; LEARNING ALGORITHMS; MARKOV PROCESSES; OPTIMAL SYSTEMS; PIECEWISE LINEAR TECHNIQUES; STOCHASTIC CONTROL SYSTEMS; STRUCTURAL PROPERTIES;

SCHEDULING;

EID: 84869390996     PISSN: 00189545     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVT.2012.2213850     Document Type: Article

References (33)

1. R. Berry and R. G. Gallager, "Communications over fading channels with delay constraints," IEEE Trans. Inf. Theory, vol. 48, no. 5, pp. 1135-1149, May 2002.
2. W. Chen, M. J. Neely, and U. Mitra, "Energy-efficient transmission with individual packet delay constraints," IEEE Trans. Inf. Theory, vol. 54, no. 5, pp. 2090-2109, May 2008.
3. M. Goyal, A. Kumar, and V. Sharma, "Optimal cross-layer scheduling of transmissions over a fading multiacess channel," IEEE Trans. Inf. Theory, vol. 54, no. 8, pp. 3518-3536, Aug. 2008.
4. M. Agarwal, V. Borkar, and A. Karandikar, "Structural properties of optimal transmission policies over a randomly varying channel," IEEE Trans. Autom. Control, vol. 53, no. 6, pp. 1476-1491, Jul. 2008.
5. D. Djonin and V. Krishnamurthy, "Structural results on optimal transmission scheduling over dynamical fading channels: A constrained Markov decision process Approach," in Wireless Communications, G. Yin, Ed. New York: Springer Verlag, 2006, pp. 75-98.
6. T. Holliday, A. Goldsmith, and P. Glynn, "Optimal power control and source-channel coding for delay constrained traffic over wireless channels," in Proc. IEEE Int. Conf. Commun., May 2002, vol. 2, pp. 831-835.
7. A. Jalali, R. Padovani, and R. Pankaj, "Data throughput of CDMA-HDR: A high-efficiency data rate personal communication wireless system," in Proc. IEEE Veh. Technol. Conf., May 2000, pp. 1854-1858.
8. L. Tassiulas and A. Ephremides, "Stability properties of constrained queuing systems and scheduling policies for maximum throughput in multihop radio networks," IEEE Trans. Autom. Control, vol. 37, no. 12, pp. 1936-1949, Dec. 1992.
9. P. R. Kumar and S. P. Meyn, "Stability of queuing networks and scheduling policies," IEEE Trans. Autom. Control, vol. 40, no. 2, pp. 251-260, Feb. 1995.
10. A. Stolyar, "Maximizing queuing network utility subject to stability: Greedy primal-dual algorithm," Queueing Syst., Theory Appl., vol. 50, no. 4, pp. 401-457, Aug. 2005.
11. L. Georgiadis, M. J. Neely, and L. Tassiulas, "Resource allocation and cross-layer control in wireless networks," Found. Trends Netw., vol. 1, no. 1, pp. 1-144, Apr. 2006.
12. F. Fu and M. van der Schaar, "Decomposition principles and online learning in cross-layer optimization for delay-sensitive applications," IEEE Trans. Signal Process., vol. 58, no. 3, pp. 1401-1415, Mar. 2010.
13. D. V. Djonin and V. Krishnamurthy, "Q-learning algorithms for constrained Markov decision processes with randomized monotone policies: Application toMIMO transmission control," IEEE Trans. Signal Process., vol. 55, no. 5, pp. 2170-2181, May 2007.
14. N. Salodkar, A. Bhorkar, A. Karandikar, and V. S. Borkar, "An online learning algorithm for energy-efficient delay-constrained scheduling over a fading channel," IEEE J. Sel. Areas Commun., vol. 26, no. 4, pp. 732-742, May 2008.
15. E. Altman, Constrained Markov Decision Processes. London, U.K.: Chapman & Hall, 1999.
16. D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Belmont, MA: Athena Scientific, 2005.
17. Q. Zhang and S. A. Kassam, "Finite-state Markov model for Rayleigh fading channels," IEEE Trans. Commun., vol. 47, no. 11, pp. 1688-1692, Nov. 1999.
18. D. Bertsekas and R. Gallager, Data Networks. Upper Saddle River, NJ: Prentice-Hall, 1987.
19. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
20. V. S. Borkar and S. P. Meyn, "The ODE method for convergence of stochastic approximation and reinforcement learning," SIAM J. Control Optim., vol. 38, no. 2, pp. 447-469, Jan. 2000.
21. A. Siem, D. Hertog, and A. Hoffmann, "A method for approximating univariate convex functions using only function value evaluations," SSRN, 2007. [Online]. Available: http://ssrn.com/abstract=1012289
22. A. Siem, D. Hertog, and A. Hoffmann, "Multivariate convex approximation and least-norm convex data-smoothing," in Proc. ICCSA, 2006, vol. 3, pp. 812-821.
23. W. Powell, A. Ruszcznski, and H. Topaloglu, "Learning algorithms for separable approximation of discrete stochastic optimization problems," Math. Oper. Res., vol. 29, no. 4, pp. 814-836, Nov. 2004.
24. F. Fu andM. van der Schaar, "Structural solutions for dynamic scheduling in wireless multimedia transmission," IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 5, pp. 727-739, May 2012.
25. F. Fu and M. van der Schaar, "A systematic framework for dynamically optimizing multiuser video transmission," IEEE J. Sel. Areas Commun., vol. 28, no. 3, pp. 308-320, Apr. 2010.
26. L. Kleinrock, Queueing Systems. New York: Wiley, 1975.
27. F. Fu and M. van der Schaar. (2010, Mar.). "Structural-Aware stochastic control for transmission scheduling," UCLA, Los Angeles, CA, Tech. Rep. [Online]. Available: http://medianetlab.ee.ucla.edu/papers/ UCLATechReport-03-11-2010.pdf
28. H. Li and K. Womer, "Stochastic resource-constrained project scheduling and its military applications," MORS Phalanx, Mar. 2011. [Online]. Available: www.umsl.edu/~lihait/files/PHALANX-Li%20and%20Womer.pdf
29. J. Ma andW. B. Powell. (2010, May). "Convergence analysis of on-policy LSPI for multidimensional continuous state and action-space MDPs and extension with orthogonal polynomial approximation," Princeton Univ., Princeton, NJ, Tech. Rep. [Online]. Available: http://www.castlelab. princeton.edu/Papers/Ma%20Powell%20-%20Online%20convergence% 20for%20high%20dimensional%20MDP.pdf
30. V. B. Tadic, A. Doucet, and S. Singh, "Two-Time-scale stochastic approximation for constrained stochastic optimization and constrained Markov decision problems," in Proc. Amer. Control Conf., Jun. 2003, vol. 6, pp. 4736-4741.
31. C. Watkins and P. Dayan, "Q-learning: Technical note," Mach. Learn., vol. 8, no. 3/4, pp. 279-292, May 1992.
32. E. Altman, B. Gaujal, and A. Hordijk, "Multimodularity, convexity and optimization properties," Math. Oper. Res., vol. 25, no. 2, pp. 324-347, May 2000.
33. D.P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Belmont, MA: Athena Scientific, 2007.