A survey on delay-aware resource Control for wireless systems-large deviation theory, stochastic lyapunov drift, and distributed stochastic learning

IEEE Transactions on Information Theory

Volumn 58, Issue 3, 2012, Pages 1677-1701

  Cui, Ying ^a   Lau, Vincent K N ^a   Wang, Rui ^b   Huang, Huang ^b   Zhang, Shunqing ^b  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^b HUAWEI TECHNOLOGIES CO LTD   (China)

Author keywords

Delay aware resource control; large deviation theory; Lyapunov stability; Markov decision process (MDP); stochastic learning

Indexed keywords

DECISION THEORY; FREQUENCY DIVISION MULTIPLE ACCESS; MARKOV PROCESSES; SURVEYS; WIRELESS NETWORKS;

LARGE DEVIATION THEORY; LYAPUNOV STABILITY; MARKOV DECISION PROCESSES; RESOURCE CONTROL; STOCHASTIC LEARNING;

STOCHASTIC SYSTEMS;

EID: 84863264023     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2178150     Document Type: Review

References (70)

1. P. Svedman, S. K. Wilson, L. J. Cimini, and B. Ottersten, "Opportunistic beamforming and scheduling for OFDMA systems," IEEE Trans. Commun., vol. 55, no. 5, pp. 941-952, May 2007.
2. M. Pischella and J.-C. Belfiore, "Power control in distributed cooperative OFDMA cellular networks," IEEE Trans. Wireless Commun., vol. 7, no. 5, pp. 1900-1906, May 2008.
3. D. P. Palomar, J. M. Cioffi, and M. A. Lagunas, "Joint tx-rx beamforming design for multicarrier MIMO channels: A unified framework for convex optimization," IEEE Trans. Signal Process., vol. 51, no. 9, pp. 2381-2401, Sep. 2003.
4. D. P. Palomar, M. A. Lagunas, and J. M. Cioffi, "Optimum linear joint transmit-receive processing forMIMO channels with QoS constraints," IEEE Trans. Signal Process., vol. 52, no. 5, pp. 1179-1197, May 2004.
5. Y. Yi and M. Chiang, "Stochastic network utility maximization: A tribute to Kelly's paper published in this journal a decade ago," Eur. Trans. Telecommun., vol. 19, no. 4, pp. 421-442, Jun. 2008.
6. X. Lin, N. B. Shroff, and R. Srikant, "A tutorial on cross-layer optimization in wireless networks," Eur. Trans. Telecommun., vol. 24, no. 8, pp. 1452-1463, Jun. 2006.
7. D.Wu and R. Negi, "Effective capacity: A wireless link model for support of quality of service," IEEE Trans. Wireless Commun., vol. 2, no. 4, pp. 630-643, Jul. 2003.
8. D. S. W. Hui, V. K. N. Lau, and H. L. Wong, "Cross-layer design for OFDMA wireless systems with heterogeneous delay requirements," IEEE Trans. Wireless Commun., vol. 6, no. 8, pp. 2872-2880, Aug. 2007.
9. J. Tang and X. Zhang, "Cross-layer resource allocation over wireless relay networks for quality of service provisioning," IEEE J. Sel. Areas Commun., vol. 25, no. 4, pp. 2318-2328, May 2007.
10. J. Tang and X. Zhang, "Quality-of-service driven power and rate adaptation over wireless links," IEEE Trans. Wireless Commun., vol. 6, no. 8, pp. 3058-3068, Aug. 2007.
11. J. Tang and X. Zhang, "Cross-layer-model based adaptive resource allocation for statistical qos guarantees in mobile wireless networks," IEEE Trans. Wireless Commun., vol. 7, no. 6, pp. 2318-2328, Jun. 2008.
12. M. Tao, Y.-C. Liang, and F. Zhang, "Resource allocation for delay differentiated traffic in multiuser OFDM systems," IEEE Trans. Wireless Commun., vol. 7, no. 6, pp. 2190-2201, Jun. 2008.
13. L. Georgiadis, M. J. Neely, and L. Tassiulas, "Resource allocation and cross-layer control in wireless networks," Found. Trends Network., vol. 1, no. 1, pp. 1-144, 2006.
14. S. Shakkottai and A. Stolyar, "Scheduling for multiple flows sharing a time varying channel: The exponential rule," Anal. Methods Appl. Probabil., vol. 207, pp. 185-202, 2003.
15. B. Sadiq, S. Baek, and G. de Veciana, "Delay-optimal opportunistic scheduling and approximations: The log rule," in Proc. IEEE Conf. Comput. Commun., Apr. 2009, pp. 1692-1700.
16. M. Andrews, K. Kumaran, K. Ramanan, A. Stolyar, R. Vijayakumar, and P. Whiting, "Scheduling in a queueing system with asynchronously varying service rates," Probabil. Eng. Informat. Sci., vol. 18, no. 2, pp. 191-217, 2004.
17. P.-P. Bhattacharya, L. Georgiadis, P. Tsoucas, and I. Viniotis, "Adaptive lexicographic optimization in multi-class M/GI/1 queues," Math. Operat. Res., vol. 18, no. 3, pp. 705-740, 1993.
18. I. Keslassy and N. McKeown, "Analysis of scheduling algorithms that provide 100 percent throughput in input-queued switches," in Proc. 39th Annu. Allerton Conf. Commun., Control, Comput., Oct. 2001.
19. P. R. Kumar and S. P. Meyn, "Stability of queueing networks and scheduling policies," IEEE Trans. Automat. Control, vol. 40, no. 2, pp. 251-260, Feb. 1995.
20. E. Leonardi, M. Mellia, F. Neri, and M. A. Marsan, "Bounds on average delays and queue size averages and variances input-queued cellbased switches," in Proc. IEEE Conf. Comput. Commun., 2001, pp. 1095-1103.
21. N. McKeown, A. Mekkittikul, V. Anantharam, and J. Walrand, "Achieving 100 percent throughput in an input-queued switch," IEEE Trans. Commun., vol. 47, no. 8, pp. 1260-1267, Aug. 1999.
22. L. Tassiulas and A. Ephremides, "Dynamic sever allocation to parallel queues with randomly varying connectivity," IEEE Trans. Inf. Theory, vol. 39, no. 2, pp. 466-478, Mar. 1993.
23. M. Andrews, K. Kumaran, K. Ramanan, A. Stolyar, and P. Whiting, "Providing quality of service over a shared wireless link," IEEE Commun. Mag., vol. 39, no. 2, pp. 150-154, Feb. 2001.
24. M. Andrews, K. Kumaran, K. Ramanan, A. Stolyar, and P. Whiting, "Power allocation and routing in multibeam satellites with time-varying channels," IEEE/ACM Trans. Network., vol. 11, no. 1, pp. 138-152, Feb. 2001.
25. M. J. Neely, "Dynamic power allocation and routing for satellite and wireless networks with time varying channels," Ph.D. dissertation, Dept. Elect. Eng. Comput. Sci. ,Mass Inst. Technol, Cambridge, 2003.
26. M. J. Neely, E. modiano, and C. E. Rohrs, "Dynamic power allocation and routing for time varing wireless networks," IEEE J. Sel. Areas Commun., vol. 23, no. 1, pp. 89-103, Jan. 2005.
27. M. J. Neely, "Energy optimal control for time-varying wireless networks," IEEE Trans. Inf. Theory, vol. 52, no. 7, pp. 2915-2934, Jul. 2006.
28. R. A. Berry and R. Gallager, "Communication over fading channels with delay constraints," IEEE Trans. Inf. Theory, vol. 48, no. 5, pp. 1135-1148, May 2002.
29. M. J. Neely, "Optimal energy and delay tradoffs for multi-user wireless downlinks," IEEE Trans. Inf. Theory, vol. 53, no. 9, pp. 3095-3113, Sep. 2007.
30. E. M. Yeh, "Multiaccess and fading in communication networks," Ph.D. dissertation, Dept. Elect. Eng. Comput. Sci., Mass Inst. Technol., Cambridge, 2001.
31. E. M. Yeh and A. S. Cohen, "Throughput and delay optimal resource allocation in multiaccess fading channels," in Proc. IEEE Int. Symp. Inf. Theory, Jul. 2003, pp. 245-245.
32. D. P. Bertsekas, Dynamic Programming and Optimal Control, 3rd ed. Belmont, MA: Athena Scientific, 2007.
33. X. Cao, Stochastic Learning and Optimization: A Sensitivity-Based Approach. New York: Springer-Verlag, 2008.
34. L. Kleinrock, Queueing Systems. Volume 1: Theory. London, U.K.: Wiley-Interscience, 1975.
35. C.-S. Chang, "Stability, queue length, and delay of deterministic and stochastic queueing networks," IEEE Trans. Automat. Control, vol. 39, no. 5, pp. 913-931, May 1994.
36. C.-S. Chang and J.-A. Thomas, "Effective bandwidth in high-speed digital networks," IEEE J. Sel. Areas Commun., vol. 13, no. 6, pp. 1091-1100, Aug. 1995.
37. J. G. Kim and M. Krunz, "Bandwidth allocation in wireless networks with gauranteed packet-loss performance," IEEE/ACM Trans. Network., vol. 8, no. 3, pp. 337-349, Jun. 2000.
38. G. Kesidis, J. Walrand, and C.-S. Chang, "Effective bandwidths for multiclass markov fluids and other atm sources," IEEE/ACM Trans. Network., vol. 1, no. 4, pp. 424-428, Aug. 1993.
39. A. Shwartz and A.Weiss, Large Deviations For Performance Analysis: QUEUES, Communication and Computing. London, U.K.: Chapman & Hall, 1995.
40. S. Meyn and R. Tweedie, Markov Chains and Stochastic Stability, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 2009.
41. S. Asmussen, Applied Probability and Queues, 2nd ed. New York: Spring-Verlag, 2003.
42. L. Tassiulas and A. Ephremides, "Stability properties of constrained queueing systems and scheduling for maximum throughput in multihop radio networks," IEEE Trans. Automat. Control, vol. 37, no. 12, pp. 1936-1949, Dec. 1992.
43. S. Asmussen, Applied Probability and Queues. New York: Wiley, 1987.
44. S. Meyn and R. Tweedie, Markov Chains and Stochastic Stability. New York: Springer-Verlag, 1993.
45. P. R. Kumar and S. P. Myen, "Duality and linear programs for stability and performance analysis of queueing networks and scheduling policies," IEEE Trans. Automat. Control, vol. 41, no. 1, pp. 4-17, Jan. 1996.
46. M. J. Neely, E. Modiano, and C. E. Rohrs, "Power allocation and routing in multibeam satellites with time-varying channels," IEEE/ACM Trans. Network., vol. 11, no. 1, pp. 138-152, Feb. 2003.
47. H. J. Kushner and G. G. Yin, Stochastic Approxiamtion and Optimization of Random Systems, 2nd ed. New York: Springer-Verlag, 2003.
48. V. S. Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge, U.K.: Cambridge Univ. Press, 2008.
49. R. E. Bellman and S. E. Dreyfus, "Functional approximation and dynamic programming, 13," Math. Tables Other Aids Comput., pp. 247-251, 1959.
50. W. Whitt, "Approximations of dynamic programs i," Math. Operat. Res., vol. 3, pp. 231-243.
51. D. Reetz, "Approximate solutions of a discounted Markovian decision process," Bonner Mathematische Schriften, 98: Dynamische Optimierung, pp. 77-92, 1977.
52. P. J. Schweitzer and A. Seidmann, "Generalized polynomial approximations in markovian decision processes," J. Math. Anal. Appl., vol. 110, pp. 568-582, 1985.
53. B. Sadiq, S. J. Baek, and G. de Veciana, "Delay-optimal opportunistic scheduling and approximations: The log rule," in Proc. 28th Conf. Comput. Commun., 2009, pp. 1692-1700.
54. B. Ata, "Dynamic power control in a wireless static channel subject to a quality-of-service constraint," Oper. Res., no. 5, pp. 842-851, Sep. 2005.
55. J. M. George and J. M. Harrison, "Dynamic control of a queue with adjustable service rate," Oper. Res., no. 5, pp. 720-731, Sep. 2001.
56. T. B. Crabill, "Optimal control of a service facility with variable exponential service times and constant arrival rate," Manage. Sci, pp. 560-566, 1972.
57. L. Bui, R. Srikant, and A. L. Stolyar, "Novel architectures and algorithms for delay reduction in back-pressure scheduling and routing," in IEEE INFOCOM Mini-Conf., Apr. 2009, pp. 2936-2940.
58. L. Ying, S. Shakkottai, and A. Reddy, "On combining shortest-path and back-pressure routing over multihop wireless networks," in IEEE INFOCOM, Apr. 2009, pp. 1674-1682.
59. L. Huang and M. J. Neely, "Delay reduction via lagrange multipliers in stochastic network optimization," IEEE Trans. Automat. Control, vol. 56, no. 4, pp. 842-857, Apr. 2011.
60. S. Moeller, A. Sridharan, B. Krishnamachari, and O. Gnawali, "Routing without routes: The backpressure collection protocol," in 9th ACM/IEEE Int. Conf. Inf. Process. Sens. Netw., 2010, pp. 279-290.
61. L. Huang, S. Moeller, M. J. Neely, and B. Krishnamachari, "Lifobackpressure achieves near optimal utility-delay tradeoff," WiOpt, pp. 842-857, May 2011.
62. M. Naghshvar and T. Javidi, "Opportunistic routing with congestion diversity in wireless multi-hop networks," in IEEE INFOCOM Mini- Conf., May 2010, pp. 1-5.
63. M. Naghshvar and T. Javidi, "Opportunistic routing with congestion diversity and tunable overhead," in 4th Int. Symp. Commun., Control Signal Process., Mar. 2010, pp. 1-6.
64. M. J. Neely and R. Urgaonkar, "Optimal backpressure routing for wireless networks with multi-receiver diversity," Ad Hoc Netw., vol. 7, no. 5, pp. 862-881, Jul. 2009.
65. S. Biswas and R. Morris, "ExOR: Opportunistic multi-hop routing for wireless networks," ACM SIGCOMM, pp. 133-144, Oct. 2005.
66. D. S. J. D. Couto, D. S. J. De, C. Daniel, R. Morris, D. Aguayo, and J. Bicket, "A high-throughput path metric for multi-hop wireless routing," Wireless Networks, Volume 11, Number 4, pp. 419-434, vol. 11, no. 4, pp. 419-434, 2003.
67. D. Palomar and M. Chiang, "Alternative distributed algorithms for network utility maximization: Framework and applications," IEEE Trans. Automat. Control, vol. 52, no. 12, pp. 2254-2269, Dec. 2007.
68. J. Abounadi, D. Bertsekas, and V. S. Borkar, "Learning algorithms for Markov decision processes with average cost," SIAM J. Control Optim., vol. 40, pp. 681-698, 1998.
69. V. S. Borkar, "Asynchronous stochastic approximation," SIAM J. Control Optim., vol. 36, pp. 840-851, 1998.
70. V. S. Borkar and S. P. Meyn, "TheODE method for convergence of stochastic approximation and reinforcement learning algorithms," SIAM J. Control Optim., vol. 38, pp. 447-469, 2000.