Joint physical-layer and system-level power management for delay-sensitive wireless communications

IEEE Transactions on Mobile Computing

Volumn 12, Issue 4, 2013, Pages 694-709

  Mastronarde, Nicholas ^a   Van Der Schaar, Mihaela ^b  

^a State University of New York   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

adaptive modulation and coding; dynamic power management; Energy efficient wireless communications; Markov decision process; power control; reinforcement learning

Indexed keywords

ADAPTIVE MODULATION AND CODING; CHANNEL CONDITIONS; CHANNEL STATISTICS; CONVENTIONAL REINFORCEMENT LEARNING; DELAY CONSTRAINTS; DELAY SENSITIVE; DYNAMIC POWER MANAGEMENT; ENERGY EFFICIENT; INFORMATION NEED; MARKOV DECISION PROCESSES; MULTIMEDIA DATA; ON DYNAMICS; ONLINE METHODS; ORDERS OF MAGNITUDE; PARTIAL INFORMATION; PHYSICAL LAYERS; POINT-TO-POINT TRANSMISSIONS; PRIORI KNOWLEDGE; RUN-TIME PERFORMANCE; STOCHASTIC OPTIMIZATION PROBLEMS; SYSTEM LEVELS; SYSTEM-LEVEL POWER MANAGEMENT; THREE ORDERS OF MAGNITUDE; UNIFIED FRAMEWORK; WIRELESS COMMUNICATIONS;

ADAPTIVE MODULATION; ENERGY EFFICIENCY; ENERGY UTILIZATION; FADING CHANNELS; MARKOV PROCESSES; NETWORK LAYERS; OPTIMIZATION; REINFORCEMENT LEARNING; STOCHASTIC SYSTEMS; WIRELESS TELECOMMUNICATION SYSTEMS;

LEARNING ALGORITHMS;

EID: 84874587259     PISSN: 15361233     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMC.2012.36     Document Type: Article

References (36)

1. D. Rajan, A. Sabharwal, and B. Aazhang, "Delay-Bounded Packet Scheduling of Bursty Traffic over Wireless Channels," IEEE Trans. Information Theory, vol. 50, no. 1, pp. 125-144, Jan. 2004.
2. R. Berry and R.G. Gallager, "Communications over Fading Channels with Delay Constraints," IEEE Trans. Information Theory, vol. 48, no. 5, pp. 1135-1149, May 2002. (Pubitemid 34555222)
3. L. Benini, A. Bogliolo, G.A. Paleologo, and G. De Micheli, "Policy Optimization for Dynamic Power Management," IEEE Trans. Computer-Aided Design of Integrated Circuits, vol. 18, no. 6, pp. 813- 833, June 1999.
4. E.-Y. Chung, L. Benini, A. Bogliolo, Y.-H. Lu, and G. De Micheli, "Dynamic Power Management for Nonstationary Service Requests," IEEE Trans. Computers, vol. 51, no. 11, pp. 1345-1361, Nov. 2002. (Pubitemid 35421768)
5. Z. Ren, B.H. Krogh, and R. Marculescu, "Hierarchical Adaptive Dynamic Power Management," IEEE Trans. Computers, vol. 54, no. 4, pp. 409-420, Apr. 2005. (Pubitemid 40556085)
6. K. Nahrstedt, W. Yuan, S. Shah, Y. Xue, and K. Chen, "QoS Support in Multimedia Wireless Environments," Multimedia over IP and Wireless Networks, M. van der Schaar and P. Chou, eds., Academic Press, 2007.
7. J.G. Proakis, Digital Communications. McGraw-Hill, 2001.
8. C. Schurgers, "Energy-Aware Wireless Communications," PhD dissertation, Univ. of California, Los Angeles, 2002.
9. D.V. Djonin and V. Krishnamurthy, "MIMO Transmission Control in Fading Channels - A Constrained Markov Decision Process Formulation with Monotone Randomized Policies," IEEE Trans. Signal Processing, vol. 55, no. 10, pp. 5069-5083, Oct. 2007. (Pubitemid 47534670)
10. D. Bertsekas and R. Gallager, Data Networks. Prentice Hall, 1987.
11. R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction. MIT, 1998.
12. N. Salodkar, A. Bhorkar, A. Karandikar, V.S. Borkar, "An On-Line Learning Algorithm for Energy Efficient Delay Constrained Scheduling over a Fading Channel," IEEE J. Selected Areas in Comm., vol. 26, no. 4, pp. 732-742, Apr. 2008.
13. M.H. Ngo and V. Krishnamurthy, "Monotonocity of Constrained Optimal Transmission Policies in Correlated Fading Channels with ARQ," IEEE Trans. Signal Processing, vol. 58, no. 1, pp. 438- 451, Jan. 2010.
14. N. Mastronarde and M. van der Schaar, "Online Reinforcement Learning for Dynamic Multimedia Systems," IEEE Trans. Image Processing, vol. 19, no. 2, pp. 290-305, Feb. 2010.
15. Cisco Aironet 802.11a/b/g Wireless CardBus Adapter, http://www.cisco.com/ en/US/prod/collateral/wireless/ps6442/ps4555/ps5818/product-data- sheet09186a00801ebc29.html, 2012.
16. F. Fu and M. van der Schaar, "Structural-Aware Stochastic Control for Transmission Scheduling," technical report, http://medianetlab.ee.ucla. edu/papers/UCLATechReport-03-11-2010. pdf, 2012.
17. E. Evan-Dar and Y. Mansour, "Learning Rates for Q-Learning," J. Machine Learning Research, vol. 5, pp. 1-25, 2003.
18. P. Auer, T. Jaksch, and R. Ortner, "Near-Optimal Regret Bounds for Reinforcement Learning," Advances in Neural Information Processing Systems, vol. 21, pp. 89-96, 2009.
19. Q. Liu, S. Zhou, and G.B. Giannakis, "Queuing with Adaptive Modulation and Coding over Wireless Links: Cross-Layer Analysis and Design," IEEE Trans. Wireless Comm., vol. 4, no. 3, pp. 1142-1153, May 2005. (Pubitemid 40683182)
20. D. Krishnaswamy, "Network-Assisted Link Adaptation with Power Control and Channel Reassignment in Wireless Networks," Proc. 3G Wireless Conf., pp. 165-170, 2002.
21. K.-B. Song and S.A. Mujtaba, "On the Code-Diversity Performance of Bit-Interleaved Coded OFDM in Frequency-Selective Fading Channels," Proc. IEEE Vehicular Technology Conf., vol. 1, pp. 572- 576, 2003.
22. A. Goldsmith and S.-G. Chua, "Adaptive Coded Modulation for Fading Channels," IEEE Trans. Comm., vol. 46, no. 5, pp. 595-602, May 1998. (Pubitemid 128739419)
23. E. Altman, Constrained Markov Decision Processes. Chapman and Hall/CRC Press, 1999.
24. V.S. Borkar and S.P. Meyn, "The ODE Method for Convergence of Stochastic Approximation and Reinforcement Learning," SIAM J. Control Optimization, vol 38, pp. 447-469, 1999.
25. D.P. Bertsekas, Dynamic Programming and Optimal Control, third ed. Athena Scientific, 2005.
26. W.B. Powell, Approximate Dynamic Programming: Solving the Curse of Dimensionality, second ed. Wiley, 2011.
27. M. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, 1994.
28. E. Altman, Constrained Markov Decision Processes. Chapman and Hall/CRC, 1999.
29. V.S. Borkar, "An Actor-Critic Algorithm for Constrained Markov Decision Processes," Systems and Control Letters, vol. 54, pp. 207- 2013, 2005.
30. N. Mastronarde, F. Verde, D. Darsena, A. Scaglione, and M. van der Schaar, "Transmitting Important Bits and Sailing High Radio Waves: A Decentralized Cross-Layer Approach to Cooperative Video Transmission," IEEE J. Selected Areas Comm., vol. 30, no. 9, pp. 1597-1604, http://arxiv.org/abs/ 1102.5437v2, Oct. 2012.
31. N. Salodkar, A. Karandikar, V.S. Borkar, "A Stable Online Algorithm for Energy-Efficient Multiuser Scheduling," IEEE Trans. Mobile Computing, vol. 9, no. 10, pp. 1391-1406, Oct. 2010.
32. C. Poellabauer and K. Schwan, "Energy-Aware Traffic Shaping for Wireless Real-Time Applications," Proc. 10th Real-Time and Embedded Technology and Applications Symp., pp. 48-55, May 2004.
33. E. Shih, P. Bahl, and M.J. Sinclair, "Wake on Wireless: An Event Driven Energy Saving Strategy for Battery Operated Devices," Proc. ACM MobiCom, 2002.
34. N. Mastronarde and M. van der Schaar, "Fast Reinforcement Learning for Energy-Efficient Wireless Communications," technical report, http://arxiv.org/abs/1009.5773, 2012.
35. D.S. Turaga and T. Chen, "Hierarchical Modeling of Variable Bit Rate Video Sources," Packet Video Workshop, May 2001.
36. N. Mastronarde and M. van der Schaar, "Fast Reinforcement Learning for Energy-Efficient Wireless Communications," technical report, http://arxiv.org/abs/1009.5773, 2012.