MIMO transmission control in fading channels - A constrained Markov decision process formulation with monotone randomized policies

IEEE Transactions on Signal Processing

Volumn 55, Issue 10, 2007, Pages 5069-5083

  Djonin, Dejan V ^a,b   Krishnamurthy, Vikram ^b  

^a Dyaptive Systems   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Constrained Markov decision process; Markovian channels; Monotone policies; Multimodularity; Power control; Randomized policies; Rate control; Stochastic dominance; Submodularity; Transmission scheduling

Indexed keywords

CHANNEL STATE INFORMATION; FADING CHANNELS; LAGRANGE MULTIPLIERS; LEARNING ALGORITHMS; MARKOV PROCESSES; PIECEWISE LINEAR TECHNIQUES; WIRELESS TELECOMMUNICATION SYSTEMS;

CONSTRAINED MARKOV DECISION PROCESS; MULTIPLE-INPUT MULTIPLE-OUTPUT SYSTEM; RATE ALLOCATION; TRANSMISSION SCHEDULING;

DATA COMMUNICATION SYSTEMS;

EID: 35148825865     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.897859     Document Type: Article

References (32)

1. G. J. Foschini and M. J. Gans, "On limits of wireless communication in a fading environment when using multiple antennas," Wireless Personal Commun., pp. 311-335, Mar. 1998.
2. G. J. Foschini, "Layered space-time architecture for wireless communication in a fading environment when using multielement antennas," Bell. Labs. Tech. J., pp. 41-59, 1996.
3. S. Chung, H. C. Howard, and A. Lozano, "Low complexity algorithm for rate quantization in extended V-BLAST," in Proc. IEEE Vehicular Technology Conf. (VTC 2001 Fall), 2001, pp. 910-914.
4. H. Zhang, L. Dai, S. Zhou, and Y. Yao, "Low complexity per-antenna rate and power control approach for closed-loop V-BLAST," IEEE Trans. Commun., vol. 51, no. 11, pp. 1783-1787, Nov. 2003.
5. B. E. Collins and R. Cruz, "Transmission policies for time varying channels with average delay constraints," presented at the Allerton Conf. Communications, Control, Computers, Monticello, IL, Jul. 1999.
6. A. K. Karmokar, D. V. Djonin, and V. K. Bhargava, "Optimal and suboptimal packet scheduling over time-varying fading channels," IEEE Trans. Wireless Commun., vol. 5, no. 2, pp. 446-457, Feb. 2006.
7. M. J. Hossain, D. V. Djonin, and V. K. Bhargava, "Delay limited optimal and suboptimal power and bit loading algorithms for OFDM systems over correlated fading," in Proc. IEEE GLOBECOM 2005, 2005, pp. 3448-3453.
8. B. Prabhakar, E. Uysal-Biyikoglu, and A. E. Gamal, "Energy-efficient transmission over a wireless link via lazy packet scheduling," in Proc. INFOCOM, 2001, pp. 386-394.
9. L. Johnston and V. Krishnamurthy, "Opportunistic file transfer over a fading channel - A POMDP search theory formulation with optimal threshold policies," IEEE Trans. Wireless Commun., vol. 44, no. 1, pp. 394-405, Feb. 2006.
10. R. Berry and R. Gallager, "Communication over fading channels with delay constraints," IEEE Trans. Inf. Theory, vol. 48, no. 5, pp. 1135-1149, May 2002.
11. D. Rajan, A. Subharwal, and B. Aazhang, "Delay and rate constrained transmission policies over wireless channels," in Proc. IEEE GLOBECOM Conf., 2001, pp. 806-810.
12. M. Goyal, A. Kumar, and V. Sharma, "Power constrained and delay optimal policies for scheduling transmission over a fading channel," in Proc. INFOCOM, 2003, pp. 311-320.
13. H. S. Wang and N. Moayeri, "Finite-state Markov channel - A useful model for radio communication channels," IEEE Trans. Veh. Technol. pp. 163-171, Feb. 1995.
14. E. Altman, Constrained Markov Decision Processes: Stochastic Modeling London, U.K.: Chapman & Hall CRC, 1999.
15. E. Altman, B. Gaujal, and A. Hordijk, Discrete-Event Control of Stochastic Networks: Multimodularity and Regularity. Berlin, Germany: Springer, 2003.
16. D. M. Topkis, Supermodularity and Complementarity. Princeton, NJ: Princeton Univ. Press, 1998.
17. M. L. Putterman, Markov Decision Procsses: Discrete Stochastic Dynammic Programming. New York: Wiley, 1994.
18. F. V. Abad and V. Krishnamurthy, "Constrained stochastic approximation algorithms for adaptive control of constrained Markov decision processes," in Proc. 42nd IEEE Conf. Decision Control, 2003, pp. 2823-2828.
19. D. V. Djonin and V. Krishnamurthy, "Q-learning algorithms for constrained Markov decision processes with randomized monotone policies: Application to MIMO transmission control," IEEE Trans. Signal Process., vol. 55, no. 5, pt. 11, pp. 2170-2181, May 2007.
20. M. J. Sobel and D. P. Heyman, Stochastic Models in Operations Research, Volume II: Stochastic Optimization, 2nd ed. New York: Dover, 2003.
21. D. Rajan, A. Sabharwal, and B. Aazhang, "Delay-bounded packet scheduling of bursty traffic over wireless channels," IEEE Trans. Inf. Theory, vol. 50, no. 1, pp. 125-144, Jan. 2004.
22. D. Gore, R. W. Heath, Jr., and A. Paurlraj, "On performance of the zero forcing receiver in presence of transmit correlation," in Proc. IEEE Int. Symp. Information Theory 2002, 2000, p. 159.
23. D. P. Bertsekas, Dynamic Programming and Optimal Control. Belmont, MA: Athena Scientific, 1996, vol. 2.
24. S. T. Chung and A. J. Goldsmith, "Degrees of freedom in adaptive modulation: A unified view," IEEE Trans. Commun., vol. 49, no. 9, pp. 1561-1571, Sep. 2001.
25. D. Kivanc, G. Li, and H. Liu, "Computationally efficient bandwidth allocation and power control of OFDMA," IEEE Trans. Wireless Commun. vol. 2, no. 6, pp. 1150-1158, Nov. 2003.
26. H. Wang and N. B. Mandayam, "A simple packet-transmission scheme for wireless data over fading channels," IEEE Trans. Commun., vol. 52, no. 7, pp. 1055-1059, Jul. 2004.
27. F. J. Beutler and K. W. Ross, "Optimal policies for controlled Markov chains with a constraint," J. Math. Anal. Appl., vol. 112, pp. 236-252, 1985.
28. K. W. Ross, "Randomized and past-dependent policies for Markov decision processes with multiple constraints," Oper. Res., vol. 37, pp. 474-477, 1987.
29. J. E. Smith and K. F. McCardle, "Structural properties of stochastic dynamic programs," Oper. Res., pp. 796-809, Sep.-Oct. 2002.
30. F. Yu and V. Krishnamurthy, "Optimal joint session admission control in integrated WLAN and CDMA cellular network," IEEE Trans. Mobile Comput., vol. 6, no. 1, pp. 126-139, Jan. 2007.
31. F. Yu, V. Krishnamurthy, and V. Leung, "Cross-layer optimal connection admission control for variable bit rate multimedia traffic in packet wireless CDMA networks," IEEE Trans. Signal Process., vol. 54, no. 2, pp. 542-555, Feb. 2006.
32. M. H. Ngo and V. Krishnamurthy, "On optimality of monotone channel-aware transmission policies: A constrained Markov decision process approach," presented at the IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), Honolulu, HI, Apr. 2007.