An application of reinforcement learning for efficient spectrum usage in next-generation mobile cellular networks

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 40, Issue 4, 2010, Pages 477-484

  Bernardo, Francisco ^a   Agustí, Ramon ^a   Pérez Romero, Jordi ^a   Sallent, Oriol ^a  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

Cellular mobile networks; frequency assignment; reinforcement learning (RL); spectral efficiency

Indexed keywords

CELLULAR MOBILE NETWORKS; COMPUTATIONAL REQUIREMENTS; FREQUENCY ASSIGNMENTS; MOBILE CELLULAR NETWORKS; NUMERICAL RESULTS; ORTHOGONAL FREQUENCY DIVISION MULTIPLE ACCESS; SPECTRAL EFFICIENCIES; SPECTRUM ACCESS; SPECTRUM ASSIGNMENT; SPECTRUM USAGE;

CELLULAR NEURAL NETWORKS; NUMERICAL ANALYSIS; REINFORCEMENT; REINFORCEMENT LEARNING; SPECTROSCOPY; SPECTRUM ANALYZERS;

WIRELESS NETWORKS;

EID: 77953724549     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2010.2041230     Document Type: Article

References (24)

1. T. S. Rappaport,Wireless Communications Principles and Practice. Englewood Cliffs, NJ: Prentice-Hall, 1996.
2. Z. Wang and R. Stirling-Gallacher, "Frequency reuse scheme for cellular OFDM systems," Electron. Lett., vol.38, no.8, pp. 387-388, Apr. 2002.
3. M. Sternad, T. Ottosson, A. Ahlen, and A. Svensson, "Attaining both coverage and high spectral efficiency with adaptive OFDM downlinks," in Proc. IEEE 58th Veh. Technol. Conf. VTC2003-Fall, Oct., pp. 2486-2490.
4. Huawei, "Soft frequency reuse scheme for UTRAN LTE," 3GPP TSG RAN WG1, Tech. Rep. TR1-050507, Orlando, FL, 2005.
5. D. López-Pérez, A. Jüttner, and J. Zhang, "Dynamic frequency planning versus frequency re-use schemes in OFDMA networks," in Proc. IEEE Veh. Technol. Conf.-Spring 2009, Apr., pp. 1-5.
6. F. Bernardo R. Agustí, J. Pérez-Romero, and O. Sallent, "Dynamic spectrum assignment in multicell OFDMA networks enabling a secondary spectrum usage," Wireless Commun. Mobile Comput. (WCMC), vol.9, pp. 1502-1519, 2009.
7. M. M. Buddhikot, "Understanding dynamic spectrum access: Models, taxonomy and challenges," in Proc. IEEE New Front. Dyn. Spectr. Access Netw. (DySPAN), 2007, pp. 649-663.
8. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, Mar. 1998.
9. Y.-S. Chen, C.-J. Chang, and F.-C. Ren, "Q-learning-basedmultirate transmission control scheme forRRMin multimediaWCDMAsystems," IEEE Trans. Veh. Technol., vol.53, no.1, pp. 38-48, Jan. 2004.
10. T.-K. Hui and C.-K. Tham, "Adaptive provisioning of differentiated services networks based on reinforcement learning," IEEE Trans. Syst.,Man, Cybern. C: Appl. Rev., vol.33, no.4, pp. 492-501, Nov. 2003.
11. A. Vasilakos,M. Saltouros, A. Atlassis, andW. Pedrycz, "Optimizing QoS routing in hierarchical ATM networks using computational intelligence techniques," IEEE Trans. Syst., Man, Cybern. C: Appl. Rev., vol.33, no.3, pp. 297-312, Aug. 2003.
12. L. Giupponi and R. Agustí, J. Pérez-Romero, O. Sallent, "Fuzzy neural control for economic-driven radio resource management in beyond 3G networks," IEEE Trans. Syst., Man, Cybern. C: Appl. Rev., vol.39, no.2, pp. 170-189, Mar. 2009.
13. J. Nie and S. Haykin, "A Q-learning-based dynamic channel assignment technique for mobile communication systems," IEEE Trans. Veh. Technol., vol.48, no.5, pp. 1676-1687, Sep. 1999.
14. S. Singh and D. Bertsekas, "Reinforcement learning for dynamic channel allocation in cellular telephone systems," in Advances in Neural Information Processing Systems, vol.9, Cambridge, MA: MIT Press, 1997, pp. 974-980.
15. U. Berthold, F. Fu, M. van der Schaar, and F. Jondral, "Detection of spectral resources in cognitive radios using reinforcement learning," in Proc. 3rd IEEE Symp. New Front. Dyn. Spectr. Access Netw. (DySPAN 2008), Oct., pp. 1-5.
16. R. Farha, N. Abji, O. Sheikh, andA. Leon-Garcia, "Market-based resource management for cognitive radios using machine learning," in Proc. IEEE Global Telecommun. Conf. (GLOBECOM 2007), Nov., pp. 4630-4635.
17. F. Bernardo, R. Agustí, J. Perez-Romero, and O. Sallent, "A novel framework for dynamic spectrum assignment in multicell OFDMA networks based on reinforcement learning," in IEEEWireless Commun. Netw. Conf. (WCNC), Apr. 2009, pp. 1-6.
18. F. Bernardo and R. Agustí, J. Pérez-Romero, O. Sallent, "A self-organized spectrum assignment strategy in next generation OFDMA networks providing secondary spectrum access," in Proc. IEEE Int. Conf. Commun., Jun. 2009, pp. 1-5.
19. F. Bernardo, R. Agustí, J. Perez-Romero, and O. Sallent, "Temporal and spatial spectrum assignment in next generationOFDMAnetworks through reinforcement learning," in Proc. IEEE 69thVeh. Technol.Conf.VTC2009-Spring, pp. 1-5.
20. V. V. Phansalkar and T. M. A., "Local and global optimization algorithms for generalized learning automata," Neural Comput., vol.7, no.5, pp. 950-973, Sep. 1995.
21. C. Wengerter, J. Ohlhorst, and A. V. Elbwart, "Fairness and throughput analysis for generalized proportional fair frequency scheduling in OFDMA," in Proc. IEEE 61st Veh. Technol. Conf. 2005-Spring, vol.3, pp. 1903-1907.
22. R. Schoenen, R. Halfmann, and B.Walke, "MAC performance of a 3GPPLTE multihop cellular network," in Proc. IEEE Int. Conf. Commun. (ICC 2008), May, pp. 4819-4824.
23. J.-G. Choi and S. Bahk, "Cell-throughput analysis of the proportional fair scheduler in the single-cell environment," IEEE Trans. Veh. Technol., vol.56, no.2, pp. 766-778, Mar. 2007.
24. 3GPP, "Physical layer aspects for evolved universal terrestrial radio access (UTRA)," 3GPP, Tech. Rep. TR 25.814 v7.1.0, Oct. 2006, release 7.