Cognitive Cellular Networks: A Q-Learning Framework for Self-Organizing Networks

IEEE Transactions on Network and Service Management

Volumn 13, Issue 1, 2016, Pages 85-98

  Mwanje, Stephen S ^a   Schmelz, Lars Christoph ^a   Mitschele Thiel, Andreas ^b  

^a NOKIA RESEARCH CENTER   (United States)

^b ILMENAU UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

Cognitive Cellular Networks; MLB; MRO; SON

Indexed keywords

MOBILE TELECOMMUNICATION SYSTEMS; NETWORK MANAGEMENT; RIGID STRUCTURES; WIRELESS NETWORKS;

CELLULAR NETWORK; HANDOVER PERFORMANCE; MOBILITY LOAD BALANCING; MOBILITY ROBUSTNESS OPTIMIZATIONS; OPERATIONAL EXPENDITURES; RULE-BASED CONTROL; SELF ORGANIZING NETWORK (SON); SELF-ORGANIZING NETWORK;

COMPLEX NETWORKS;

EID: 84963946935     PISSN: 19324537     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNSM.2016.2522080     Document Type: Article

References (36)

1. 3GPP, "Evolved universal terrestrial radio access network (E-UTRAN); Self-configuring and self-optimizing network (SON) use cases and solutions," 3GPP, Tech. Rep. 36.902, version 9.3.1, Release 9, May 2011 [Online]. Available: http://www.3gpp.org
2. J. Alonso-Rubio, "Self-optimization for handover oscillation control in LTE," in Proc. IEEE/IFIP Netw. Oper. Manage. Symp., Osaka, Japan, 2010, pp. 950-953.
3. I. Balan, T. Jansen, B. Sas, I. Moerman, T. Kürner, "Enhanced weighted performance based handover optimization in LTE," in Proc. Future Netw. Mobile Summit (FNMS), Warsaw, Poland, 2011, pp. 1-8.
4. G. Hui and P. Legg, "Soft metric assisted mobility robustness optimization in LTE networks," in Proc. Int. Symp. Wireless Commun. Syst. (ISWCS'12), Paris, France, 2012, pp. 1-5.
5. V. Capdevielle, A. Feki, A. Fakhreddine, "Self-optimization of handover parameters in LTE networks," in Proc. 11th Int. Symp. Model. Optim. Mobile Ad Hoc Wireless Netw. (WiOpt), May 2013, pp. 133-139.
6. H. Eckhardt, S. Klein, M. Gruber, "Vertical antenna tilt optimization for LTE base stations," in Proc. IEEE 73rd Veh. Technol. Conf. (VTCSpring), May 2011, pp. 1-5.
7. A. Gerdenitsch, S. Jakl, Y. Chong, M. Toeltsch, "A rule-based algorithm for common pilot channel and antenna tilt optimization in UMTS FDD networks," ETRI J., vol. 26, no. 5, pp. 437-442, 2004.
8. M. Garcia-Lozano, S. Ruiz, J. Olmos, "UMTS optimum cell load balancing for inhomogeneous traffic patterns," in Proc. IEEE 60th Veh. Technol. Conf. (VTC-Fall), Sep. 2004, pp. 909-913.
9. L. Du, J. Bigham, L. Cuthbert, C. Parini, P. Nahi, "Using dynamic sector antenna tilting control for load balancing in cellular mobile communications," in Proc. Int. Conf. Telecommun. (ICT), Beijing, China, 2002, pp. 344-348.
10. M. N. U. Islam and A. Mitschele-Thiel, "Reinforcement learning strategies for self-organized coverage and capacity optimization," in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC), 2012, pp. 2818-2823.
11. M. Dirani and Z. Altman, "A cooperative reinforcement learning approach for inter-cell interference coordination in OFDMA cellular networks," in Proc. 8th Int. Symp. Model. Optim. Mobile Ad Hoc Wireless Netw. (WiOpt), 2010, pp. 170-176.
12. M. Bennis and D. Niyato, "A Q-learning based approach to interference avoidance in self-organized femtocell networks," in Proc. IEEE GLOBECOM Workshops, 2010, pp. 706-710.
13. S. Mwanje and A. Mitschele-Thiel, "Distributed cooperative Q-learning formobility-sensitive handover optimization in LTE SON," in Proc. IEEE Symp. Comput. Commun. (ISCC), 2014, pp. 1-6.
14. S. S. Mwanje and A. Mitschele-Thiel, "A Q-learning strategy for LTE mobility load balancing," in Proc. IEEE 24th Int. Symp. Pers. Indoor Mobile Radio Commun. (PIMRC), 2013, pp. 2154-2158.
15. C. Watkins, "Learning from delayed rewards," Ph.D. dissertation, Kings College, Cambridge Univ., Cambridge, U.K., May 1989.
16. R. Sutton, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 1998.
17. C. J. Watkins and P. Dayan, "Q-learning," Mach. Learn., vol. 8, pp. 279-292, 1992.
18. E. Even-Dar, S. Mannor, Y. Mansour, "PAC bounds for multi-armed bandit and Markov decision processes," in Proc. 15th Conf. Comput. Learn. Theory, Jul. 2002, pp. 258-259.
19. R. Razavi, S. Klein, H. Claussen, "Self-optimization of capacity and coverage in LTE networks using a fuzzy reinforcement learning approach," in Proc. IEEE 21st Int. Symp. Pers. Indoor Mobile Radio Commun. (PIMRC), 2010, pp. 1865-1870.
20. P. Munoz, R. Barco, I. de la Bandera, M. Toril, S. Luna-Ramirez, "Optimization of a fuzzy logic controller for handover-based load balancing," in Proc. IEEE 73rd Veh. Technol. Conf. (VTC-Spring), 2011, pp. 1-5.
21. P. Munoz, R. Barco, J. M. Ruiz-Aviles, I. de la Bandera, A. Aguilar, "Fuzzy rule-based reinforcement learning for load balancing techniques in enterprise LTE femtocells," IEEE Trans. Veh. Technol., vol. 62, no. 5, pp. 1962-1973, Jun. 2013.
22. N. Sinclair, D. Harle, I. A. Glover, J. Irvine, R. C. Atkinson, "An advanced SOM algorithm applied to handover management within LTE," IEEE Trans. Veh. Technol., vol. 62, no. 5, pp. 1883-1894, Jun. 2013.
23. Institute of Communication Networks, Computer Engineering (IKR), "IKR simulation and emulation library," Universitt Stuttgart, Stuttgart, Germany, Tech. Rep., Sep. 2013, v.4.0 [Online]. Available: http://www.ikr.uni-stuttgart.de/en/Content/IKRSimLib/
24. 3GPP, "Physical layer aspect for evolved universal terrestrial radio access (UTRAN)," 3GPP TR 25.814, Tech. Rep., Oct. 2006 [Online]. Available: http://www.3gpp.org
25. A. M. Rao, A. Weber, S. Gollamudi, R. Soni, "LTE and HSPA+: Revolutionary and evolutionary solutions for global mobile broadband," Bell Labs Tech. J., vol. 13, no. 4, pp. 7-34, 2009.
26. 3GPP, "E-UTRA radio resource control (RRC) protocol specification (Release 8)," 3GPP, Tech. Rep. 36.331 V8.16.0, Dec. 2011.
27. T. Jansen, I. Balany, I. Moermanz, T. Kürner, "Handover parameter optimization in LTE self-organizing networks," in Proc. 72nd Veh. Technol. Conf., Athens, Greece, Feb. 2010, pp. 1-5.
28. T. Jansen, I. Balan, S. Stefanski, I. Moerman, T. Kurner, "Weighted performance based handover parameter optimization in LTE," in Proc. IEEE 73rd Veh. Technol. Conf. (VTC-Spring), 2011, pp. 1-5.
29. I. Balan, B. Sas, T. Jansen, I. Moerman, K. Spaey, P. Demeester, "An enhanced weighted performance-based handover parameter optimization algorithm for LTE networks," EURASIP J. Wireless Commun. Netw., p. 98, 2011.
30. S. Mwanje, N. Zia, A. Mitschele-Thiel, "Self organised handover parameter configuration for LTE," in Proc. Int. Symp. Wireless Commun. Syst. (ISWCS'12), Paris France, Aug. 2012, pp. 26-30.
31. G. Hui and P. Legg, "Soft metric assisted mobility robustness optimization in LTE networks," in Proc. 9th Int. Symp. Wireless Commun. Syst. (ISWCS), Paris France, Aug. 2012, pp. 1-5.
32. J. Lorca and A. Sierra, "A simple speed estimation algorithm for mobility-aware SON RRM strategies in LTE," in Proc. 6th IFIP/IEEE Wireless Days Conf. (WD'13), Valencia, Spain, 2013, pp. 1-3.
33. H. Wang, L. Ding, P. Wu, Z. Pan, N. Liu, X. You, "Dynamic load balancing and throughput optimization in 3GPP LTE networks," in Proc. 6th Int. Wireless Commun. Mobile Comput. Conf., 2010, pp. 939-943.
34. A. Lobinger, S. Stefanski, T. Jansen, I. Balan, "Load balancing in downlink LTE self-optimizing networks," in Proc. Veh. Technol. Conf. (VTC-Spring), Taipei, Taiwan, May 2010, pp. 1-5.
35. S. Koenig and R. G. Simmons, "Complexity analysis of real-time reinforcement learning," in Proc. 11th Nat. Conf. Artif. Intell. (AAAI'93), 1993, vol. 11, p. 99.
36. M. Kearns and S. Singh, "Finite-sample convergence rates for Q-learning and indirect algorithms," in Proc. Adv. Neural Inf. Process. Syst., 1999, pp. 996-1002.