A reinforcement learning based solution for cognitive network cooperation between co-located, heterogeneous wireless sensor networks

Ad Hoc Networks

Volumn 17, Issue , 2014, Pages 98-113

  Rovcanin, Milos ^a   Poorter, Eli De ^a   Moerman, Ingrid ^a   Demeester, Piet ^a  

^a GHENT UNIVERSITY   (Belgium)

Author keywords

Least Squares Policy Iteration; Machine learning; Network cooperation; Reinforcement learning

Indexed keywords

HETEROGENEOUS WIRELESS SENSOR NETWORKS; INTERFERENCE AVOIDANCE; LEAST SQUARES POLICY ITERATIONS; MACHINE LEARNING TECHNIQUES; NETWORK COOPERATION; POSITIVE AND NEGATIVE EFFECT; WIRELESS COMMUNICATION DEVICES; WIRELESS COMMUNICATIONS;

CHEMICAL ACTIVATION; ITERATIVE METHODS; LEARNING SYSTEMS; MEDIUM ACCESS CONTROL; REINFORCEMENT LEARNING; WIRELESS SENSOR NETWORKS; WIRELESS TELECOMMUNICATION SYSTEMS;

LEAST SQUARES APPROXIMATIONS;

EID: 84896715082     PISSN: 15708705     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.adhoc.2014.01.009     Document Type: Article

References (29)

1. N. Wakamiya, S. Arakawa, M. Murata, Self-organization based network architecture for new generation networks, in: 2009 First International Conference on Emerging Network Intelligence, 11-16 October 2009.
2. R.W. Thomas, L.A. DaSilva, A.B. MacKenzie, Cognitive networks, in: Proc. IEEE DySPAN, 2005, pp. 352-360.
3. E. De Poorter, P. Becue, M. Rovcanin, I. Moerman, P. Demeester, A negotiation-based networking methodology to enable cooperation across heterogeneous co-located networks, Ad Hoc Netw. 10 (2012) 901-917.
4. http://www-01.ibm.com/software/commerce/optimization/linear-programming/, official IBM webpage, optimization tools, 12.07.2013.
5. M. Lagoudakis, R. Parr, Model-free least-squares policy iteration, in: Proc. of NIPS, 2001.
6. T.G. Dietterich, and O. Langley Machine Learning for Cognitive Networks: Technology Assessment and Research Challenges in Cognitive Networks: Towards Self Aware Networks 2007 John Wiley and Sons, Ltd. Chichester, UK doi: 10.1002/9780470515143.ch5
7. A. Forster, Machine learning techniques applied to wireless ad-hoc networks: guide and survey, in: 3rd International Conference on Intelligent Sensors Sensor Networks and Information (ISSNIP), Melbourne, Australia, 3-6 December 2007.
8. S. Haykin Cognitive radio: brain-empowered wireless communications IEEE J. Select. Areas Commun. 23 2 2005 201 220 10.1109/JSAC.2004.839380
9. F.K. Jondral Software-dened radio - basics and evolution to cognitive radio EURASIP J. Wireless Commun. Netw. 2005 3 2005 275283
10. N. Vucevic, I.F. Akyildiz, J. Perez-Romero, Cooperation reliability based on reinforcement learning for cognitive radio networks, in: Fifth IEEE Workshop on Networking Technologies for Software Defined Radio (SDR) Networks, 21st of June 2010, Boston, MA, USA, 2010.
11. M. Yang, and D. Grace Cognitive radio with reinforcement learning applied to multicast downlink transmission with power adjustment Wireless Personal Commun. 57 2011 73 87
12. H. Li, Multi-agent Q-learning of channel selection in multi-user cognitive radio systems: a two by two case, in: IEEE International Conference on Systems, Man and Cybernetics, 2009 (SMC 2009), 2009, pp. 1893-1898.
13. Christopher J.C.H. Watkins, and Peter. Dayan Technical note Q-learning Machine Learn. 8 1992 279 292
14. A. Galindo-Serrano, L. Giupponi, Decentralized Q-learning for aggregated interference control in completely and partially observable cognitive radio networks, in: 7th Annual IEEE Conference on Consumer Communications and Networking Conference (CCNC), 9-12 January 2010, Las Vegas, Nevada, USA, 2010.
15. S. Shankar, C. Cordeiro, Analysis of aggregated interference at DTV receivers in TV bands, in: Proc. Int. Conf. CROWNCOM, Singapore, May 15-17, 2008, pp. 16.
16. R.W. Thomas, D.H. Friend, L.A. Dasilva, and A.B. Mackenzie Cognitive networks: adaptation and learning to achieve end-to-end performance objectives IEEE Commun. Mag. 44 12 2006 51 57 10.1109/MCOM.2006.273099
17. P. Wang, and T. Wang Adaptive routing for sensor networks using reinforcement learning CIT '06 Proceedings of the Sixth IEEE International Conference on Computer and Information Technology, October 22-25, 2006 2006 Charlotte Convention Center Charlotte, NC
18. Z. Ye, A.A. Abouzeid, Layered sequential decision policies for cross-layer design of multihop wireless networks, Information Theory and Applications Workshop (ITA'10), San Diego, CA, February 2010.
19. Minsoo Lee, Dan Marconett, Xiaohui Ye, and S. Yoo Cognitive network management with reinforcement learning for wireless mesh networks IP Oper. Manage. 2007 168 179 10.1007/978-3-540-75853-2-15
20. Ad hoc on-demand distance vector (AODV) routing, networking group request for comments (rfc): 3561, July 2003. < http://tools.ietf.org/html/rfc3561 >.
21. http://www.me.utexas.edu/bard/LP/LP 20Handouts/CPLEX 20Tutorial 20Handout.pdf.
22. Y. Wu, P. Chou, Q. Zhang, K. Jain, W. Zhu, and S.-Y. Kung Network planning in wireless ad hoc networks: a cross-layer approach IEEE J. Select. Areas Commun. 23 1 2005 136 150 10.1109/JSAC.2004.837362(410) 23
23. Michail G. Lagoudakis, and Ronald Parr Least-squares policy iteration J. Machine Learn. Res. 4 2003 1107 1149
24. L.P. Kaelblign, M.L. Littman, and A.W. Moore Reinforcement learning: a survey J. Artif. Intell. Res. 4 1996 237 285
25. Lucian Busoniu, Robert Babuska, Bart De Schutter, Damien Ernst, Reinforcement Learning and Dynamic Programming Using Function Approximators, 2010 by Taylor and Francis Group, LLC, ISBN: 978-1-4398-2108-4 (Hardback).
26. John Myles. White Bandit Algorithms for Website Optimization 2012 Published by O'Rilley Media Inc.
27. M. Tokic, and G. Palm Value-difference based exploration: adaptive control between epsilon-greedy and softmax J. Bach, S. Edelkamp, KI 2011: Advances in Artificial Intelligence Lecture Notes in Artificial Intelligence vol. 7006 2011 Springer Berlin/Heidelberg 335 346
28. http://www.iminds.be/en/develop-test/ilab-t, Official iMinds webpage, 12.02.2013.
29. L. Tytgat, B. Jooris, P. De Mil, B. Latr, I. Moerman, P. Demeester, Demo abstract: Wilab, a real-life wireless sensor testbed with environment emulation, published in European conference on Wireless Sensor Networks, EWSN adjunct poster proceedings (EWSN), Cork, Ireland, February 2009.