Semisupervised Deep Reinforcement Learning in Support of IoT and Smart City Services

IEEE Internet of Things Journal

Volumn 5, Issue 2, 2018, Pages 624-635

  Mohammadi, Mehdi ^a   Al Fuqaha, Ala ^a   Guizani, Mohsen ^b   Oh, Jun Seok ^a  

^a Parkview Campus   (United States)

^b UNIVERSITY OF IDAHO   (United States)

Author keywords

Bluetooth low energy indoor localization; deep learning; deep reinforcement learning (DRL); indoor positioning; Internet of Things (IoT); IoT smart services; reinforcement learning; semisupervised deep reinforcement learning; smart city

Indexed keywords

BLUETOOTH; INDOOR POSITIONING SYSTEMS; INTERNET OF THINGS; REINFORCEMENT LEARNING; SMART CITY;

DEEP REINFORCEMENT LEARNING (DRL); INDOOR LOCALIZATION; INDOOR POSITIONING; INTERNET OF THINGS (IOT); SEMI-SUPERVISED; SMART SERVICES;

DEEP LEARNING;

EID: 85045442794     PISSN: None     EISSN: 23274662     Source Type: Journal    
DOI: 10.1109/JIOT.2017.2712560     Document Type: Article

References (42)

1. A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, "Internet of Things: A survey on enabling technologies, protocols, and applications," IEEE Commun. Surveys Tuts., vol. 17, no. 4, pp. 2347-2376, 4th Quart., 2015.
2. N. E. Klepeis et al., "The national human activity pattern survey (NHAPS): A resource for assessing exposure to environmental pollutants," J. Expo. Anal. Environ. Epidemiol., vol. 11, no. 3, pp. 231-252, 2001.
3. L. Mainetti, V. Mighali, and L. Patrono, "A location-aware architecture for heterogeneous building automation systems," in Proc. IFIP/IEEE Int. Symp. Integr. Netw. Manag. (IM), Ottawa, ON, Canada, 2015, pp. 1065-1070.
4. S. Alletto et al., "An indoor location-aware system for an IoT-based smart museum," IEEE Internet Things J., vol. 3, no. 2, pp. 244-253, Apr. 2016.
5. M. V. Moreno, J. L. Hernández, and A. F. Skarmeta, "A new locationaware authorization mechanism for indoor environments," in Proc. 28th Int. Conf. Adv. Inf. Netw. Appl. Workshops (WAINA), Victoria, BC, Canada, 2014, pp. 791-796.
6. G. Sunkada, "System for and method of location aware marketing," U.S. Patent 12 851 968, Feb. 9, 2012.
7. P. Dickinson, G. Cielniak, O. Szymanezyk, and M. Mannion, "Indoor positioning of shoppers using a network of Bluetooth low energy beacons," in Proc. Int. Conf. Indoor Position. Indoor Navig. (IPIN), Alcalá de Henares, Spain, 2016, pp. 1-8.
8. J. Torres-Sospedra et al., "Enhancing integrated indoor/outdoor mobility in a smart campus," Int. J. Geograph. Inf. Sci., vol. 29, no. 11, pp. 1955-1968, 2015.
9. F. Zafari, I. Papapanagiotou, and K. Christidis, "Microlocation for Internet-of-Things-equipped smart buildings," IEEE Internet Things J., vol. 3, no. 1, pp. 96-112, Feb. 2016.
10. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015.
11. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, vol. 1. Cambridge, MA, USA: MIT Press, 1998.
12. V. Mnih et al., "Playing Atari with deep reinforcement learning," arXiv preprint arXiv:1312.5602v1 [cs.LG], 2013.
13. R. Faragher and R. Harle, "Location fingerprinting with Bluetooth low energy beacons," IEEE J. Sel. Areas Commun., vol. 33, no. 11, pp. 2418-2428, Nov. 2015.
14. P. Christiano. (2016). Semi-Supervised Reinforcement Learning. [Online]. Available: https://medium.com/ai-control/semi-supervisedreinforcement-learning-cf7d5375197f
15. D. Amodei et al., "Concrete problems in AI safety," arXiv preprint arXiv:1606.06565v2 [cs.AI], 2016.
16. D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling, "Semisupervised learning with deep generative models," in Proc. Adv. Neural Inf. Process. Syst., Montreal, QC, Canada, 2014, pp. 3581-3589.
17. S. Nemati, M. M. Ghassemi, and G. D. Clifford, "Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach," in Proc. IEEE 38th Annu. Int. Conf. Eng. Med. Biol. Soc. (EMBC), Orlando, FL, USA, 2016, pp. 2978-2981.
18. D. Zhao, Y. Chen, and L. Lv, "Deep reinforcement learning with visual attention for vehicle classification," IEEE Trans. Cogn. Develop. Syst., to be published.
19. J. C. Caicedo and S. Lazebnik, "Active object localization with deep reinforcement learning," in Proc. IEEE Int. Conf. Comput. Vis., Santiago, Chile, 2015, pp. 2488-2496.
20. Y. Zhu et al., "Target-driven visual navigation in indoor scenes using deep reinforcement learning," arXiv preprint arXiv:1609.05143v1 [cs.CV], 2016.
21. L. Li, Y. Lv, and F.-Y. Wang, "Traffic signal timing via deep reinforcement learning," IEEE/CAA J. Autom. Sinica, vol. 3, no. 3, pp. 247-254, Jul. 2016.
22. H. Mao, M. Alizadeh, I. Menache, and S. Kandula, "Resource management with deep reinforcement learning," in Proc. 15th ACM Workshop Hot Topics Netw., Atlanta, GA, USA, 2016, pp. 50-56.
23. K. Narasimhan, T. Kulkarni, and R. Barzilay, "Language understanding for text-based games using deep reinforcement learning," arXiv preprint arXiv:1506.08941v2 [cs.CL], 2015.
24. J. He et al., "Deep reinforcement learning with a natural language action space," arXiv preprint arXiv:1511.04636v5 [cs.AI], 2015.
25. V. François-Lavet, D. Taralla, D. Ernst, and R. Fonteneau, "Deep reinforcement learning solutions for energy microgrids management," in Proc. Eur. Workshop Reinforcement Learn., 2016, pp. 1-7.
26. B. Wang, Q. Chen, L. T. Yang, and H.-C. Chao, "Indoor smartphone localization via fingerprint crowdsourcing: Challenges and approaches," IEEE Wireless Commun., vol. 23, no. 3, pp. 82-89, Jun. 2016.
27. W. Zhang, K. Liu, W. Zhang, Y. Zhang, and J. Gu, "Deep neural networks for wireless localization in indoor and outdoor environments," Neurocomputing, vol. 194, pp. 279-287, Jun. 2016.
28. S. Kajioka, T. Mori, T. Uchiya, I. Takumi, and H. Matsuo, "Experiment of indoor position presumption based on RSSI of Bluetooth LE beacon," in Proc. IEEE 3rd Glob. Conf. Consum. Electron. (GCCE), Tokyo, Japan, 2014, pp. 337-339.
29. Y. Wang, X. Yang, Y. Zhao, Y. Liu, and L. Cuthbert, "Bluetooth positioning using RSSI and triangulation methods," in Proc. IEEE 10th Consum. Commun. Netw. Conf. (CCNC), Las Vegas, NV, USA, 2013, pp. 837-842.
30. X. Wang, L. Gao, S. Mao, and S. Pandey, "Deepfi: Deep learning for indoor fingerprinting using channel state information," in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC), New Orleans, LA, USA, 2015, pp. 1666-1671.
31. Y. Gu, Y. Chen, J. Liu, and X. Jiang, "Semi-supervised deep extreme learning machine for Wi-Fi based localization," Neurocomputing, vol. 166, pp. 282-293, Oct. 2015.
32. G. Ding, Z. Tan, J. Zhang, and L. Zhang, "Fingerprinting localization based on affinity propagation clustering and artificial neural networks," in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC), Shanghai, China, 2013, pp. 2317-2322.
33. J. Luo and H. Gao, "Deep belief networks for fingerprinting indoor localization using ultrawideband technology," Int. J. Distrib. Sensor Netw., vol. 2016, pp. 1-8, Jan. 2016.
34. X. Zhang, J. Wang, Q. Gao, X. Ma, and H. Wang, "Device-free wireless localization and activity recognition with deep learning," in Proc. IEEE Int. Conf. Pervasive Comput. Commun. Workshops (PerCom Workshops), Sydney, NSW, Australia, 2016, pp. 1-5.
35. T. Pulkkinen, T. Roos, and P. Myllymäki, "Semi-supervised learning for WLAN positioning," in Proc. Int. Conf. Artif. Neural Netw., Espoo, Finland, 2011, pp. 355-362.
36. J. Weston, F. Ratle, H. Mobahi, and R. Collobert, "Deep learning via semi-supervised embedding," in Neural Networks: Tricks of the Trade. Heidelberg, Germany: Springer, 2012, pp. 639-655.
37. O. Chapelle, B. Schölkopf, and A. Zien, Semi-Supervised Learning. Cambridge, MA, USA: MIT Press, 2006.
38. J. Walker, C. Doersch, A. Gupta, and M. Hebert, "An uncertain future: Forecasting from static images using variational autoencoders," in Proc. Eur. Conf. Comput. Vis., Amsterdam, The Netherlands, 2016, pp. 835-851.
39. V. Mnih et al., "Human-level control through deep reinforcement learning," Nature, vol. 518, no. 7540, pp. 529-533, 2015.
40. L. Mainetti, L. Patrono, and I. Sergi, "A survey on indoor positioning systems," in Proc. 22nd Int. Conf. Softw. Telecommun. Comput. Netw. (SoftCOM), Split, Croatia, 2014, pp. 111-120.
41. M. Abadi et al., "Tensorflow: Large-scale machine learning on heterogeneous distributed systems," arXiv:1603.04467v2 [cs.DC], 2016.
42. F. Chollet. (2015). Keras: Deep Learning Library for Theano and TensorFlow. [Online]. Available: https://keras.io/