RLMan: An Energy Manager Based on Reinforcement Learning for Energy Harvesting Wireless Sensor Networks

IEEE Transactions on Green Communications and Networking

Volumn 2, Issue 2, 2018, Pages 408-417

  Aoudia, Faycal Ait ^a   Gautier, Matthieu ^a   Berder, Olivier ^a  

^a CAMPUS DE BEAULIEU   (France)

Author keywords

autonomous systems; energy harvesting; energy management; learning systems; Wireless sensor networks

Indexed keywords

ENERGY HARVESTING; ENERGY MANAGEMENT; ENERGY POLICY; ENERGY STORAGE; ENERGY UTILIZATION; MANAGERS; QUALITY OF SERVICE; REINFORCEMENT LEARNING; WIRELESS SENSOR NETWORKS;

ENERGY MANAGEMENT ALGORITHMS; EXHAUSTIVE SIMULATION; LEARNING (ARTIFICIAL INTELLIGENCE); STATE OF CHARGE; STATE-OF-THE-ART APPROACH; TIME VARYING BEHAVIOR; TIME-VARYING ENVIRONMENTS; WIRELESS SENSOR NODE;

SENSOR NODES;

EID: 85061389718     PISSN: None     EISSN: 24732400     Source Type: Journal    
DOI: 10.1109/TGCN.2018.2801725     Document Type: Conference Paper

References (19)

1. R. J. M. Vullers, R. V. Schaijk, H. J. Visser, J. Penders, and C. V. Hoof, "Energy harvesting for autonomous wireless sensor networks," IEEE Solid State Circuits Mag., vol. 2, no. 2, pp. 29-38, Jun. 2010.
2. A. Kansal, J. Hsu, S. Zahedi, and M. B. Srivastava, "Power management in energy harvesting sensor networks," ACM Trans. Embedded Comput. Syst., vol. 6, no. 4, 2007, Art. no. 32.
3. A. Castagnetti, A. Pegatoquet, C. Belleudy, and M. Auguin, "A framework for modeling and simulating energy harvesting WSN nodes with efficient power management policies," EURASIP J. Embedded Syst., vol. 1, no. 1, p. 8, 2012.
4. T. N. Le, A. Pegatoquet, O. Berder, and O. Sentieys, "Energy-efficient power manager and MAC protocol for multi-hop wireless sensor networks powered by periodic energy harvesting sources," IEEE Sensors J., vol. 15, no. 12, pp. 7208-7220, Dec. 2015.
5. F. A. Aoudia, M. Gautier, and O. Berder, "GRAPMAN: Gradual power manager for consistent throughput of energy harvesting wireless sensor nodes," in Proc. IEEE 26th Annu. Int. Symp. Pers. Indoor Mobile Radio Commun. (PIMRC), Aug. 2015, pp. 954-959.
6. R. Margolies et al., "Energy-harvesting active networked tags (EnHANTs): Prototyping and experimentation," ACM Trans. Sensor Netw., vol. 11, no. 4, pp. 1-27, Nov. 2015.
7. C. M. Vigorito, D. Ganesan, and A. G. Barto, "Adaptive control of duty cycling in energy-harvesting wireless sensor networks," in Proc. 4th Annu. IEEE Commun. Soc. Conf. Sensor Mesh Ad Hoc Commun. Netw. (SECON), San Diego, CA, USA, Jun. 2007, pp. 21-30.
8. S. Peng and C. P. Low, "Prediction free energy neutral power management for energy harvesting wireless sensor nodes," Ad Hoc Netw., vol. 13, pp. 351-367, Feb. 2014.
9. F. A. Aoudia, M. Gautier, and O. Berder, "Fuzzy power management for energy harvesting wireless sensor nodes," in Proc. IEEE Int. Conf. Commun. (ICC), Kuala Lumpur, Malaysia, May 2016, pp. 1-6.
10. A. Ortiz, H. Al-Shatri, X. Li, T. Weber, and A. Klein, "Reinforcement learning for energy harvesting decode-and-forward two-hop communications," IEEE Trans. Green Commun. Netw., vol. 1, no. 3, pp. 309-319, Sep. 2017.
11. R. C. Hsu, C.-T. Liu, and H. L. Wang, "A reinforcement learning-based ToD provisioning dynamic power management for sustainable operation of energy harvesting wireless sensor node," IEEE Trans. Emerg. Topics Comput., vol. 2, no. 2, pp. 181-191, Jun. 2014.
12. C. Liu, X. Xu, and D. Hu, "Multiobjective reinforcement learning: A comprehensive overview," IEEE Trans. Syst., Man, Cybern., Syst., vol. 45, no. 3, pp. 385-398, Mar. 2015.
13. C. R. Shelton, "Balancing multiple sources of reward in reinforcement learning," in Advances in Neural Information Processing Systems 13, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds. Cambridge, MA, USA: MIT Press, 2001, pp. 1082-1088.
14. I. Grondman, L. Busoniu, G. A. D. Lopes, and R. Babuska, "A survey of actor-critic reinforcement learning: Standard and natural policy gradients," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 42, no. 6, pp. 1291-1307, Nov. 2012.
15. R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour, "Policy gradient methods for reinforcement learning with function approximation," in Advances in Neural Information Processing Systems 12, vol. 99. Cambridge, MA, USA: MIT Press, 2000, pp. 1057-1063. [Online]. Available: https://dl.acm.org/citation.cfm?id=3009806& qualifier=LU1042262
16. R. S. Sutton, "Learning to predict by the methods of temporal differences," Mach. Learn., vol. 3, no. 1, pp. 9-44, 1988.
17. O. Berder and O. Sentieys, "PowWow: Power optimized hardware/ software framework for wireless motes," in Proc. Int. Conf. Archit. Comput. Syst. (ARCS), Hanover, Germany, Feb. 2010, pp. 1-5.
18. M. Gorlatova, A. Wallwater, and G. Zussman, "Networking low-power energy harvesting devices: Measurements and algorithms," in Proc. IEEE INFOCOM, Shanghai, China, Apr. 2011, pp. 1602-1610.
19. National Renewable Energy Laboratory (NREL). Accessed: May 2017. [Online]. Available: http://www.nrel.gov/