Reinforcement Learning of Adaptive Energy Management with Transition Probability for a Hybrid Electric Tracked Vehicle

IEEE Transactions on Industrial Electronics

Volumn 62, Issue 12, 2015, Pages 7837-7846

  Liu, Teng ^a   Zou, Yuan ^a   Liu, Dexing ^a   Sun, Fengchun ^a  

^a BEIJING INSTITUTE OF TECHNOLOGY   (China)

Author keywords

Adaptability; energy management; hybrid electric tracked vehicle; programming (SDP); Q learning algorithm; state of charge (SOC); stochastic dynamic

Indexed keywords

CHARGING (BATTERIES); DYNAMIC PROGRAMMING; ELECTRIC BATTERIES; ELECTRIC GENERATORS; ENERGY MANAGEMENT; ENGINES; FUEL ECONOMY; HYBRID VEHICLES; LEARNING ALGORITHMS; REINFORCEMENT LEARNING; SECONDARY BATTERIES; STOCHASTIC SYSTEMS; TRACKED VEHICLES; VEHICLES;

ADAPTABILITY; CONTROL ORIENTED MODELS; Q-LEARNING ALGORITHMS; STATE OF CHARGE; STOCHASTIC DYNAMIC PROGRAMMING; STOCHASTIC DYNAMICS; TRANSITION PROBABILITIES; TRANSITION PROBABILITY MATRIX;

BATTERY MANAGEMENT SYSTEMS;

EID: 84947601121     PISSN: 02780046     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIE.2015.2475419     Document Type: Article

References (38)

1. I. Boldea, L. N. Tutelea, L. Parsa, and D. Dorrell, "Automotive electric propulsion systems with reduced or no permanent magnets: An overview," IEEE Trans. Ind. Electron., vol. 61, no. 10, pp. 5696-5711, Oct. 2014.
2. M. Kesler, M. C. Kisacikoglu, and L. M. Tolbert, "Vehicle-to-grid reactive power operation using plug-in electric vehicle bidirectional of board charger," IEEE Trans. Ind. Electron., vol. 61, no. 12, pp. 6778-6784, Dec. 2014.
3. C. D. Tong, P. Zheng, Q. Wu, J. G. Bai, and Q. B. Zhao, "A brushless claw-pole double-rotor machine for power-split hybrid electric vehicles," IEEE Trans. Ind. Electron., vol. 61, no. 8, pp. 4295-4305, Aug. 2014.
4. J. P. F. Trovao, V. D. N. Santos, C. Henggeler Antunes, P. G. Pereirinha, and H. M. Jorge, "A real-time energy management architecture for multisource electric vehicles," IEEE Trans. Ind. Electron., vol. 62, no. 5, pp. 3223-3233, May 2015.
5. X. N. Lu, K. Sun, J. M. Guerrero, J. C. Vasquez, and L. P. Huang, "Stateof-charge balance using adaptive droop control for distributed energy storage systems in DC microgrid applications," IEEE Trans. Ind. Electron., vol. 61, no. 6, pp. 2804-2815, Jun. 2014.
6. Z. Y. Song, H. Hofmann, J. Q. Li, X. B. Han, and M. G. Ouyang, "Optimization for a hybrid energy storage system in electric vehicles using dynamic programing approach," Appl. Energy, vol. 139, pp. 151-162, Feb. 2015.
7. J. M. Liu and H. Peng, "Modeling and control of a power-split hybrid vehicle," IEEE Trans. Control Syst. Technol., vol. 16, no. 6, pp. 1242-1251, Nov. 2008.
8. J. Cao and A. Emadi, "A new battery/ultracapacitor hybrid energy storage system for electric, hybrid, and plug-in hybrid electric vehicles," IEEE Trans. Power Electron., vol. 27, no. 1, pp. 122-132, Jan. 2012.
9. Y. F. Wang, B. M. Nguyen, H. Fujimoto, and Y. C. Hori, "Multirate estimation and control of body slip angle for electric vehicles based on onboard vision system," IEEE Trans. Ind. Electron., vol. 61, no. 2, pp. 1133-1143, Feb. 2014.
10. X. Hu, N. Murgovski, L. Johannesson, and B. Egardt, "Energy efficiency analysis of a series plug-in hybrid electric bus with different energy management strategies and battery sizes," Appl. Energy, vol. 111, pp. 1001-1009, Nov. 2013.
11. S. N. Motapon, L. A. Dessaint, and K. Al-Haddad, "A comparative study of energy management schemes for a fuel-cell hybrid emergency power system of more-electric aircraft," IEEE Trans. Ind. Electron., vol. 61, no. 3, pp. 1320-1334, Mar. 2014.
12. C. Sun, S. J. Moura, X. S. Hu, and F. C. Sun, "Dynamic traffic feedback data enabled energy management in plug-in hybrid electric vehicles," IEEE Trans. Control Syst. Technol., vol. 23, no. 3, pp. 1075-1086, May 2015.
13. E. Fuentes, D. Kalise, J. Rodríguez, and R. M. Kennel, "Cascade-free predictive speed control for electrical drives," IEEE Trans. Ind. Electron., vol. 61, no. 5, pp. 2176-2184, May 2014.
14. H. Y. Li, X. J. Jing, and H. R. Karimi, "Output-feedback-based H control for vehicle suspension systems with control delay," IEEE Trans. Ind. Electron., vol. 61, no. 1, pp. 436-446, Jan. 2014.
15. N. Kim, S. Cha, and H. Peng, "Optimal control of hybrid electric vehicles based on pontryagin's minimum principle," IEEE Trans. Control Syst. Technol., vol. 19, no. 5, pp. 1279-1287, Sep. 2011.
16. C. H. Zheng, G. Q. Xu, and Y. I. Park, "Prolonging fuel cell stack lifetime based on pontryagin's minimum principle in fuel cell hybrid vehicles and its economic influence evaluation," J. Power Sources, vol. 248, pp. 533-544, Feb. 2014.
17. Y. Zou, T. Liu, F. C. Sun, and H. Peng, "Comparative study of dynamic programming and pontryagin's minimum principle on energy management for a parallel hybrid electric vehicle," Energies, vol. 6, no. 4, pp. 2305-2318, Apr. 2013.
18. L. Serrao, S. Onori, and G. Rizzoni, "A comparative analysis of energy management strategies for hybrid electric vehicles," J. Dyn. Syst., Meas. Control, vol. 133, no. 3, Mar. 2011, Art. ID. 031012.
19. X. Zhang, H. Peng, and J. Sun, "A near-optimal power management strategy for rapid component sizing of multimode power split hybrid vehicles," IEEE Trans. Control Syst. Technol., vol. 23, no. 2, pp. 609-618, Mar. 2015.
20. B. Geng, J. K. Mills, and D. Sun, "Energy management control of microturbine-powered plug-in hybrid electric vehicles using the telemetry equivalent consumption minimization strategy," IEEE Trans. Veh. Technol., vol. 60, no. 9, pp. 4238-4248, Nov. 2011.
21. C. Dextreit and I. V. Kolmanovsky, "Game theory controller for hybrid electric vehicles," IEEE Trans. Control Syst. Technol., vol. 22, no. 2, pp. 652-663, Mar. 2014.
22. T. Nuësch, P. Elbert, M. Flankl, C. Onder, and L. Guzzella, "Convex optimization for the energy management of hybrid electric vehicles considering engine start and gearshift costs," Energies, vol. 7, no. 2, pp. 834-856, Feb. 2014.
23. C. Liu and Y. L. Murphey, "Power management for plug-in hybrid electric vehicles using reinforcement learning with trip information," in Proc. IEEE ITEC, 2014, pp. 1-6.
24. X. Lin, Y. Z.Wang, P. Bogdan, N. Chang, andM. Pedram, "Reinforcement learning based power management for hybrid electric vehicles," in Proc. IEEE ICCAD, Nov. 2014, pp. 33-38.
25. J. Perron, B. Moulin, and J. Berger, "A hybrid approach based on multiagent geosimulation and reinforcement learning to solve a UAV patrolling problem," in Proc. Winter Simul., 2008, pp. 1259-1267.
26. Q. Wang and C. Phillips, "Cooperative collision avoidance for multivehicle systems using reinforcement learning," in Proc. MMAR, Aug. 2013, pp. 98-102.
27. D. C. Ngai and N. H. Yung, "Automated vehicle overtaking based on a multiple-goal reinforcement learning framework," in Proc. IEEE Intell. Transp. Syst., Oct. 2007, pp. 818-823.
28. T. Hester, M. Quinlan, and P. Stone, "RTMBA: A real-time model-based reinforcement learning architecture for robot control," in Proc. IEEE Robot. Autom., May 2012, pp. 85-90.
29. S. Y. Oh, J. H. Lee, and D. H. Choi, "A new reinforcement learning vehicle control architecture for vision-based road following," IEEE Trans. Veh. Technol., vol. 49, no. 3, pp. 997-1005, May 2000.
30. R. C. Hsu, C. T. Liu, and D. Y. Chan, "A reinforcement-learning-based assisted power management with QoR provisioning for human-electric hybrid bicycle," IEEE Trans. Ind. Electron., vol. 59, no. 8, pp. 3350-3359, Aug. 2012.
31. Y. Zou, F. C. Sun, X. S. Hu, and L. Guzzella, "Combined optimal sizing and control for a hybrid tracked vehicle," Energies, vol. 5, no. 11, pp. 4697-4710, Nov. 2012.
32. J. Shin, S. Y. Shin, Y. S. Kim, and S. Y. Ahn, "Design and implementation of shaped magnetic-resonance-based wireless power transfer system for roadway-powered moving electric vehicles," IEEE Trans. Ind. Electron., vol. 61, no. 3, pp. 1179-1192, Mar. 2014.
33. J. Ko et al., "Development of brake system and regenerative braking cooperative control algorithm for automatic-transmission-based hybrid electric vehicles," IEEE Trans. Veh. Technol., vol. 64, no. 2, pp. 431-440, Feb. 2015.
34. C. C. Lin, H. Peng, and J. W. Grizzle, "A stochastic control strategy for hybrid electric vehicles," in Proc. ACC, Jul. 2004, pp. 4710-4715.
35. T. Brazdil et al., "Verification ofMarkov decision processes using learning algorithms," in Proc. Logic Comput. Sci., 2014, vol. 8837, pp. 98-114.
36. L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey," J. Artif. Intell. Res., vol. 4, pp. 237-285, 1996.
37. I. Chades, G. Chapron, and M. J. Cros, "Markov decision processes toolbox," Jul. 2014. [Online]. Available: http://cran.r-project.org/web/packages/MDPtoolbox/
38. P. Dimitar and K. Ilya, "Generalized Markov models for real-time modeling of continuous systems," IEEE Trans. Fuzzy Syst., vol. 22, no. 4, pp. 983-998, Aug. 2014.