PDP: Parallel dynamic programming

IEEE/CAA Journal of Automatica Sinica

Volumn 4, Issue 1, 2017, Pages 1-5

  Wang, Fei Yue ^a   Zhang, Jie ^a   Wei, Qinglai ^a   Zheng, Xinhu ^b   Li, Li ^c  

^a INSTITUTE OF AUTOMATION   (China)

^b University of Minnesota   (United States)

^c TSINGHUA UNIVERSITY   (China)

Author keywords

Adaptive dynamic programming; Artificial intelligence; Deep learning; Dynamic programming; Neural networks; Parallel dynamic programming; Reinforcement learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; DEEP LEARNING; DEEP NEURAL NETWORKS; INTELLIGENT COMPUTING; LEARNING SYSTEMS; NEURAL NETWORKS; REINFORCEMENT LEARNING;

ADAPTIVE DYNAMIC PROGRAMMING; PARALLEL DYNAMIC PROGRAMMING; PRINCIPLE OF OPTIMALITY; REINFORCEMENT LEARNING METHOD;

DYNAMIC PROGRAMMING;

EID: 85010049728     PISSN: 23299266     EISSN: 23299274     Source Type: Journal    
DOI: 10.1109/JAS.2017.7510310     Document Type: Article

References (46)

1. D. Silver et al., "Mastering the game of Go with deep neural networks and tree search, " Nature 529.7587, pp. 484-489, 2016.
2. R. E. Bellman, Dynamic Programming. Princeton, NJ: Princeton University Press, 1957.
3. P. J. Werbos, "Advanced forecasting methods for global crisis warning and models of intelligence, " General Syst. Yearbook, vol. 22, 1977.
4. P. J. Werbos, "A menu of designs for reinforcement learning over time, " in Neural Networks for Control, W. T. Miller, R. S. Sutton and P. J. Werbos (Eds.), Cambridge: MIT Press, 1991, pp. 67-95.
5. F.-Y. Wang, et al., "Where does AlphaGo go: from church-Turing thesis to AlphaGo thesis and beyond", IEEE/CAA J. Autom. Sinica, vol. 3, no. 2, pp. 113-120, April 2016.
6. F.-Y. Wang, "A big-data perspective on AI: Newton, Merton, and analytics intelligence", IEEE Intell. Syst., vol. 27, no. 5, pp. 2-4, 2012.
7. L. Li, Y.-L. Lin, D.-P. Cao, N.-N. Zheng, and F.-Y. Wang, "Parallel learning-A new framework for machine learning, " Acta Autom. Sinica, vol. 43, no. 1, pp. 1-8, 2017 (in Chinese).
8. J. Li, W. Xu, J. Zhang, M. Zhang, Z. Wang, and X. Li, "Efficient video stitching based on fast structure deformation, " IEEE Trans. Cybern., article in press, 2015. DOI: 10.1109/TCYB.2014.2381774.
9. C. Vagg, S. Akehurst, C. J. Brace, and L. Ash, "Stochastic dynamic programming in the real-world control of hybrid electric vehicles, " IEEE Trans. Control Syst. Technol., vol. 24, no. 3, pp. 853-866, Mar. 2016.
10. P. M. Esfahani, D. Chatterjee, and J. Lygeros, "Motion planning for continuous-Time stochastic processes: A dynamic programming approach, " IEEE Trans. Autom. Control, vol. 61, pp. 2155-2170, 2016.
11. P. J. Werbos, "Approximate dynamic programming for real-Time control and neural modeling, " in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D.A. White and D.A. Sofge (Eds.), New York: Van Nostrand Reinhold, 1992, ch. 13.
12. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
13. D. V. Prokhorov and D. C. Wunsch, "Adaptive critic designs, " IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
14. J. Han, S. Khushalani-Solanki, J. Solanki, and J. Liang, "Adaptive critic design-based dynamic stochastic optimal control design for a microgrid with multiple renewable resources, " IEEE Trans. Smart Grid, vol. 6, no. 6, pp. 2694-2703, Jun. 2015.
15. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
16. J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, "Adaptive dynamic programming, " IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 140-153, May 2002.
17. Q. Wei, F. L. Lewis, D. Liu, R. Song, and H. Lin, "Discrete-Time local value Iteration adaptive dynamic programming: Convergence analysis, " IEEE Trans. Syst., Man, Cybern. A, Syst., article in press, 2016. DOI: 10.1109/TSMC.2016.2623766.
18. Q. Wei, F. L. Lewis, Q. Sun, P. Yan, and R. Song, "Discrete-Time deterministic Q-learning: A novel convergence analysis, " IEEE Trans. Cybern., article in press, 2016. DOI: 10.1109/TCYB.2016.2542923.
19. Q. Wei, D. Liu, and G. Shi, "A novel dual iterative Q-learning method for optimal battery management in smart residential environments, " IEEE Trans. Ind. Electron., vol. 62, no. 4, pp. 2509-2518, Apr. 2015.
20. Q. Wei and D. Liu, "A novel iterative-Adaptive dynamic programming for discrete-Time nonlinear systems, " IEEE Trans. Autom. Sci. Eng., vol. 11, no. 4, pp. 1176-1190, Oct. 2014.
21. Q. Wei, D. Liu, Q. Lin, and R. Song, "Discrete-Time optimal control via local policy iteration adaptive dynamic programming, " IEEE Trans. Cybern., article in press, 2016. DOI: 10.1109/TCYB.2016.2586082.
22. R. Enns and J. Si, "Helicopter trimming and tracking control using direct neural dynamic programming, " IEEE Trans. Neural Netw., vol. 14, no. 4, pp. 929-939, Aug. 2003.
23. R. Kamalapurkar, J. R. Klotz, and W. E. Dixon, "Concurrent learningbased approximate feedback-Nash equilibrium solution of N-player nonzero-sum differential games, " IEEE/CAA J. Autom. Sinica, vol. 1, no. 3, pp. 239-247, Jul. 2014.
24. Q. Wei, D. Liu, and Q. Lin, "Discrete-Time local iterative adaptive dynamic programming: Terminations and admissibility analysis, " IEEE Trans. Neural Netw. Learn. Syst., article in press, 2016. DOI: 10.1109/TNNLS.2016.2593743.
25. Q. Wei, R. Song, and P. Yan, "Data-driven zero-sum neuro-optimal control for a class of continuous-Time unknown nonlinear systems with disturbance using ADP, " IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 2, pp. 444-458, Feb. 2016.
26. H. Zhang, C. Qin, B. Jiang, and Y. Luo, "Online adaptive policy learning algorithm for H1 state feedback control of unknown affine nonlinear discrete-Time systems, " IEEE Trans. Cybern., vol. 44, no. 12, pp. 2706-2718, Dec. 2014.
27. F.-Y. Wang and G. N. Saridis, "Suboptimal control for nonlinear stochastic systems, " Proc. 31st IEEE Conf. Decision Control, 1992.
28. G. N. Saridis and F.-Y. Wang, "Suboptimal control of nonlinear stochastic systems, " Control Theory and Advanced Technology, vol. 10, no. 4, pp. 847-871, 1994.
29. Q. Wei, D. Liu, and X. Yang, "Infinite horizon self-learning optimal control of nonaffine discrete-Time nonlinear systems, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 4, pp. 866-879, Apr. 2015.
30. Q. Wei, D. Liu, Y. Liu, and R. Song, "Optimal constrained self-learning battery sequential management in microgrid via adaptive dynamic programming, " IEEE/CAA J. Autom. Sinica, article in press, 2016. DOI: 10.1109/JAS.2016.7510262.
31. Q. Zhao, H. Xu, and S. Jagannathan, "Near optimal output feedback control of nonlinear discrete-Time systems based on reinforcement neural network learning, " IEEE/CAA J. Autom. Sinica, vol. 1, no. 4, pp. 372-384, Oct. 2014.
32. Q. Wei, D. Liu, G. Shi, and Y. Liu, "Optimal multi-battery coordination control for home energy management systems via distributed iterative adaptive dynamic programming, " IEEE Trans. Ind. Electron., vol. 42, no. 7, pp. 4203-4214, Jul. 2015.
33. Q. Wei, D. Liu, and H. Lin, "Value iteration adaptive dynamic programming for optimal control of discrete-Time nonlinear systems, " IEEE Trans. Cybern., vol. 46, no. 3, pp. 840-853, Mar. 2016.
34. Q. Wei, F. Wang, D. Liu, and X. Yang, "Finite-Approximation-error based discrete-Time iterative adaptive dynamic programming, " IEEE Trans. Cybern., vol. 44, no. 12, pp. 2820-2833, Dec. 2014.
35. H. Li and D. Liu, "Optimal control for discrete-Time affine non-linear systems using general value iteration, " IET Control Theory Appl., vol. 6, no. 18, pp. 2725-2736, Dec. 2012.
36. W. Gao and Z.-P. Jiang, "Adaptive dynamic programming and adaptive optimal output regulation of linear systems, " IEEE Trans. Autom. Control, vol. 61, no. 12, pp. 4164-4169, Dec. 2016.
37. Y. Duan, Y. Lv, J. Zhang, X. Zhao, and F.-Y. Wang, "Deep learning for control: The state of the art and prospects, " Acta Autom. Sinica, vol 42, no. 5, pp. 643-654, 2016.
38. F.-Y. Wang, "Building knowledge structure in neural nets using fuzzy logic, " Robotics and Manufacturing: Recent Trends in Research Education and Applications, M. Jamshidi (Eds.), New York, NY, ASME (American Society of Mechanical Engineers) Press, 1992.
39. F.-Y. Wang and H.-A. Kim, "Implementing adaptive fuzzy logic controllers with neural networks: A design paradigm, " J. Intell. Fuzzy Syst., vol. 3, no. 2, pp. 165-180, 1995.
40. F.-Y. Wang, "The emergence of intelligent enterprises: From CPS to CPSS, " IEEE Intell. Syst., vol. 25, no. 4, pp. 85-88, 2010.
41. C. Nyce, "Predictive analytics white paper, " American Institute for Chartered Property Casualty Underwriters/Insurance Institute of America, 2007.
42. W. Eckerson, "Extending the value of your data warehousing investment, " The Data Warehouse Institute, USA, 2007.
43. J. R. Evans and C. H. Lindner, "Business analytics: The next frontier for decision sciences, " Decision Line, vol. 43, no. 2, pp. 1-4, Mar. 2012.
44. J. Zhang, Q. Wei, and F.-Y. Wang, "Parallel dynammic programming with an average-greedy mechanism for discrete systems, " SKLMCCS/QAII Tech Report 01-09-2016, ASIA, Beijing, China.
45. F.-Y. Wang, "Parallel control: A method for data-driven and computational control, " Acta Autom.a Sinica, vol.39, no. 2, pp. 293-302, 2013.
46. F.-Y. Wang, "Control 5.0: From Newton to Merton in Poppers Cyber-Social-Physical Spaces, " IEEE/CAA J. Autom. Sinica, vol. 3, no. 3, pp. 233-234, 2016.