Training Recurrent Neural Networks with the Levenberg-Marquardt Algorithm for Optimal Control of a Grid-Connected Converter

IEEE Transactions on Neural Networks and Learning Systems

Volumn 26, Issue 9, 2015, Pages 1900-1912

  Fu, Xingang ^a   Li, Shuhui ^a   Fairbank, Michael ^b   Wunsch, Donald C ^c   Alonso, Eduardo ^b  

^a University of Alabama   (United States)

^b CITY UNIVERSITY   (United Kingdom)

^c University of Missouri Rolla   (United States)

Author keywords

Backpropagation through time (BPTT); d q vector control; dynamic programming (DP); forward accumulation through time (FATT); grid connected converter (GCC); Jacobian matrix; Levenberg Marquardt (LM); optimal control; Recurrent neural network (RNN)

Indexed keywords

ALGORITHMS; BACKPROPAGATION; BACKPROPAGATION ALGORITHMS; DYNAMIC PROGRAMMING; JACOBIAN MATRICES; RECURRENT NEURAL NETWORKS; VECTOR CONTROL (ELECTRIC MACHINERY);

BACKPROPAGATION THROUGH TIME; D-Q VECTOR CONTROLS; FORWARD ACCUMULATION THROUGH TIME (FATT); GRID-CONNECTED CONVERTERS; LEVENBERG-MARQUARDT; OPTIMAL CONTROLS; RECURRENT NEURAL NETWORK (RNN);

CONTROLLERS;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SYSTEM; HUMAN; LEARNING; PROCEDURES; TIME FACTOR;

ALGORITHMS; COMPUTER SYSTEMS; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; TIME FACTORS;

EID: 84939811733     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2014.2361267     Document Type: Article

References (37)

1. G. Buticchi, D. Barater, E. Lorenzani, and G. Franceschini, "Digital control of actual grid-connected converters for ground leakage current reduction in PV transformerless systems," IEEE Trans. Ind. Inform., vol. 8, no. 3, pp. 563-572, Aug. 2012.
2. J.-M. Shen, H.-L. Jou, and J.-C. Wu, "Novel transformerless gridconnected power converter with negative grounding for photovoltaic generation system," IEEE Trans. Power Electron., vol. 27, no. 4, pp. 1818-1829, Apr. 2012.
3. X. Zhou, S. Lukic, S. Bhattacharya, and A. Huang, "Design and control of grid-connected converter in bi-directional battery charger for plug-in hybrid electric vehicle application," in Proc. IEEE Veh. Power Propuls. Conf., Sep. 2009, pp. 1716-1721.
4. S. Li, T. A. Haskew, Y.-K. Hong, and L. Xu, "Direct-current vector control of three-phase grid-connected rectifier-inverter," Electr. Power Syst. Res., vol. 81, no. 2, pp. 357-366, Feb. 2011.
5. S. Li, M. Fairbank, D. C. Wunsch, and E. Alonso, "Vector control of a grid-connected rectifier/inverter using an artificial neural network," in Proc. IEEE World Congr. Comput. Intell., Brisbane, Australia, Jun. 2012, pp. 1-7.
6. S. Li, M. Fairbank, C. Johnson, D. C. Wunsch, E. Alonso, and J. L. Proao, "Artificial neural networks for control of a grid-connected rectifier/inverter under disturbance, dynamic and power converter switching conditions," IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 4, pp. 738-750, Apr. 2014.
7. S. Li, M. Fairbank, X. Fu, D. C. Wunsch, and E. Alonso, "Nestedloop neural network vector control of permanent magnet synchronous motors," in Proc. Int. Joint Conf. Neural Netw., Dallas, TX, USA, Aug. 2013, pp. 1-8.
8. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Comput., vol. 1, no. 2, pp. 270-280, 1989.
9. R. J. Williams and D. Zipser, "Gradient-based learning algorithms for recurrent networks and their computational complexity," in Back-Propagation: Theory, Architectures and Applications. Hillsdale, NJ, USA: Erlbaum, 1995, ch. 13, pp. 433-486.
10. H. Jaeger, "A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the 'echo state network' approach," German Nat. Res. Center Inf. Technol., Saint Augustin, German, Tech. Rep. GMD 159, 2002.
11. S. Li, D. C. Wunsch, E. O'Hair, and M. G. Giesselmann, "Extended Kalman filter training of neural networks on a SIMD parallel machine," J. Parallel Distrib. Comput., vol. 62, no. 4, pp. 544-562, Apr. 2002.
12. G. V. Puskorius and L. A. Feldkamp, "Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks," IEEE Trans. Neural Netw., vol. 5, no. 2, pp. 279-297, Mar. 1994.
13. R. Ilin, R. Kozma, and P. J. Werbos, "Beyond feedforward models trained by backpropagation: A practical training tool for a more efficient universal approximator," IEEE Trans. Neural Netw., vol. 19, no. 6, pp. 929-937, Jun. 2008.
14. K. Levenberg, "A method for the solution of certain non-linear problems in least squares," Quart. J. Appl. Math., vol. 2, no. 2, pp. 164-168, 1944.
15. D.W. Marquardt, "An algorithm for least-squares estimation of nonlinear parameters," J. Soc. Ind. Appl. Math., vol. 11, no. 2, pp. 431-441, Jun. 1963.
16. M. T. Hagan and M. B. Menhaj, "Training feedforward networks with the Marquardt algorithm," IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 989-993, Nov. 1994.
17. Y. Tanoto, W. Ongsakul, and C. O. P. Marpaung, "Levenberg-Marquardt recurrent networks for long-term electricity peak load forecasting," TELKOMNIKA Telecommun., Comput., Electron. Control, vol. 9, no. 2, pp. 257-266, Aug. 2011.
18. L.-W. Chan and C.-C. Szeto, "Training recurrent network with blockdiagonal approximated Levenberg-Marquardt algorithm," in Proc. Int. Joint Conf. Neural Netw., vol. 3. Washington, DC, USA, 1999, pp. 1521-1526.
19. L.-W. Chan and C.-C. Szeto, "Weight groupings in the training of recurrent networks," in Proc. Int. Joint Conf. Neural Netw., vol. 3. Como, Italy, 2000, pp. 21-26.
20. D. Zhi, L. Xu, and B. W. Williams, "Improved direct power control of grid-connected DC/AC converters," IEEE Trans. Power Electron., vol. 24, no. 5, pp. 1280-1292, May 2009.
21. J. A. Restrepo, J. M. Aller, J. C. Viola, A. Bueno, and T. G. Habetler, "Optimum space vector computation technique for direct power control," IEEE Trans. Power Electron., vol. 24, no. 6, pp. 1637-1645, Jun. 2009.
22. N. Mohan, T. M. Undeland, and W. P. Robbins, Power Electronics: Converters, Applications, and Design, 3rd ed. New York, NY, USA: Wiley, Oct. 2002.
23. R. E. Bellman, Dynamic Programming. Princeton, NJ, USA: Princeton Univ. Press, 1957.
24. D. V. Prokorov and D. C.Wunsch, "Adaptive critic designs," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 997-1007, Sep. 1997.
25. M. Fairbank, S. Li, X. Fu, E. Alonso, and D. Wunsch, "An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances," Neural Netw., vol. 49, pp. 74-86, Jan. 2013. [Online]. Available: http://dx.doi.org/10.1016/j.neunet.2013.09.010
26. S. Haykin, Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ, USA: Prentice-Hall, 1999.
27. P. J. Werbos, "Backpropagation through time: What it does and how to do it," Proc. IEEE, vol. 78, no. 10, 1550-1560, Oct. 1990.
28. P. Werbos, T. McAvoy, and T. Su, "Neural networks, system identification, and control in the chemical process industries," in Handbook of Intelligent Control, D. A. White and D. A. Sofge, Eds. New York, NY, USA: Van Nostrand Reinhold, 1992, ch. 10, sec. 10.6.1-10.6.2, pp. 339-343. [Online]. Available: http://www.werbos.com
29. P. J. Werbos, "Backwards differentiation in ad and neural nets: Past links and new opportunities," in Automatic Differentiation: Applications, Theory, and Implementations (Lecture Notes in Computational Science and Engineering), H. M. Bücker, G. Corliss, P. Hovland, U. Naumann, and B. Norris, Eds. Berlin, Germany: Springer-Verlag, 2005, pp. 15-34.
30. M. T. Hagan, H. B. Demuth, and M. H. Beale, Neural Network Design. Boston, MA, USA: PWS-Kent, 2002, ch. 12, pp. 19-23.
31. M. Fairbank and E. Alonso, "Efficient calculation of the Gauss-Newton approximation of the Hessian matrix in neural networks," Neural Comput., vol. 24, no. 3, pp. 607-610, 2012.
32. W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, Oct. 1992, p. 994.
33. A. Mullane, G. Lightbody, and R. Yacamini, "Wind-turbine fault ride-through enhancement," IEEE Trans. Power Syst., vol. 20, no. 4, pp. 1929-1937, Nov. 2005.
34. R. Pena, J. C. Clare, and G. M. Asher, "Doubly fed induction generator using back-to-back PWM converters and its application to variable-speed wind-energy generation," IEE Proc.-Electr. Power Appl., vol. 143, no. 3, pp. 231-241, May 1996.
35. S. Li, T. A. Haskew, and L. Xu, "Control of HVDC light system using conventional and direct current vector control approaches," IEEE Trans. Power Electron., vol. 25, no. 12, pp. 3106-3118, Dec. 2010.
36. M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm," in Proc. IEEE Int. Conf. Neural Netw., San Francisco, CA, USA, 1993, pp. 586-591.
37. G. F. Franklin, J. D. Powell, and M. L. Workman, Digital Control of Dynamic Systems, 3rd ed. Reading, MA, USA: Addison-Wesley, 1998.