Second-order training of adaptive critics for online process control

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 35, Issue 2, 2005, Pages 381-385

  Govindhasamy, James J ^a   McLoone, Seán F ^b   Irwin, George W ^a  

^a QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

^b NATIONAL UNIVERSITY OF IRELAND   (Ireland)

Author keywords

Action dependent adaptive critic; Intelligent control; Multilayer perceptrons; Neural networks; Nonlinear process control; Process optimization; Reinforcement learning

Indexed keywords

APPROXIMATION THEORY; CHEMICAL REACTORS; COMPUTER SIMULATION; IDENTIFICATION (CONTROL SYSTEMS); INTELLIGENT CONTROL; LEARNING ALGORITHMS; MATHEMATICAL MODELS; MULTILAYER NEURAL NETWORKS; NONLINEAR CONTROL SYSTEMS; OPTIMAL CONTROL SYSTEMS; PROCESS CONTROL;

ACTION DEPENDENT ADAPTIVE CRITIC; CONTINUOUSLY STIRRED TANK REACTOR PROCESS; NONLINEAR PROCESS CONTROL; ONLINE PROCESS CONTROL; PROCESS OPTIMIZATION; REINFORCEMENT LEARNING; TEMPORAL DIFFERENCE STOCHASTIC BACKPROPAGATION;

ADAPTIVE CONTROL SYSTEMS;

ALGORITHM; ARTIFICIAL INTELLIGENCE; BIOREACTOR; COMPUTER SIMULATION; EVALUATION; FEEDBACK SYSTEM; LETTER; ONLINE SYSTEM; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIOREACTORS; COMPUTER SIMULATION; FEEDBACK; MODELS, THEORETICAL; ONLINE SYSTEMS;

EID: 17444428905     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2004.843276     Document Type: Article

References (28)

1. P. J. Werbos, "A menu of designs for reinforcement learning over time," in Neural Networks for Control, W. T. Miller, R. S. Sutton, and P. J. Werbos, Eds. Cambridge, MA: MIT Press, 1990, pp. 67-95.
2. P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling," in Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. New York: Van-Nostrand Reinhold, 1992, pp. 493-525.
3. Q. H. Wu and A. C. Pugh, "Reinforcement learning control of unknown dynamic systems," Proc. Inst. Elect. Eng. D, vol. 140, pp. 313-322, 1993.
4. Q. H. Wu, "Reinforcement learning control using interconnected learning automata," Int. J. Control, vol. 62, no. 1, pp. 1-16, 1995.
5. K. H. Chan, L. Jiang, P. Tiloston, and Q. H. Wu, "Reinforcement learning for the control of large-scale power systems," in Proc. 2nd Int. Symp. Engineering of Intelligent Systems (EIS'2000), Paisley, U.K., 2000.
6. X. Zeng, J. Zhou, and C. Vasseur, "A strategy for controlling nonlinear systems using a learning automaton," Automatica, vol. 36, pp. 1517-1524, 2000.
7. M. Riedmiller, "Concepts and facilities of a neural reinforcement learning control architecture for technical process control," in Neural Computing and Applications. London, U.K.: Springer-Verlag, 1999, vol. 8, pp. 323-338.
8. D. Ernst, M. Glavic, and L. Wehenkel, "Power system stability control: Reinforcement learning framework," IEEE Trans. Power Syst., vol. 19, no. 1, pp. 427-436, Feb. 2004.
9. J. W. Park, R. G. Harley, and G. K. Venayagamoorthy, "Adaptive critic designs and their implementations on different neural network architectures," in Proc. Int. Joint Conf. Neural Networks, vol. 3, 2003, pp. 1879-1884.
10. J. W. Park, "Adaptive critic based optimal neurocontrol for synchronous generator in power system using MLP/RBF neural networks," in Conf. Record - IAS Annu. Meeting, vol. 2, 2002, pp. 1447-1454.
11. G. K. Venayagamoorthy, R. G. Harley, and D. C. Wunsch, "A nonlinear voltage controller with derivative adaptive critics for multimachine power systems," in Proc. IEEE Power Industry Computer Applications Conf., 2001, pp. 324-329.
12. G. K. Venayagamoorthy, "Excitation and turbine neurocontrol with derivative adaptive critics of multiple generators on the power grid," in Proc. Int. Joint Conf. Neural Networks, vol. 2, 2001, pp. 984-989.
13. G. K. Venayagamoorthy, "Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogenerator," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 764-773, May 2002.
14. M. S. Iyer and D. C. Wunsch II, "Dynamic re-optimization of a fed-batch fermentor using adaptive critic designs," IEEE Trans. Neural Netw. vol. 12, no. 6, pp. 1433-1444, Nov. 2001.
15. P. Radhakant and S. N. Balakrishnan, "Proper orthogonal decomposition based optimal neurocontrol synthesis of a chemical reactor process using approximate dynamic programming," Neural Netw., vol. 16, pp. 719-728, 2003.
16. D. A. Sofge and D. A. White, "Neural network based process optimization and control," in Proc. IEEE Conf. Decision and Control, vol. 6, 1990, pp. 3270-3276.
17. J. C. Hoskins and D. M. Himmelblau, "Process control via artificial neural networks and reinforcement learning," Comput. Chem. Eng., vol. 16, no. 4, pp. 241-251, 1992.
18. C. W. Anderson, "Learning and Problem Solving With Mulitlayer Connectionist Systems," Ph.D. dissertation, Dept. Comput. Inform. Sci., Univ. Massachusetts, Amherst, 1986.
19. C. W. Anderson, "Strategy Learning With Multilayer Connectionist Representations," GTE Laboratories, Waltham, MA, Tech. Rep. TR87-507.3, 1987.
20. R. E. Bellman, Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957.
21. J. Si and Y. T. Wang, "Online learning control by association and reinforcement," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 264-276, Mar. 2001.
22. R. S. Sutton, "Learning to predict by the method of temporal differences," in Mach. Learn., 1988, vol. 3, pp. 9-44.
23. S. McLoone, "Neural network identification: A survey of gradient based methods," in IEE Colloq. Optimization in Control: Methods and Applications, London, U.K., Nov. 1998, Dig. 98/521.
24. L. S. H. Ngia and J. Sjöberg, "Efficient training of neural nets for nonlinear adaptive filtering using a recursive Levenberg- Marquardt algorithm," IEEE Trans. Signal Process., vol. 48, no. 7, pp. 1915-1926, Jul. 2000.
25. J. D. Morningred, B. E. Paden, D. E. Seborg, and D. A. Mellichamp, "An adaptive nonlinear predictive controller," Chem. Eng. Sci., vol. 47, no. 4, pp. 755-762, 1992.
26. J. D. Morningred, "An adaptive nonlinear predictive controller," in Proc. American Control Conf. (ACC), vol. 2, 1990, pp. 1614-1619.
27. J. A. Rovlak and R. Corlis, "Dynamic matrix based control of fossil power plants," IEEE Trans. Energy Convers., vol. 6, no. 2, pp. 320-326, Jun. 1991.
28. G. W. Irwin, P. O'Reilly, G. Ligthbody, M. Brown, and E. Swidenbank, "Electrical power and chemical process applications," in Neural Network Applications in Control. ser. IEE Control Engineering Series 53, G. W. Irwin, K. Warwick, and K. J. Hunt, Eds. London, U.K.: IEE, 1995.