A hybrid linear/nonlinear training algorithm for feedforward neural networks

IEEE Transactions on Neural Networks

Volumn 9, Issue 4, 1998, Pages 669-684

  McLoone, Seán ^b   Brown, Michael D ^c   Irwin, George ^a,b   Lightbody, Gordon ^d  

^a IEEE

^b QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

^c UNIVERSITY OF LEEDS   (United Kingdom)

^d UNIVERSITY COLLEGE CORK   (Ireland)

Author keywords

Feedforward neural networks; Off line training algorithms; Second order methods

Indexed keywords

COMPUTER SIMULATION; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; OPTIMIZATION;

LOCAL MODEL NETWORKS (LMN); OFF LINE TRAINING ALGORITHMS; SINGULAR VALUE DECOMPOSITION (SVD);

FEEDFORWARD NEURAL NETWORKS;

EID: 0032123629     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.701180     Document Type: Article

References (33)

1. R. Battiti and F. Masulli, "BFGS optimization for faster automated supervised learning," in Int. Neural-Network Conf., vol. 2, 1990, pp. 757-760.
2. R. Battiti, "First- and second-order methods for learning: Between steepest descent and newton's method," Neural Comput., vol. 4, pp. 141-166, 1992.
3. M. D. Brown, G. W. Irwin, and G. Lightbody, "Local model networks for nonlinear system identification," in Proc. 11th IFAC Syst. Identification Conf., SYSID '97, Fukuoka, Japan, vol. 2, July 1997, pp. 709-714.
4. C. L. Chen, W. C. Chen, and F. Y. Chang, "Hybrid learning algorithm for Gaussian potential function networks," Proc. Inst. Elect. Eng., vol. 140, pt. D, no. 6, pp. 442-448, 1993.
5. S. Chen, S. A. Billings, and P. M. Grant, "Recursive hybrid algorithm for nonlinear system identification using radial basis function networks," Int. J. Contr., vol. 55, no. 5, pp. 1051-1070, 1992.
6. S. Chen, C. F. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 302-309, 1991.
7. S. E. Fahlman, "Fast learning variations on backpropagation: An empirical study," in Proc. 1988 Connectionist Models Summer School, D. S. Touretzky, G. Hinton, and T. Sejnowski, 1988, pp. 38-51.
8. S. Haykin, Adaptive Filter Theory, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1991.
9. S. J. Huang and S. L. Huang, "Static security assessment of a large-scale power system using genetic-enhanced neural-network approaches," in Proc. Nat. Sci. Council, China, Part A: Physical Science and Engineering, vol. 20, no. 2, Mar. 1996, pp. 228-235.
10. A. Jacobs, "Increased rates of convergence through learning adaptation," Neural Networks, vol. 1, pp. 295-307, 1988.
11. T. A. Johansen and B. A. Foss, "Constructing NARMAX models using ARMAX models," Int. J. Contr., vol. 58, no. 5, pp. 1125-1153, 1993.
12. _, "State-space modeling using operating regime decomposition and local models," in Proc. IFAC World Congr., vol. 4, no. 251, pp. 39-42, 1993.
13. _, "Identification of nonlinear system structure and parameters using regime decomposition," Automatica, vol. 31, no. 2, pp. 321-326, 1995.
14. T. A. Johansen, B. A. Foss, and A. V. Sorensen, "Nonlinear predictive control using local models - applied to a batch fermentation process," Contr. Eng. Practice, vol. 3, no. 3, pp. 389-396, 1995.
15. G. Lightbody, "Identification and control using neural networks," Ph.D. dissertation, Advanced Control Engineering Research Group, Queen's Univ. Belfast, U.K., May 1993.
16. S. McLoone and G. W. Irwin, "Fast gradient based off-line training of multilayer Perceptrons," in Neural-Network Engineering in Dynamic Control Systems, K. J. Hunt, G. W. Irwin, and K. Warwick, Eds. New York: Springer-Verlag, 1995.
17. S. F. McLoone and G. W. Irwin, "Fast parallel off-line training of multilayer perceptrons," IEEE Trans. Neural Networks, vol. 8, pp. 646-653, May 1997.
18. S. F. McLoone, "Neural-Network Training for Modeling and Control," Ph.D. dissertation, Advanced Control Engineering Research Group, Queen's University of Belfast, U.K., Sept. 1996.
19. S. F. McLoone and G. W. Irwin, "Improving neural-network training solutions using regularization," submitted to Neurocomputing, 1997.
20. M. F. Møller, "A scaled conjugate gradient algorithm for fast supervised learning," Neural Networks, vol. 6, pp. 525-533, 1993.
21. O. Nelles, S. Sinsel, and R. Isermann, "Local basis function networks for identification of a turbocharger," in Proc. UKACC Int. Conf. Contr. '96, Exeter, U.K., July 1996, pp. 7-12.
22. A. G. Parlos, B. Fernandez, F. Atiya, J. Muthusami, and W. K. Tsai, "An accelerated learning algorithm for multilayer perceptron networks," IEEE Trans. Neural Networks, vol. 5, pp. 493-397, 1994.
23. L. Prechelt, "Proben 1 - A set of neural network benchmark problems and benchmarking rules," Tech. Rep. 21/94, Fakultät für Informatik, Univ. Karlsruhe, Germany, Sept 1994. Available FTP: ftp.ira.uka.de in /pub/neuron/proben1.tar.gz
24. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart and J. L. McClelland, Eds., vol. 1. Cambridge, MA: MIT Press, 1986, ch. 8.
25. W. Schiffmann, M. Joost, and R. Werner, "Optimization of the backpropagation algorithm for training multilayer perceptrons," Univ. Koblenz, Inst. Physics, Rheinau 3-4, W-5400 Koblenz, Germany.
26. J. Sjöberg and L. Ljung, "Overtraining, regularization, and searching for minimum with application to neural networks," Int. J. Contr., vol. 62, no. 6, pp. 1391-1407.
27. T. Ichimura, T. Takano, and E. Tazaki, "Learning of neural networks using hybrid genetic algorithm," in 4th European Congr. Intell. Techniques and Soft Computing (EUFIT'96), 1996.
28. P. Van Der Smagt, "Minimization methods for training feedforward neural networks," Neural Networks, vol. 7, no. 1, pp. 1-11, 1994.
29. P. D. Wasserman, Neural Computing, Theory and Practice. New York: Van Nostrand Reinhold, 1989.
30. P. B. Watta, M. H. Hassoun, and J. Meisel, "Design of optimal neurocontrollers for the separately excited dc motor using a hybrid genetic algorithm-neural network approach," in Proc. SPIE - Int. Soc. Opt. Eng., vol. 2760, pp. 230-241, 1996.
31. A. R. Webb, D. Lowe, and M. D. Bedworth, "A comparison of nonlinear optimization strategies for feedforward adaptive layered networks," Royal Signals and Radar Establishment, Memorandum 4157, Controller HMSO, London, U.K., 1988.
32. L. F. Wessels and E. Barnard, "Avoiding false local minima by proper initialization of connections," IEEE Trans. Neural Networks, vol. 3, pp. 899-905, 1992.
33. M. J. Willis, C. Di Massimo, G. A. Montague, and G. A. Tham, "Artificial neural networks in process engineering," Proc. Inst. Elect. Eng., Control Theory and Applicat., vol. 128, no. 3, pp. 256-204, 1991.