Fast parallel off-line training of multilayer perceptrons

IEEE Transactions on Neural Networks

Volumn 8, Issue 3, 1997, Pages 646-653

  McLoone, Seán ^b   Irwin, George W ^a,b  

^a IEEE

^b QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

Author keywords

Feedforward neural networks; Layered perceptron; Multilayer perceptrons; Neural networks; Parallel processors; Parallel training algorithms; Supervised training

Indexed keywords

COMPUTATIONAL METHODS; COMPUTER ARCHITECTURE; CONSTRAINT THEORY; LEARNING ALGORITHMS; MATRIX ALGEBRA; PARALLEL PROCESSING SYSTEMS; STORAGE ALLOCATION (COMPUTER); TRANSPUTERS;

FULL MEMORY BROYDEN-FLETCHER-GOLDFARB-SHANNO ALGORITHMS; LIMITED BROYDEN-FLETCHER-GOLDFARB-SHANNO ALGORITHMS; PARALLEL TRAINING ALGORITHMS; PARALLEL VIRTUAL MACHINE (PVM);

NEURAL NETWORKS;

EID: 0031140463     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.572103     Document Type: Article

References (26)

1. K. Warwick, G. W. Irwin, and K. J. Hunt, Eds., Neural Networks for Control and Systems. Stevenage, U.K.: Peter Peregrinus, 1992.
2. K. J. Hunt, D. Sbarbaro, R. Zbikowski, and P. J. Gawthrop, "Neural networks for control systems - A survey," Automatica, vol. 28, no. 6, pp. 1083-1112, 1992.
3. D. E. Seborg, M. A. Henson, and E. P. Nahas, "Nonlinear internal model control strategy for neural network models," Comp. Chem. Eng., vol. 16, no. 12, pp. 1039-1057, 1992.
4. C. J. Harris, C. G. Moore, and M. Brown, Intelligent Control Aspects of Fuzzy Logic and Neural Nets. London, UK: World Scientific, 1992.
5. G. W. Irwin, M. D. Brown, B. W. Hogg, and E. Swidenbank, "Neural-network modeling of a 200 MW boiler system," IEE Proc. D, to appear.
6. D. McDowell, G. W. Irwin, and G. McConnell, "Online neural control applied to a bank-to-turn missile autopilot," in Proc. AIAA Guidance, Navigation Cont. Conf., Baltimore, MD, Aug. 1995, vol. 2, pp. 1286-1294.
7. P. E. An, M. Brown, C. J. Harris, and S. Chen, "Comparative aspects of neural network algorithms for on-line modeling of dynamic processes," Proc. Inst. Mech. Eng., vol. 207, 1993, pp. 223-241.
8. S. W. Piché, "Steepest descent algorithms for neural network controllers and filters," IEEE Trans. Neural Networks, vol. 5, pp. 198-212, Mar. 1994.
9. M. F. Møller, "Supervised learning on large redundant training sets," Int. J. Neural Syst., vol. 4, no. 1, pp. 15-25, 1993.
10. S. McLoone and G. W. Irwin, "Fast gradient-based off-line training of multilayer perceptrons," in Neural Network Engineering in Dynamic Control Systems, K. J. Hunt, G. W. Irwin, and K. Warwick, Eds. New York: Springer, 1995, ch. 9, pp. 279-200.
11. _, "Parallel off-line training of multilayer perceptrons," in Proc. 3rd IFAC/IFIP Wkshp. Algorithms Architectures Real-Time Cont., Ostend, Belgium, May 1995.
12. _, "Efficient step-size evaluation in gradient-based neural-network training algorithms," in Proc. Irish DSP Cont. Colloquium, June 1995, pp. 315-322.
13. D. E. Rumelhart, G. Hinton, and R. Williams, "Learning internal representations by error propagation," Parallel Distributed Processing, D. E Rumelhart and J. L. McClelland, Ed., vol. 1. Cambridge, MA: MIT Press, 1986, pp. 318-364.
14. R. Battiti, "Accelerated backpropagation learning: Two optimization methods," Complex Syst. 3, pp. 331-342, 1989.
15. R. A. Jacobs, "Increased rates of convergence through learning adaption," Neural Networks, vol. 1, pp. 295-307, 1988.
16. W. Schiffmann, M. Joost, and R. Werner, Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons, Univ. Koblenz, Germany, Instit. Physics, Tech. Rep., 1992.
17. S. E. Fahlman, "An empirical study of learning speed in backpropagation networks," Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-88-162, 1988.
18. J. J. McKeown, D. Meegan, and D. Sprevak, An Introduction to Unconstrained Optimization. Bristol, U.K.: Adam Hilger, 1990.
19. A. R. Webb, D. Lowe, and M. D. Bedworth, "A comparison of nonlinear optimization strategies for feedforward adaptive layered networks," Royal Signals and Radar Establishment Memorandum 4157, 1988.
20. M. F. Møller, "A scaled conjugate gradient algorithm for fast supervised learning," Neural Networks, vol. 6, pp. 525-533, 1990.
21. R. Fletcher, Practical Methods of Optimization. New York: Wiley, 1980, vol. 1, p. 51.
22. P. E. Gill, W. Murray, and M. H. Wrights, Practical Optimization. London: Academic, 1981.
23. S. McLoone and G. W. Irwin, "Insights into multilayer perceptrons and their training," in Proc. Irish DSP and Control Colloquium, 1994, pp. 61-68.
24. G. Lightbody and G. W. Irwin, "A parallel algorithm for training neural-network-based nonlinear models," in Proc. 2nd IFAC Wkshp. Algorithms Architectures for Real-Time Contr., 1992, pp. 99-104.
25. A. Beguelin, J. J. Dongarra, G. A. Geist, W. Jiang, R. Manchek, K. Moore, and V. S. Sunderam, PVM 3 User's Guide and Reference Manual. Oak Ridge, TN: Oak Ridge Nat. Lab., 1993.
26. G. Lightbody, G. W. Irwin, A. Taylor, K. Kelly, and J. McCormick, "Neural network modeling of a polymerization reactor," in Proc. IEE Int. Conf., Cont. '94, vol. 1, pp. 237-242.