Parallel implementation of backpropagation neural networks on a heterogeneous array of transputers

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 27, Issue 1, 1997, Pages 118-126

  Foo, Shou King ^a   Saratchandran, P ^a   Sundararajan, N ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

BACKPROPAGATION; INTEGER PROGRAMMING; MONTE CARLO METHODS; OPTIMIZATION; PARALLEL ALGORITHMS; PARALLEL PROCESSING SYSTEMS; RESPONSE TIME (COMPUTER SYSTEMS); TRANSPUTERS;

BENCHMARK PROBLEM; EPOCH TIME; PIPELINE RING NETWORK TOPOLOGY; TRAINING SET PARALLELISM;

NEURAL NETWORKS;

EID: 0031076917     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/3477.552191     Document Type: Article

References (25)

1. D. E. Rumelhart et al., "Learning representations by back propagating errors," Nature, vol. 323, pp. 533-536, 1986.
2. Y. Fujimoto, N. Fakuda, and T. Akabane, "Massively parallel architectures for large scale neural network simulations," IEEE Trans. Neural Networks, vol. 3, no. 6, pp. 876-888, 1992.
3. S. W. Aiken et al., "A parallel neural network simulator," in Proc. ICNN-90, San Diego, CA, 1990, vol. 2, pp. 611-616.
4. R. Battiti and A. N. Colla, "Parallel learning strategies on the Emma-2 multiprocessor," in Artificial Neural Networks: Proc. ICANN-91, T. Kohonen et al., Eds. Amsterdam, The Netherlands: Elsevier, 1991, vol. 2, pp. 1493-1500.
5. S.-Y. Kung and W.-H. Chou, "Mapping neural networks onto VLSI array processors," Parallel Digital Implementation of Neural Networks, K. W. Przytula and V. K. Prasanna, Eds. Englewood Cliffs, NJ: Prentice-Hall, 1993, pp. 77-104.
6. X. Zhang et al., "An efficient implementation of the backpropagation algorithm on the connection machine CM-2," Tech. Rep. RL-89-1, Thinking Machines Corp., Cambridge, MA.
7. Also in Advances in NIPS 2, D. S. Touretzkey, Ed. San Mateo, CA: Morgan Kauffman, 1989, pp. 801-809.
8. D. A. Pomerleau et al., "Neural network simulation at warp speed: How we got 17 million connections per second," in Proc. IEEE ICNN, 1988, vol. 2, pp. 143-150.
9. J. M. J. Murre, "Transputers and neural networks: An analysis of implementation constraints and performance," IEEE Trans. Neural Networks, vol. 4, no. 2, pp. 284-292, 1993.
10. H. Muhlenbein and K. Wolf, "Neural network simulation on parallel computers," Parallel Computing 89, D. J. Evans, G. R. Joubert, and F. J. Peters, Eds. Amsterdam, The Netherlands: Elsevier, 1989, pp. 365-374.
11. M. Witbrock and M. Zagha, "Backpropagation learning on the IBM GF 11," in Parallel Digital Implementation of Neural Networks, K. W. Przytula and V. K. Prasanna, Eds. Englewood Cliffs, NJ: Prentice-Hall, 1993, pp. 77-104.
12. E. Baum and D. Hassler, "What size net gives valid generalization?," Neural Computat., vol. 1, no. 1, pp. 151-160, 1989.
13. S. E. Fahlman, "An empirical study of learning speed in backpropagation networks," Tech. Rep. CMU-CS-88-162, Carnegie Mellon University, Pittsburgh, PA, 1988.
14. K. W. Przytula and V. K. Prasanna, Parallel Digital Implementation of Neural Networks. Englewood Cliffs, NJ: Prentice-Hall, 1993.
15. R. C. Eberhart and R. W. Dobbins, Neural Network PC Tools - A Practical Guide. San Diego, CA: Academic, 1990.
16. H. Yoon, J. H. Nang, and S. R. Maeng, "Parallel simulation of multilayered neural networks on distributed-memory multiprocessors," Microprocess. Microprogr., no. 29, pp. 185-195, 1990.
17. H. M. Salkin and K. Mathur, Foundations of Integer Programming. Amsterdam, The Netherlands: Elsevier, 1989.
18. NAG Fortran Library, Mark 14, NAG, Oxford, U.K., 1990.
19. R. P. Gorman and T. J. Sejnowski, "Analysis of hidden units in a layered network trained to classify sonar targets," Neural Networks, vol. 1, pp. 75-89, 1988.
20. A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing. New York: Wiley, 1993.
21. G. Richards, "Implementation of backpropagation on a transputer array," in Developments in Occam, J. Kerridge, Ed. Amsterdam, The Netherlands: IOS, 1988, pp. 173-179.
22. S. K. Foo, P. Saratchandran, and N. Sundararajan, "Analysis of training set parallelism for backpropagation neural networks," Int. J. Neural Syst., vol. 6, no. 1, pp. 61-78, 1995.
23. M. Forti, "A neural network for signal decomposition problems," Int. J. Circuit Theory Appl., vol. 19, pp. 65-75, 1991.
24. S. Bozinovski, "A representation theorem for linear pattern classifier training," IEEE Trans. Syst., Man, Cybern., vol. 15, no. 1, pp. 159-161, 1985.
25. A. Barto and M. Duff, "Monte Carlo matrix inversion and reinforcement learning," in Advances in Neural Information Processing Systems, J. Cowan, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kauffman, 1994, pp. 655-662.