Efficient mapping of backpropagation algorithm onto a network of workstations

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 28, Issue 6, 1998, Pages 841-848

  Sudhakar, V ^a   Murthy, C Siva Ram ^a  

^a INDIAN INSTITUTE OF TECHNOLOGY MADRAS   (India)

Author keywords

Backpropagation algorithm; Distributed memory multiprocessors; Multilayered neural networks; Network of workstations; Network partitioning; Pattern partitioning; Performance analysis

Indexed keywords

BACKPROPAGATION; COMPUTER NETWORKS; COMPUTER WORKSTATIONS; DATA COMMUNICATION SYSTEMS; DATA STORAGE EQUIPMENT; LEARNING ALGORITHMS; MULTIPROCESSING SYSTEMS; RESPONSE TIME (COMPUTER SYSTEMS);

NETWORK OF WORKSTATIONS (NOW); VERTICAL PARTITIONING SCHEMES;

MULTILAYER NEURAL NETWORKS;

EID: 0032303431     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/3477.735393     Document Type: Article

References (41)

1. G. Blelloch and C. R. Rosenberg, "Network learning on the connection machine," 10th Int. Joint Conf. Artificial Intelligence, 1987, pp. 323-326.
2. L. A. Crowl, "How to measure, present, and compare parallel performance," IEEE Parallel Dist. Technol., Spring 1994, pp. 9-25.
3. E. Deprit, "Implementing recurrent backpropagation on the connection machine," Neural Networks, vol. 2, pp. 295-314, 1989.
4. S. E. Fahlman and G. E. Hinton, "Connectionist architectures for artificial intelligence," IEEE Comput., Jan. 1987, pp. 100-109.
5. J. A. Feldman et al., "Computing with structured connectionist networks," Commun. ACM, vol. 31, no. 2, pp. 170-187, 1988.
6. B. M. Forrest, D. Roweth, N. Stroud, D. J. Wallace, and G. V. Wilson, "Implementing neural network models on parallel computers," Comput. J., vol. 30, no. 5, pp. 413-419, 1987.
7. K. Fukushima, "Neocognitron: A hierarchical neural network capable of visual pattern recognition," Neural Networks, vol. 1, no. 2, pp. 119-130, 1988.
8. J. Ghosh and K. Hwang, "Mapping neural networks onto message passing multicomputers," J. Parallel Dist. Comput., vol 6, pp. 291-330, 1989.
9. R. P. Gorman and T. J. Sejnowski, "Learned classification of sonar targets using a massively parallel network," IEEE Trans. Acoust., Speech, Signal Processing, vol. 36, pp. 1135-1140, July 1988.
10. S. Haykin and C. Deng, "Classification of radar clutter using neural networks," IEEE Trans. Neural Networks, vol. 2, pp. 589-600, 1991.
11. J. Hicklin and H. Demuth, "Modeling neural networks on the MPP," in Proc. 2nd Symp. Frontiers Massively Parallel Computation, 1988, pp. 39-42.
12. B. H. Juang, S. Y. Kung, and C. A. Kamm, Eds., Neural Networks for Signal Processing, Proc. 1991 IEEE Workshop, Princeton, NJ, 1991, pp. 7-185.
13. B. Kosko, Ed., Neural Networks for Signal Processing. Englewood Cliffs, NJ: Prentice Hall, 1992.
14. V. Kumar, A. Grama, A. Gupta, and G. Karypis, Introduction to Parallel Computing - Design and Analysis of Algorithms. New York: Benjamin/Cummings, 1994.
15. V. Kumar, S. Shekhar, and M. B. Amin, "A scalable parallel formulation of the backpropagation algorithm for hypercubes and related architectures," IEEE Trans. Parallel Distrib. Syst., vol. 5, pp. 1073-1090, Oct. 1994.
16. S. Y. Kung, Digital Neural Networks. Englewood Cliffs, NJ: Prentice-Hall, 1993.
17. S. Y. Kung and J. N. Hwang, "Systolic architectures for artificial neural nets," IEEE Int. Conf. Neural Networks, 1988.
18. _, "A unified systolic architecture for artificial neural networks," J. Parallel Dist. Comput., vol. 6, pp. 358-387, 1989.
19. S. P. Luttrell, "Image compression using a multilayer neural network," Pattern Recognit., vol. 10, pp. 1-7, 1989.
20. J. Millan and P. Bofill, "Learning by backpropagation: A systolic algorithm and its transputer implementation," Neural Networks, vol. 3, pp. 119-137, 1989.
21. M. Misra and V. K. P. Kumar, "Neural network simulation on a reduced mesh of trees organization," in SPIE Symp. Electronic Imaging, 1990.
22. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Networks, vol. 1, pp. 4-27, Mar. 1990.
23. D. S. Newhall and J. C. Horvath, "Analysis of text using a neural network: A hypercube implementation," in Proc. Conf. Hypercubes, Concurrent Computers, Applications, 1989, pp. 1119-1122.
24. D. H. Nguyen and B. Widrow, "Neural networks for self learning control systems," IEEE Contr. Syst. Mag., Apr. 1990, pp. 18-23.
25. M. Nikoonahad and D. C. Liu, "Medical ultrasound imaging using neural networks," Electron. Lett., vol 26, pp. 545-546, Apr. 1990.
26. B. P. Paris, G. Orsak, M. Varanasi, and B. Aazhang, "Neural net receivers in multiple-access communications," Adv. Neural Inf. Processing, vol. 1, pp. 272-280, 1989.
27. D. A. Pomerleau, G. L. Gusciora, D. S. Touretzky, and H. T. Kung, "Neural network simulation at warp speed: How we got 17 million connections per second," Proc. Int. Conf. Systolic Arrays, 1988.
28. M. W. Roth, "Neural networks for extraction of weak targets in high clutter environments," IEEE Trans. Syst., Man, Cybern., vol. 19, pp. 1210-1217, Sept/Oct. 1989.
29. D. E. Rumelhart and J. L. McClelland, Eds., Parallel and Distributed Processing (PDP): Explorations in the Microstructure of the Cognitron. Cambridge, MA: MIT Press, 1986.
30. T. J. Sejnowski and C. R. Rosenberg, "Nettalk: A parallel network that learns to read aloud," Tech. Rep. JHU/EECS-86/01, Dept. of Elec. Engg. and Comp. Sci., John Hopkins Univ., Baltimore, MD, USA, 1986.
31. _, "Parallel networks that learn to pronounce English text," Complex Syst., vol. 1, pp. 145-168, 1987.
32. S. Shams and J. Gaudiot, "Efficient implementation of neural networks on the DREAM machine," in Proc. 11th Int. Conf. Pattern Recognition, Hague, The Netherlands, 1992, vol. 4, pp. 204-208.
33. S. Shams and K. W. Przytula, "Implementation of multilayer neural networks on parallel programmable digital computers," in M. Bayoumi, Ed., Parallel Algorithms and Architectures for DSP Applications. Norwell, MA: Kluwer, 1991, pp. 225-253.
34. V. Sunderam, "PVM: A framework for parallel and distributed computing," Concurrency, Practice, Experience, vol. 12, pp. 315-339, Dec. 1990.
35. B. W. Wah and L. Chu, "Efficient mapping of neural networks on multicomputers," Int. Conf. Parallel Processing, 1990, pp. 1234-1241.
36. B. Widrow and R. Winter, "Neural nets for adaptive filtering and adaptive pattern recognition," IEEE Computer, vol. 21, pp. 25-39, Mar. 1988.
37. H. Yoon and J. H. Nang, "Multilayer neural networks on distributed memory multiprocessors," in Proc. Int. Neural Network Conf., Paris, France, July 1990, pp. 669-672.
38. H. Yoon, J. H. Nang, and S. R. Maeng, "Parallel simulation of multilayered neural networks on distributed memory multiprocessors," Microprocess. Microprogr., vol. 29, pp. 185-195, 1990.
39. M. Zeidenberg, Neural Networks in Artificial Intelligence. London, U.K.: Ellis Horwood, 1990.
40. X. Zhang and M. McKenna, "The backpropagation algorithm on grid and hypercube architectures," Tech. Rep. RL90-9, Thinking Machines Corp., 1990.
41. Y. Zhou, R. Chellappa, A. Vaid, and R. K. Jenkins, "Image restoration using a neural network," IEEE Trans. Acoust., Speech, Signal Processing, vol. 36, pp. 1141-1151, July 1988.