Study of neural net training methods in parallel and distributed architectures

Future Generation Computer Systems

Volumn 26, Issue 2, 2010, Pages 267-275

  Menéndez de Llano, Rafael ^a   Bosque, José Luis ^a  

^a UNIVERSITY OF CANTABRIA   (Spain)

Author keywords

Neural nets; Parallel algorithms; Parallel architectures; Performance and scalability evaluation

Indexed keywords

ARTIFICIAL NEURAL NET; COLLIDER; DATA PREPROCESSING; DISTRIBUTED ARCHITECTURE; GRID ENVIRONMENTS; HIGH ENERGY; MULTI LAYER PERCEPTRON; NEURAL NET; NEURAL NETS; PARALLEL IMPLEMENTATIONS; PERFORMANCE AND SCALABILITY EVALUATION; SCALABILITY EVALUATION; SHARED MEMORY MULTIPROCESSOR; TRAINING ALGORITHMS; TRAINING METHODS; TRAINING PHASE;

DATA PROCESSING; NEURAL NETWORKS; PACKET NETWORKS; PARALLEL ARCHITECTURES; SCALABILITY;

PARALLEL ALGORITHMS;

EID: 70349777679     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2009.01.001     Document Type: Article

References (31)

1. Craven M.W., and Shavlik J.W. Using neural networks for data mining. Future Generation Computer Systems 13 2-3 (1997) 211-229
2. Higgs P. Broken symmetries and the masses of gauge bosons. Physical Review Letters 13 16 (1964) 508-509
3. Cheng T.P., and Li L.F. Gauge Theory of Elementary Particle Physics (1984), Oxford University Press
4. Artificial Neural Networks in High Energy Physics, Consulted in 2007 http://neuralnets.web.cern.ch/NeuralNets/nnwInHep.html
5. Aristoklis D., Anastasiadisa G.D., Magoulasa N., and Vrahatisb. New globally convergent training scheme based on the resilient propagation algorithm. Neurocomputing 64 (2005) 253-270
6. I. Foster, K. Kesselman, S. Tuecke, The Anatomy of the Grid, Editorial: Globus Alliance. 2001. Ref. Consulted September 2004 http://www.globus.org/research/papers/anatomy.pdf
7. C. Foster, J. Kesselman, S. Nick, Tuecke, The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration, Globus Project. 2002, Consulted 2008 http://www.globus.org/alliance/publications/papers/ogsa.pdf
8. Bubak M., Malawski M., and Zaja̧c K. The crossgrid architecture: Applications, tools, and grid services. Lecture Notes in Computer Science 2970/2004 (2004), Springer, Berlin, Heidelberg 309-316
9. Soulié F.F. Neural networks and computing. Future Generation Computer Systems 7 1 (1991) 69-77
10. Fletcher R. Practical Methods of Optimization (1987), Wiley, New York
11. Baum E.B., and Haussler D. What size net gives valid generalization?. Neural Computation 1 1 (1989) 151-160
12. Gene M. Amdahl, Validity of the single processor approach to achieving large scale computing capabilities, in: Proc. of the AFIPS Spring Joint Computer Conference, vol. 31, 1967, pp. 483-485
13. Blackford L.S., Choi J., et al. ScaLAPACK. Users' Guide (1997), Society for Industrial and Applied Mathematics, Philadelphia, PA
14. J. Dongarra, R.C. Whaley, A user's guide to the BLACS v1.1, Computer Science Dept., Technical Report CS-95-281, University of Tennessee, Knoxville, TN, 1995
15. Cappello F., Richard O., and Etiemble D. Understanding performance of SMP clusters running MPI programs. Future Generation Computer Systems 17 6 (2001) 711-720
16. Karonis N., Toonen B., and Foster I. MPICH-G2: A grid-enabled implementation of the message passing interface. Journal of Parallel and Distributed Computing (JPDC) 63 5 (2003) 551-563
17. Anderson E., Bai Z., et al. LAPACK Users' Guide. second ed. (1995), Society for Industrial and Applied Mathematics, Philadelphia, PA
18. Foster I., and Kesselman C. The Globus project: A status report. Future Generation Computer Systems 15 5-6 (1999) 607-621
19. Sjöstrand T., Edén P., et al. High-energy-physics event generation with PYTHIA 6.1. Computer Physics Communications 13 2 (2001) 238-259
20. J. Guy, et al. DELSIM reference manual, Last update Jan 1998, Consulted 2008
21. Y. Zhang, Block size selection of parallel LU factorization, in: Proc. Fourth International Conference/Exhibition on High Performance Computing in Asia-Pacific Region, vol. 1, 2000, pp. 247-249
22. S. Blackford, Choice of block size, July 1996. Consulted in December 2008, http://www.netlib.org/utk/papers/sc96-scalapack/NODE22.HTM
23. Tan Ker Vin, Pang Zhe Seng, M. Ng Pek Kuan, F. Haron, A framework for grid-based neural networks, in: Proceedings of the First International Conference on Distributed Frameworks for Multimedia Applications, Feb. 2005, pp. 246-253
24. Leander Krammer, Erich Schikuta, Helmut Wanek, A grid-based neural network execution service source, in: Proceedings of the 24th IASTED International Conference on Parallel and Distributed Computing and Networks, 2006, pp. 35-40
25. E. Schikuta, T. Weishaupl, N2Grid: Neural networks in the grid. Neural Networks, in: Proceedings of IEEE International Joint Conference on Neural Networks 2, July 2004, pp. 1409-1414
26. J. Valdes, Graeme Bonham-Carter, Time dependent neural network models for detecting changes of state in earth and planetary processes, in: Proceedings of International Joint Conference on Neural Networks, Summer 2005, pp. 1710-1715
27. J.C. Werner, Grid computing in High Energy Physics using LCG: The BaBar experience, UK eScience All Hands Meeting, 2006
28. C.-C. Peng, G.D. Magoulas, Effective modification of the bfgs method for training recurrent neural networks, in: Proceedings of the Conference in Numerical Analysis, September 2007, pp. 113-117
29. Peng C., and Magoulas GD. Adaptive self-scaling non-monotone bfgs training algorithm for recurrent neural networks. Artificial Neural Networks. Lecture Notes in Computer Science 4668/2007 (2007) 259-268
30. S. Vijanth, McLoone Asirvadam, George W. Irwin, Memory efficient BFGS neural-network learning algorithms using MLP-NETwork: A survey, in: Proceedings of the IEEE International Conference on Control Applications, September 2004, pp. 586-591
31. Production Resource Broker, Consulted in 2007 http://www.lip.pt/computing/projects/crossgrid/crossgrid-services/resource-broker.htm