An evaluation of multiple feed-forward networks on GPUs

International Journal of Neural Systems

Volumn 21, Issue 1, 2011, Pages 31-47

  Lopes, Noel ^a,b   Ribeiro, Bernardete ^a  

^a UNIVERSITY OF COIMBRA   (Portugal)

^b POLYTECHNIC INSTITUTE OF GUARDA   (Portugal)

Author keywords

autonomous training system; GPU computing; Neural networks

Indexed keywords

DATA PARALLEL; FEED-FORWARD NETWORK; GPU COMPUTING; GRAPHICS HARDWARE; GRAPHICS PROCESSING UNIT; HIGH QUALITY; LEARNING PROCESS; LOW COSTS; MACHINE-LEARNING; MEMORY BANDWIDTHS; PARALLEL IMPLEMENTATIONS; TRAINING SYSTEMS;

BACKPROPAGATION ALGORITHMS; COMPUTER GRAPHICS; COMPUTER GRAPHICS EQUIPMENT; NEURAL NETWORKS; PROGRAM PROCESSORS;

LEARNING ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER GRAPHICS; EVALUATION; INSTRUMENTATION;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER GRAPHICS; NEURAL NETWORKS (COMPUTER);

EID: 78751474732     PISSN: 01290657     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129065711002638     Document Type: Conference Paper

References (80)

1. A. Khashman, A neural network model for credit risk evaluation, International Journal of Neural Systems 19(4) (2009) 285-294.
2. M. Christodoulou and C. Kontogeorgou, Collision avoidance in commercial aircraft free flight via neural networks and non-linear programming, International Journal of Neural Systems 18(5) (2008) 371-387.
3. H. Adeli and X. Jiang, Dynamic fuzzy wavelet neural network model for structural system identification, Journal of Structural Engineering, ASCE 132(1) (2006) 102-111.
4. X. Jiang and H. Adeli, Dynamic wavelet neural network for nonlinear identification of highrise buildings, Computer-Aided Civil and Infrastructure Engineering 20(5) (2005) 316-330. (Pubitemid 40906713)
5. D. Boto-Giralda, F. Daz-Pernas, D. Gonzlez-Ortega, J. Dez-Higuera, M. Antn-Rodrguez and M. Martnez- Zarzuela, Wavelet-based denoising for traffic volume time series forecasting with self-organizing neural networks, Computer-Aided Civil and Infrastructure Engineering 25(7) (2010) 530-545.
6. X. Jiang and H. Adeli, Dynamic wavelet neural network model for traffic flow forecasting, Journal of Transportation Engineering, ASCE 131(10) (2005) 771-779. (Pubitemid 41419813)
7. S. Hooshdar and H. Adeli, Toward intelligent variable message signs in freeway work zones: A neural network approach, Journal of Transportation Engineering, ASCE 131(1) (2004) 83-93.
8. A. Karim and H. Adeli, Radial basis function neural network for work zone capacity and queue estimation, Journal of Transportation Engineering, ASCE 129(5) (2003) 494-503.
9. S. Ghosh-Dastidar and H. Adeli, Wavelet-clusteringneural network model for freeway incident detection, Computer-Aided Civil and Infrastructure Engineering 18(5) (2003) 325-338.
10. H. Adeli and X. Jiang, Neuro-fuzzy logic model for freeway work zone capacity estimation, Journal of Transportation Engineering, ASCE 129(5) (2003) 484-493. (Pubitemid 37164325)
11. A. Dharia and H. Adeli, Neural network model for rapid forecasting of freeway link travel time, Engineering Applications of Artificial Intelligence 16(7-8) (2003) 607-613.
12. H. Adeli and A. Samant, An adaptive conjugate gradient neural network - wavelet model for traffic incident detection, Computer-Aided Civil and Infrastructure Engineering 15(4) (2000) 251-260.
13. H. Adeli and A. Karim, Fuzzy-wavelet RBFNN model for freeway incident detection, Journal of Transportation Engineering, ASCE 126(6) (2000) 464-471.
14. A. Khashman and B. Sekeroglu, Document image binarisation using a supervised neural network, International Journal of Neural Systems 18(5) (2008) 405-418.
15. F. Rasheed, M. Alshalalfa and R. Alhajj, Adapting machine learning technique for periodicity detection in biological sequences, International Journal of Neural Systems 19(1) (2009) 11-24.
16. A. Khashman, Blood cell identification using a simple neural network, International Journal of Neural Systems 18(5) (2008) 453-458.
17. A. Senouci and H. Adeli, Resource scheduling using neural dynamics model of Adeli and Park, Journal of Construction Engineering and Management, ASCE 127(1) (2001) 28-34. (Pubitemid 32158956)
18. G. Sirca and H. Adeli, Neural network model for uplift load capacity of metal roof panels, Journal of Structural Engineering, ASCE 127(11) (2001) 1276-1285.
19. D. Theodoridis, Y. Boutalis and M. Christodoulou, Indirect adaptive control of unknown multi variable nonlinear systems with parametric and dynamic uncertainties using a new neuro-fuzzy system description, International Journal of Neural Systems 20(2) (2010) 129-148.
20. F. Ahmadkhanlou and H. Adeli, Optimum cost design of reinforced concrete slabs using neural dynamics model, Engineering Applications of Artificial Intelligence 18(1) (2005) 65-72.
21. G. Sirca and H. Adeli, Cost optimization of prestressed concrete bridges, Journal of Structural Engineering, ASCE 131(3) (2005) 380-388.
22. H. Adeli and H. Kim, Cost optimization of composite floors using the neural dynamics model, Communications in Numerical Methods in Engineering 17 (2001) 771-787. (Pubitemid 33109612)
23. H. Adeli and A. Karim, Neural network model for optimization of cold-formed steel beams, Journal of Structural Engineering, ASCE 123(11) (1997) 1535-1543.
24. H. Adeli and H. Park, Fully automated design of superhighrise building structure by a hybrid AI model on a massively parallel machine, AI Magazine 17(3) (1996) 87-93.
25. H. Adeli and H. Park, Counter propagation neural network in structural engineering, Journal of Structural Engineering, ASCE 121(8) (1995) 1205-1212.
26. H. Adeli and H. Park, A neural dynamics model for structural optimization - theory, Computers and Structures 57(3) (1995) 383-390.
27. H. Adeli and H. Park, Optimization of space structures by neural dynamics, Neural Networks 8(5) (1995) 769-781.
28. S. Hung and H. Adeli, Object-oriented back propagation and its application to structural design, Neurocomputing 6(1) (1994) 45-55.
29. A. Karim and H. Adeli, CONSCOM: An OO construction scheduling and change management system, Journal of Construction Management and Engineering, ASCE 125(5) (1999) 368-376.
30. H. Adeli and M. Wu, Regularization neural network for construction cost estimation, Journal of Construction Management and Engineering, ASCE 124(1) (1998) 18-24.
31. H. Adeli and A. Karim, Scheduling/cost optimization and neural dynamics model for construction, Journal of Construction Management and Engineering, ASCE 123(4) (1997) 450-458. (Pubitemid 127590588)
32. W. Graf, S. Freitag, M. Kaliske and J. Sickert, Recurrent neural networks for uncertain timedependent structural behavior, Computer-Aided Civil and Infrastructure Engineering 25(5) (2010) 322-333.
33. X. Jiang and H. Adeli, Dynamic fuzzy wavelet neuroemulator for nonlinear control of irregular highrise building structures, International Journal for Numerical Methods in Engineering 74(7) (2008) 1045-1066.
34. X. Jiang and H. Adeli, Neuro-genetic algorithm for nonlinear active control of highrise buildings, International Journal for Numerical Methods in Engineering 75(8) (2008) 770-786.
35. A. Panakkat and H. Adeli, Recurrent neural network for approximate earthquake time and location prediction using multiple seismicity indicators, Computer-Aided Civil and Infrastructure Engineering 24(4) (2009) 280-292.
36. H. Adeli and A. Panakkat, A probabilistic neural network for earthquake magnitude prediction, Neural Networks 22 (2009) 1018-1024.
37. A. Panakkat and H. Adeli, Neural network models for earthquake magnitude prediction using multiple seismicity indicators, International Journal of Neural Systems 17(1) (2007) 13-33.
38. X. Jiang and H. Adeli, Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings, International Journal for Numerical Methods in Engineering 71(5) (2007) 606-629.
39. A. Haidar, A. Mohamed, M. Al-Dabbagh, A. Hussain and M. Masoum, An intelligent load shedding scheme using neural networks & neuro-fuzzy, International Journal of Neural Systems 19(6) (2009) 473-479.
40. U. Reuter and B. Moeller, Artificial neural networks for forecasting of fuzzy time series, Computer-Aided Civil and Infrastructure Engineering 25(5) (2010) 363-374.
41. D. Wu, K. Warwick, Z. Ma, M. Gasson, J. Burgess, S. Pan and T. Aziz, Prediction of Parkinson's disease tremor onset using radial basis function neural network based on particle swarm optimization, International Journal of Neural Systems 20(2) (2010) 109-116.
42. S. Ghosh-Dastidar and H. Adeli, Spiking neural networks, International Journal of Neural Systems 19(4) (2009) 295-308.
43. S. Ghosh-Dastidar and H. Adeli, A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection, Neural Networks 22(10) (2009) 1419-1431.
44. H. Adeli, S. Ghosh-Dastidar and N. Dadmehr, A spatio-temporal wavelet-chaos methodology for EEG-based diagnosis of Alzheimer's disease, Neuroscience Letters 444(2) (2008) 190-194.
45. S. Ghosh-Dastidar and H. Adeli, Improved spiking neural networks for EEG classification and epilepsy and seizure detection, Integrated Computer-Aided Engineering 14(3) (2007) 187-212.
46. S. Ghosh-Dastidar, H. Adeli and N. Dadmehr, Mixed-band wavelet-chaos-neural network methodology for epilepsy and epileptic seizure detection, IEEE Transactions on Biomedical Engineering 54(9) (2007) 1545-1551.
47. H. Adeli, S. Ghosh-Dastidar and N. Dadmehr, Alzheimers disease and models of computation: Imaging, classification, and neural models, Journal of Alzheimer's Disease 7(3) (2005) 187-199.
48. M. Ahmadlou and H. Adeli, Enhanced probabilistic neural network with local decision circles: A robust classifier, Integrated Computer-Aided Engineering 17(3) (2010) 197-210.
49. P. Anand, B. Prasad and C. Venkateswarlu, Modeling and optimization of a pharmaceutical formulation system using radial basis function network, International Journal of Neural Systems 19(2) (2009) 127-136.
50. C. Zanchettin and T. Ludermir, Hybrid neural systems for pattern recognition in artificial noses, International Journal of Neural Systems 15(1-2) (2005) 137-149. (Pubitemid 44099518)
51. A. Brandstetter and A. Artusi, Radial basis function networks GPU based implementation, IEEE Trans. on Neural Networks 19(12) (2008) 2150-2154.
52. D. Steinkrau, P. Simard and I. Buck, Using GPUs for machine learning algorithms, Proc. 8th International Conf. on Doc. Analysis and Recognition (2005) 1115-1119.
53. B. Catanzaro, N. Sundaram and K. Keutzer, Fast support vector machine training and classification on graphics processors, Proc. 25th International Conference on Machine Learning (2008) 104-111.
54. L. Zhongwen, L. Hongzhi, Y. Zhengping and W. Xincai, Self-organizing maps computing on graphic process unit, Proc. 13th European Symposium on Artificial Neural Networks (2005) 557-562.
55. D. Schaa and D. Kaeli, Exploring the multiple-GPU design space, Proc. 23rd IEEE International Parallel and Distributed Processing Symposium (2009) 1-12.
56. J. D. Owens, D. Luebke, N. Govindaraju,M. Harris, J. Krger, A. E. Lefohn and T. J. Purcell, A survey of general-purpose computation on graphics hardware, Computer Graphics Forum 26(1) (2007) 80-113.
57. K. Oh and K. Jung, GPU implementation of neural networks, Pattern Recognit. 37(6) (2004) 1311-1314.
58. Q. Junfei and H. Honggui, A repair algorithm for radial basis function neural network with application to chemical oxygen demand modeling, International Journal of Neural Systems 20(1) (2010) 63-74.
59. P. Patrinos, A. Alexandridis, K. Ninos and H. Sarimveis, Variable selection in nonlinear modeling based on RBF networks and evolutionary computation International Journal of Neural Systems 20(5) (2010).
60. R. Savitha, S. Suresh and N. Sundararajan, A fully complex-valued radial basis function network and its learning algorithm, International Journal of Neural Systems 19(4) (2009) 253-267.
61. H. Jang, A. Park and K. Jung, Neural network implementation using CUDA and OpenMP, Proc. 2008 Digital Image Computing: Techniques and Applications (2008) 155-161.
62. L. B. Almeida, C1.2 Multilayer perceptrons, in Handbo ok of Neural Computation. IOP Publishing Ltd and Oxford University Press (1997) C1.2:1-C1.2:30.
63. S. Che, M. Boyer, J. Meng, D. Tarjan, J. W. Sheaffer and K. Skadron, A performance study of generalpurpose applications on graphics processors using CUDA, Journal of Parallel and Distributed Computing 68(10) (2008) 1370-1380.
64. S. Ryoo, C. Rodrigues, S. Baghsorkhi, S. Stone, D. Kirk and W. Hwu, Optimization principles and application performance evaluation of a multithreaded GPU using CUDA, Proc. 13th ACM Symp. on Principles and practice of parallel programming (2008) 73-82.
65. NVIDIA CUDA programming guide ver. 2.3.1 (2009).
66. N. Lopes and B. Ribeiro, An efficient gradient-based learning algorithm applied to neural networks with selective actuation neurons, Neural, Parallel & Scientific Computations 11(3) (2003) 253-272.
67. N. Lopes and B. Ribeiro, Hybrid learning in a multi-neural network architecture, Proc. International Joint Conference on Neural Networks 4 (2001) 2788-2793.
68. N. Lopes and B. Ribeiro, GPU implementation of the multiple back-propagation algorithm, in Corchado, E., Yin, H. (eds.): IDEAL. Lecture Notes in Computer Science 5788, Springer (2009) 449-456.
69. S. Hung and H. Adeli, A parallel genetic/neural network learning algorithm for MIMD shared memory machines, IEEE Transactions on Neural Networks 5(6) (1994) 900-909.
70. H. Adeli and S. Hung, A concurrent adaptive conjugate gradient learning algorithm on MIMD machines, Journal of Supercomputer Applications, MIT Press 7(2) (1993) 155-166.
71. H. Adeli and S. Hung, Parallel backpropagation learning algorithms on Cray Y-MP8/864 supercomputer, Neurocomputing 5(6) (1993) 287-302.
72. N. Lopes, B. Ribeiro and R. Quintas, GPUMLib: A new library to combine machine learning algorithms with graphics processing units, 10th International Conference on Hybrid Intelligent Systems (2010).
73. T. Jorgensen, B. Haynes and C. Norlund, Pruning artificial neural networks using neural complexity measures, International Journal of Neural Systems 18(5) (2008) 389-403.
74. M. Nicoletti, J. Bertini Jr., D. Elizondo, L. Franco and J. Jerez, Constructive neural network algorithms for feedforward architectures suitable for classification tasks, Constructive Neural Networks: Studies in Computational Intelligence 258, Springer (2009) 1-23.
75. A. Zapranis and A. Refenes, Principles of neural model identification, selection and adequacy: With applications in financial econometrics, Springer (2001).
76. S. Fahlman and C. Lebiere, The cascade-correlation learning architecture, Advances in Neural Inf. Proc. Systems 2, Morgan Kaufmann (1990) 524-532.
77. A. Asuncion and D. Newman, UCI Machine Learning Repository [http://www.ics.uci.edu/mlearn/ MLRepository.html], Irvine, CA: Univ. of California, School of Inf. and Computer Science (2007).
78. A. Marques, Feature extraction and PVC detection using neural networks and support vector machines, Masters thesis, University of Coimbra (2007).
79. J. A. Blackard and D. J. Dean, Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables, Computer and Electronics in Agriculture 24 (1999) 131-151.
80. N. Lopes and B. Ribeiro, Fast pattern classification of ventricular arrhythmias using graphics processing units, Proc. 14th Iberoamerican Congress on Pattern Recognition, LNCS 5856, Springer (2009) 603-610.