Parallel Architectures for Learning the RTRN and Elman Dynamic Neural Networks

IEEE Transactions on Parallel and Distributed Systems

Volumn 26, Issue 9, 2015, Pages 2561-2570

  Bilski, Jaroslaw ^a   Smoląg, Jacek ^a  

^a CZESTOCHOWA UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

Parallel architectures; recurrent neural networks; supervised learning

Indexed keywords

ARCHITECTURE; COMPLEX NETWORKS; LEARNING SYSTEMS; MEMORY ARCHITECTURE; NETWORK ARCHITECTURE; NEURAL NETWORKS; RECURRENT NEURAL NETWORKS; SUPERVISED LEARNING;

DIGITAL HARDWARE; DYNAMIC NEURAL NETWORKS; INHERENT PARALLELISM; LEARNING PERFORMANCE; LEARNING PROCESS; PARALLEL REALIZATIONS; THREE-DIMENSIONAL ARCHITECTURE; VECTOR PROCESSORS;

PARALLEL ARCHITECTURES;

EID: 84939220433     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2014.2357019     Document Type: Article

References (28)

1. R. Hecht-Nielsen, Neurocomputing. Reading, MA, USA: Addison-Wesley, 1990.
2. S. Haykin, Neural Networks: A Comprehensive Foundation. New York, NY, USA: Macmilan, 1994.
3. L. Rutkowski, New Soft Computing Techniques for System Modelling, Pattern Classification and Image Processing. Berlin, Germany: Springer-Verlag, 2004.
4. R. Tadeusiewicz, "New Trends in Neurocybernetics," Comput. Methods Materials Sci., vol. 10, no. 1, pp. 1-7, 2010.
5. R. Tadeusiewicz, R. Chaki, and N. Chaki, Exploring Neural Networks with C#, Boca Raton: CRC Press, Taylor & Francis Group, 2014.
6. B. M. Wilamowski, J. Bifnet, and M. O. Kaynak, "VLSI implementation of neural networks," Int. J. Neural Syst., vol. 10, no. 3, pp. 191-197, Jun. 2000.
7. J. Bilski, Parallel Structures for Feedforward and Dynamic Neural Networks (in Polish). Warszawa, Poland: EXIT, 2013.
8. U. Seiffert, "Artificial neural networks on massively parallel computer hardware," Neurocomputing, vol. 57, pp. 135-150, 2004.
9. S. C. Stilkerich, "Graph theoretical representation of ANN architectures on regular two-dimensional grids for VLSI implementations," Neurocomputing, vol. 71, pp. 45-70, 2007.
10. J. Werbos, "Backpropagation through time: What it does and how to do it," Proc. IEEE, vol. 78, no. 10, pp. 1550-1560, Oct. 1990.
11. B. Widrow and M. Lehr, "30 years of adaptive neural networks: perceptron, madaline, and backpropagation," Proc. IEEE, vol. 78, no. 9, pp. 1415-1442, Sep. 1990.
12. J. Žurada, Introduction to artificial neural systems. St. Paul, MN, USA: West, 1992.
13. B. Wilamowski and H. Yu, "Neural network learning without backpropagation," IEEE Trans. Neural Netw., vol. 21, no. 11, pp. 1793-1803, Nov. 2010.
14. J. Kolen and S. Kremer, A Field Guide to Dynamical Recurrent Neural Networks. New York, NY, USA: Wiley-IEEE, 2001.
15. D. Mandic and J. Chambers, Recurrent Neural Networks for Prediction. New York, NY, USA: Wiley, 2001.
16. K. Patan and M. Patan, "Optimal Training Strategies for Locally Recurrent Neural Networks," J. Artificial Intell. Soft Comput. Research, vol. 1, no. 2, pp. 103-114, 2011.
17. S. Osowski, Sieci neuronowe w ujeciu algorytmicznym. Warszawa, Poland: WNT, 1996.
18. L. B. Almeida, "Backpropagation in non-feedforward networks," presented at the NATO ARW Neural Comput., Heidelberg, Germany, 1993.
19. F. J. Pineda, "Generalization of backpropagation to recurrent anf higher order neural networks," in presented at the Neural Information Processing Systems. New York, NY, USA, pp. 602-611.
20. M. T. Hagan, O. De Jezúsi, and R. Schultz, "Training recurrent neural networks for filtering and control," Recurrent Neural Networks - Models, Capacities and Applications. Boca Raton, FL, USA: CRC Press, 2001, pp. 327-356.
21. J. Elman, "Finding structure in time," Cognitive Sci., vol. 14, pp. 179-211, 1990.
22. R. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent," Neural Comput., vol. 1, pp. 270-280, 1989.
23. J. Bilski and J. Smolag, "Parallel realisation of the recurrent RTRN neural network learning," in Proc. 9th Int. Conf. Artif. Intell. Soft Comput., 2008, no. 5097, pp. 11-16.
24. J. Bilski and J. Smoląg, "Parallel realisation of the recurrent Elman neural network learning," in Proc. 10th Int. Conf. Artif. Intell. Softw. Comput., 2010, no. 6114, pp. 19-25.
25. J. Sanders and E. Kandrot, CUDA by Example: An Introduction to General-Purpose GPU Programming. Reading, MA, USA: Addison-Wesley, 2001.
26. P. Ferreira, P. Ribeiro, A. Antunesi, and F. M. Dias, "A high bit resolution FPGA implementation of a FNN with a new algorithm for the activation function," Neurocomputing, vol. 71, pp. 71-77, 2007.
27. Y. Maeda and M. Wakamura, "Simultaneous perturbation learning rule for recurrent neural networks and its FPGA implementation," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1664-1672, Nov. 2005.
28. H. Jang, A. Park, and i K. Jung, "Neural network implementation using CUDA and OpenMP," presented at the Digital Image Comput.: Tech. Appl., Canberra, Australia, 2008.