High-performance reconfigurable hardware architecture for restricted boltzmann machines

IEEE Transactions on Neural Networks

Volumn 21, Issue 11, 2010, Pages 1780-1792

  Ly, Daniel Le ^a,b   Chow, Paul ^a  

^a UNIVERSITY OF TORONTO   (Canada)

^b School of Operations Research and Industrial Engineering   (United States)

Author keywords

Boltzmann machines; computer architecture; field programmable gate arrays; neural network hardware; parallel processing

Indexed keywords

ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER; COMPUTER LANGUAGE; COMPUTER PROGRAM; COMPUTER SIMULATION; STANDARD;

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; COMPUTERS; NEURAL NETWORKS (COMPUTER); PROGRAMMING LANGUAGES; SOFTWARE VALIDATION;

EID: 78149361170     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2073481     Document Type: Article

References (21)

1. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., vol. 18, no. 7, pp. 1527-1554, Jul. 2006.
2. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, Jul. 2006.
3. G. W. Taylor, G. E. Hinton, and S. Roweis, "Modeling human motion using binary latent variables," in Advances in Neural Information Processing Systems, vol. 19. Cambridge, MA: MIT Press, 2007, pp. 1345-1352.
4. C. S. Lindsey and T. Lindblad, "Survey of neural network hardware," in Proc. SPIE Appl. Sci. Artificial Neural Netw., 1995, pp. 1194-1205.
5. J. Zhu and P. Sutton, "FPGA implementations of neural networks-A survey of a decade of progress," in Proc. Int. Conf. Field Program. Logic Appl., 2003, pp. 1062-1066.
6. F. M. Dias, A. Antunesa, and A. M. Motab, "Artificial neural networks: A review of commercial hardware," Eng. Appl. Artif. Intell., vol. 17, nos. 1-3, pp. 945-952, Dec. 2004.
7. P. Ferreira, P. Ribeiro, A. Antunes, and F. M. Dias, "A high bit resolution FPGA implementation of a FNN with a new algorithm for the activation function," Neurocomput., vol. 71, nos. 1-3, pp. 71-77, Dec. 2007.
8. A. R. Omondi and J. C. Rajapakse, FPGA Implementations of Neural Networks. New York: Springer-Verlag, 2006.
9. S. K. Kim, L. C. MacAfee, P. L. McMahon, and K. Olukotun, "A highly scalable restricted Boltzmann machine FPGA implementation," in Proc. Int. Conf. Field Program. Logic Appl., 2009, pp. 367-372.
10. D. L. Ly and P. Chow, "A high-performance FPGA architecture for restricted Boltzmann machines," in Proc. ACM Int. Symp. FPGAs, 2009, pp. 73-82.
11. D. L. Ly and P. Chow, "A multi-FPGA architecture for stochastic restricted Boltzmann machines," in Proc. Int. Conf. Field Program. Logic Appl., 2009, pp. 168-173.
12. D. Geman and S. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 6, no. 6, pp. 721-741, Nov. 1984.
13. R. Raina, A. Madhavan, and A. Y. Ng, "Large-scale deep unsupervised learning using graphics processors," in Proc. ACM Int. Conf. Mach. Learn., 2009, pp. 873-880.
14. K. Goto and R. Van De Geijn, "High-performance implementation of the level-3 BLAS," ACM Trans. Math. Softw., vol. 35, no. 1, pp. 1-18, Jul. 2008.
15. M. Saldana, A. Patel, C. Madill, D. Nunes, D. Wang, H. Styles, A. Putnam, R. Wittig, and P. Chow, "MPI as an abstraction for software-hardware interaction for HPRCs," in Proc. 2nd Int. Workshop High-Perform. Reconfigurable Comput. Technol. Appl., Nov. 2008, pp. 1-10.
16. M. Tommiska, "Efficient digital implementation of the sigmoid function for reprogrammable logic," IEE Proc.-Comput. Digital Tech., vol. 150, no. 6, pp. 403-411, Nov. 2003.
17. A. W. Savich, M. Moussa, and S. Areibi, "The impact of arithemetic representation on implementing MLP-BP on FPGAs: A study," IEEE Trans. Neural Netw., vol. 18, no. 1, pp. 240-252, Jan. 2007.
18. B. K. Bharkhada, J. Hauser, and C. Purdy, "Efficient FPGA implementation of a generic function approximator and its application to neural net computation," in Proc. IEEE Int. Symp. Micro-NanoMechatronics Human Sci., Dec. 2003, pp. 843-846.
19. P. L'Ecuyer, "Maximally equidistributed combined Tausworthe generators," Math. Comput., vol. 65, no. 213, pp. 203-213, Jan. 1996.
20. C. Chang, J. Wawrzynek, and R. W. Brodersen, "BEE2: A high-end reconfigurable computing system," IEEE Design & Test Comput., vol. 22, no. 2, pp. 114-125, Mar.-Apr. 2005.
21. D. L. Ly, M. Saldana, and P. Chow, "The challenges of using an embedded MPI for hardware-based processing nodes," in Proc. Int. Conf. Field-Program. Technol., 2009, pp. 120-127.