The impact of arithmetic representation on implementing MLP-BP on FPGAs: A study

IEEE Transactions on Neural Networks

Volumn 18, Issue 1, 2007, Pages 240-252

  Savich, Antony W ^a   Moussa, Medhat ^a   Areibi, Shawki ^a  

^a UNIVERSITY OF GUELPH   (Canada)

Author keywords

Artificial neural networks (AANs); Fixed point (FXP) arithmetic; Floating point (FLP) arithmetic

Indexed keywords

ALGORITHMS; COMPUTER HARDWARE DESCRIPTION LANGUAGES; DIGITAL ARITHMETIC; FIELD PROGRAMMABLE GATE ARRAYS;

ARITHMETIC REPRESENTATION; DATA REPRESENTATION; ERROR BACKPROPAGATION ALGORITHM; FIXED-POINT (FXP) ARITHMETIC;

MULTILAYER NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER AIDED DESIGN; COMPUTER SIMULATION; EQUIPMENT; EQUIPMENT DESIGN; INSTRUMENTATION; MATHEMATICAL COMPUTING; SIGNAL PROCESSING; THEORETICAL MODEL;

COMPUTER SIMULATION; COMPUTER-AIDED DESIGN; EQUIPMENT DESIGN; EQUIPMENT FAILURE ANALYSIS; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); NUMERICAL ANALYSIS, COMPUTER-ASSISTED; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 33846101011     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.883002     Document Type: Article

References (24)

1. D. Rumelhart, J. McClelland, and P. R. Group, Parallel Distrubuted Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press, 1986, vol. 1, Foundations.
2. G. N. Karystinos and D. A. Pados, "On overfitting, generalization, and randomly expanded training sets," IEEE Trans. Neural Netw., vol. 11, no. 5, pp. 1050-1057, Sep. 2000.
3. H. McCartor, "A highly parallel digital architecture for neural network emulation," VLSI Artif. Intell. Neural Netw., pp. 357-366, 1991.
4. U. Ramacher, W. Raab, and J. A. et al., "Multiprocessor and memory architecture of the neurocomputers SYNAPSE-1," in Proc. 3rd Int. Conf. Microelectron. Neural Netw., 1993, pp. 227-231.
5. S. Tam, B. Gupta, H. Castro, and M. Holler, "Learning on an analog VLSI neural network chip," in Proc. IEEE Int. Conf. Syst., Man Cybern., Nov. 1990, pp. 701-703.
6. J. G. Eldredge, "FPGA density enhancement of a neural network through run-time reconfiguration," M.S. thesis, Dept. Elect. Comput. Eng., Brigham Young Univ., Provo, UT, 1994.
7. D. Anguita, A. Boni, and S. Ridella, "A digital architecture for support vector machines: Theory, algorithm, and FPGA implementation," IEEE Trans. Neural Netw., vol. 14, no. 5, pp. 993-1009, Sep. 2003.
8. H. Ng and K. Lam, "Analog and digital FPGA implementation of brin for optimization problems," IEEE Trans. Neural Netw., vol. 14, no. 5, pp. 1413-1425, Sep. 2003.
9. Y. Maeda and M. Wakamura, "Simultaneous perturbation learning rule for recurrent neural networks and its fpga implementation," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1664-1672, Nov. 2005.
10. J. Zhu and P. Sutton, "FPGA implementations of neural networks - A survey of a decade of progress," in Proc. Conf. Field Programm. Logic, Lisbon, Portugal, 2003, pp. 1062-1066.
11. IEEE Standard for Binary Floating Point Arithmetic, Std 754-1985 edition, New York ANSI/IEEE, 1985.
12. G. Even and P. Wolfgang, "On the design of IEEE compliant floating point units," IEEE Trans. Comput., vol. 49, no. 5, pp. 398-413, May 2000.
13. D. Goldberg, "What every computer scientist should know about floating-point arithmetic," ACM Comput. Surv., vol. 23, no. 1, pp. 5-48, Mar. 1991.
14. J. Holt and T. Baker, "Backpropagation simulations using limited precision calculations," in Proc. Int. Joint Conf. Neural Netw. (IJCNN-91), Seattle, WA, Jul. 1991, vol. 2, pp. 121-126.
15. G. Govindu, L. Zhuo, S. Choi, and V. Prasanna, "Analysis of high-performance floating-point arithmetic on FPGAs," in Proc. IEEE Int. Parallel and Distributed Process. Symp., Santa Fe, NM, Apr. 2004, pp. 149-156.
16. C. T. Ewe, "Dual fixed-point: An efficient alternative to floating-point computation for DSP applications," in Proc. Field Programm. Logic Appl., Tampere, Finland, Aug. 2005, pp. 715-716.
17. C. T. Ewe, P. Y. K. Cheung, and G. A. Constantinides, "Error modelling of dual fixed-point arithmetic and its application in field programmable logic," in Proc. Field Programm. Logic Appl., Tampere, Finland, Aug. 2005, pp. 124-129.
18. S. Li, M. Moussa, and S. Areibi, "Arithmetic formats for implementing artificial neural networks on FPGAs," Can. J. Elect. Comput. Eng., vol. 31, no. 1, pp. 31-40, 2006.
19. W. Ligon, III, S. McMillan, G. Monn, K. Schoonover, F. Stivers, and K. Underwood, "A re-evaluation of the practicality of floating point operations on FPGAs," in Proc. IEEE Symp. FPGAs Custom Comput. Mach., K. L. Pocek and J. Arnold, Eds., Los Alamitos, CA, 1998, pp. 206-215 Online. Available: citeseer.nj.nec.com/95888.html
20. Xilinx, Virtex-II Platform FPGAs: Detailed Description Mar. 2004 Online. Available: www.xilinx.com, DS031-2 v3.2 ed.
21. "Intel Architecture Software Developer's Manual, Volume 2: Instruction Set Reference (243191)" Intel Corp., 1999 Online. Available: www.intel.com
22. P. Belanovic, "Library of parameterized hardware modules for floating-point arithmetic with an example application," M.A.Sc. thesis, Elect. Comp. Eng. Dept., Northeastern Univ., Boston, MA, 2002.
23. Sun Microsystems, What Every Computer Scientist Should Know About Floating-Point Arithmetic Mountain View, CA, Tech. Rep., 1994.
24. S. Haykin, Neural Networks: A Comprehensive Foundation, Edition II. Englewood Cliffs, NJ: Prentice-Hall, 1999.