A new classification approach for neural networks hardware: From standards chips to embedded systems on chip

Artificial Intelligence Review

Volumn 41, Issue 4, 2014, Pages 491-534

  Izeboudjen, N ^a,b   Larbes, C ^b   Farah, A ^b  

^a CENTRE DE DÉVELOPPEMENT DES TECHNOLOGIES AVANCÉES   (Algeria)

^b ECOLE NATIONALE POLYTECHNIQUE   (Algeria)

Author keywords

ANN hardware; ASICs; Classification; Embedded systems on chip; FPGA; Neurocomputers; VLSI

Indexed keywords

CLASSIFICATION (OF INFORMATION); DESIGN; EMBEDDED SYSTEMS; FIELD PROGRAMMABLE GATE ARRAYS (FPGA); MICROELECTRONICS; NEURAL NETWORKS;

ASICS; CLASSIFICATION APPROACH; CLASSIFICATION CRITERION; DESIGN TOOLS AND TECHNIQUES; MICROELECTRONIC TECHNOLOGIES; NEUROCOMPUTERS; RESEARCH INSTITUTIONS; VLSI;

HARDWARE;

EID: 84896995829     PISSN: 02692821     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10462-012-9321-7     Document Type: Article

References (136)

1. Aleksander I, Thomas W, Bowden P (1984) WISARD, a radical new step forward in image recognition. Sensor Rev 4(3): 120-124
2. Alspector J (1991). VLSI architecture for neural networks: concepts, applications, and implementations, vol 1. Prentice-Hall, Englewood Cliffs, pp 180-213
3. AN221E04 Datasheet (2010) Dynamically reconfigurable FPAA with Enhanced I/O, DS030100-U006b. Available in the internet at: http://www.anadigm.com (accessed November 2011)
4. Anderson D, McNeill G (1992) Artificial neural networks technology. DACS State-of-the-Art Report, Contract Number F30602-89-C-0082
5. Ayala JL et al. (2002) Design of a pipelined hardware architecture for real-time neural network computations. In: Proceedings of the 45th midwest symposium on circuits and systems MWSCAS'02 Tulsa, Okla, USA, vol 1, pp 419-422
6. Aybay I, Cetinkaya S, Halici U (1996) Classification of neural network hardware. Neural Network World. IDG Co 6: 11-29
7. Bavan P, Lee MS, Trealeven P (1988) A simple VLSI architecture for Neurocomputing. In: Proceedings of the International Neural Network Society. First annual Meeting, Boston, Massachusetts, p 398
8. Beetem J, Denneau M, Weingarten D (1987) the GF11 parallel computer, experimental parallel computing architectures. J J Dongarra Elsevier science Publisher, North-Holland
9. Beiu V (1997) Digital integrated circuit implementations. IOP Publishing Ltd and Oxford University Press Handbook of Neural Computation release 97/1, pp 1-34
10. Beuchat JL, Haeni JO, Sanchez E (1998) Hardware reconfigurable neural networks. IPPS/SPDP'98 Workshop no 10 Orlando- Florida, vol 1388, pp 91-98
11. Blair GM (1992) PLA design for single-clock CMOS. IEEE J Solid State Circuit 27(8): 1211-1213
12. Bor JC, Wu CY (1996) Analog electronic cochlea design using multilexing switched capacitor circuits. IEEE Trans Neural Netw 7(1): 155-166
13. Bower MJ, Beeman D (1998) The book of GENESIS: exploring realistic neural models with the GEneral NEural SImulation System. Springer, New York, ISBN 978-0387949383
14. Castillo FJ, Cabestany J, Moreno JM (1992) The dynamic ring architecture. In: Proceedings of the ICANN-92-Brighton UK. Elsevier, Amsterdam, pp 1439-1442
15. Castro HA, Tam SM, Holler MA (1993) Implementation and performance of an analog nonvolatile neural network. Analog Integr Circuits Signal Process 4(2): 97-113
16. Christy P (1990) Software to support massively parallel computing on the MasPar MP-1. In: Proceedings of COMPCON, pp 29-33
17. Cichocki A, Unbehauen R (1994) Neural networks for optimization and signal processing. Wiley, London
18. Cox CE, Blanz WE (1992) GANGLION: a fast field programmable gate array implementation of a connectionist classifier. IEEE J Solid State Circuits 3: 288-299
19. Cox CE, Mathia K, Saeks R (1995) Learning flight control and LoFLYTE. WESCON'96 Microelectronics Communication Technology Producing Quality Products Mobile and Portable Power Emerging Technologies. doi: 10.1109/WESCON.1995.485490
20. Culurciello E, Cummings RE, Boahen KA (2003) A biomorphic digital image sensor. IEEE J Solid-State Circuits 38(2): 281-294
21. David R, Williams E, De Tremiolles G, Tannhof P (1999) Description and practical uses of IBM ZISC036. Proc SPIE 3728: 198-211
22. De Garis H, Korkin M (2002) The Cam-Brain machine: an FPGA based hardware tool which evolves a 1000 neuron net module in seconds and updates a 75 million neuron artificial brain for real time robot. Neurocomput J 42(1-4): 35-68
23. De Groot AJ, Parker SR (1989) Systolic implementation of neural networks. SPIE, The International Society for Optical Engineering. In: Bromley K (eds) High speed computing II 1058, pp 182-190
24. Dehon A (2000) The density advantage of configurable computing. IEEE Comput 33(5): 41-49
25. Demuth H, Beale M (1992) Neural network toolbox for use with MATLAB. User Guide Version-4
26. Denby B (1993) The use of neural networks in high energy physics. Neural Comput 5(4): 505-549
27. DeYong MR, Findley RL, Fields C (1992) The design, fabrication, and test of a new VLSI hybrid analog-digital neural processing element. IEEE Trans Neural Netw 3(3): 363-374
28. Diepenhorst M et al (1999) Automatic generation of VHDL code for neural applications. In: International Joint Conference on Neural Networks (IJCNN), vol 4, pp 2302-2305
29. Draghici S (2000) Neural NETWORKS IN analog hardware-design and implementation issues. Int J Neural Syst 1: 19-42
30. Driancourt X (1994) Personnal communication. Neuristique Inc. France
31. Duranton M, Sirat JA (1990) Learning on VLSI: a general-purpose digital neurochip. Philips J Res 45(1): 1-17
32. Dvorak JC (1991) Best of 1990: BrainMaker Professional, Version 1.5, PC Magazine, January 15. Available in the internet at: http://www.calsci.com/ referenc.html (accessed November 2011)
33. Eguci H et al (1991) Neural network LSI chip with on-chip learning. In: International Joint Conference on Neural Networks, vol 1, pp 453-456
34. Eldredge JG, Hutchings BL (1994) RRANN: the run time reconfiguration artificial neural network. IEEE Custom Integrated Circuits Conference, San Diego, pp 77-80
35. Eldredge JG (1994) FPGA density enhancement of a neural network through run-time reconfiguration. Master thesis Department of Electrical and Computer engineering, Brigham Young University
36. Elmasry MI (1994) VLSI artificial neural networks engineering. Kluwer, Dordrecht
37. Ferrucci AT (1994) ACME: a field programmable gate array implementation of a self adapting and scalable connectionist network. Master Thesis University of California SANTA CRUZ
38. Fisher WA et al (1991) A programmable analog neural network processor. IEEE Trans Neural Netw 2: 222-229
39. Flynn MJ (1972) Some computer organization and their effectiveness. IEEE Trans Comput 21: 948-960
40. Garth SCJ (1987) A chipset for high speed simulation of neural network systems. In: Proceedings of the IEEE first International Conference on Neural Networks III, pp 443-452
41. Gascuel JD et al (1991) A digital CMOS fully connected neural network within circuit learning capability and automatic identification of spurious attractors. IEEE Conference on Euro ASIC, pp 247-250
42. Gigliotti P (2004) Implementing barrel shifters using multipliers. Application Note: XAPP195 (v1.1). Available in the internet at: http://www.xilinx.com (accessed on November 2011)
43. Glesner M, Huch M, Pochmuller V, Palm G (1989) Hardware implementations for neural networks. Workshop on Parallel Architectures on Silicon, pp 65-7
44. Glesner M, Pochmuller W (1994) Neurocomputers: an overview of neural networks in VLSI. Chapman & Hall, London, ISBN 0-412-56390-8
45. Goser K, Hilleringmann U, Rueckert U, Schumacher K (1989) VLSI technologies for artificial neural networks IEEE MICRO, pp 28-44
46. Gorse D, Taylor GJ, Klarkson TG (1994) Extended functionality for pRAMs. In: International Conference on Artificial Neural Networks, ICANN'94, pp 705-708
47. Hammerstrom D (1990) A VLSI architecture for high-performance, low cost, on-chip learning. Int Joint Conf Neural Netw 2: 537-543
48. Hanson WA, Cruz CA, Tam JY (1987) CONE-computational network environment. In: Proceedings of IEEE First International Conference on Neural Networks III, pp 531-538
49. Hasler P, Diorio C, Minch BA, Mead C (1995) Single transistor learning synapses. Advances in neural information processing systems 2. MIT Press, Cambridge, pp 817-824
50. Hecht-Nielsen R (1990) Neurocomputing. Addison-Wesley Publishing Company, Reading
51. Heemskerk JNH (1995) Neurocomputers for brain-style processing: design, implementation and application. PhD Thesis, Leiden University the Netherlands
52. Hillis WD, Steel GLJ (1986) Data parallel algorithms. Commun ACM 29(12): 1170-1183
53. Holler M et al (1989) An electrically trainable artificial neural network (ETANN) with 1024 floating gate synapse. In: Proceedings of IACNN, pp 191-196
54. Holt JL, Baker TE (1991) Backpropagation simulations using limited precision calculations. Int Joint Conf Neural Netw 2: 121-126
55. Hunt DJ (1989) AMT DAP: a processor array in a workstation environment. Comput Syst Sci Eng 4(2): 107-114
56. Ienne P (1993) GENES IV: a bit serial processing element for a multi model neural network accelerator. In: Proceedings of the International Conference on Application Specific Array Processors IEEE Computer Society, pp 345-356
57. Ienne P (1995) Digital systems for neural networks. Digital signal processing technology CR57 of critical reviews series. SPIE Optical Engineering, pp 314-345
58. Jabri MA, Flower B (1992) Weight perturbation: an optimal architecture learning technique for Analog VLSI feed forward and recurrent multilayer networks. IEEE Trans Neural Netw 3(1): 154-157
59. Jackson D, Hammerstrom D (1991) Distributing back propagation networks over the Intel iPSC Hypercube. IEEE Int Joint Conf Neural Netw 1: 569-574
60. Jahnke A, Roth U, Klar H (1996) A SIMD/dataflow architecture for a neurocomputer for spike-processing neural networks (NESPINN), MicroNeuro'96, pp 232-237
61. James-Roxby P, Blodget BA (2000) Adapting constant multipliers in a neural network implementation. In: Proceedings of IEEE symposium on field-programmable custom computing machines, pp 335-336
62. Kakkar V (2009) Comparative study on analog and gital neural networks. Int J Comput Sci Netw Secur (IJCSNS) 9(7): 14-19
63. Kameda S, Yagi T (2003) An analog VLSI chip emulating sustained and transient response channels of the vertebrate retina. IEEE Trans Neural Netw 14(5): 1405-1412
64. Kane J, Paquin M (1993) POPART: practical optical implementation of adaptative resonance theory 2. IEEE Trans Neural Netw 4: 695-702
65. Kato H et al (1990) A parallel neurocomputer architecture towards billion connection updates per second. Int Joint Conf Neural Netw 2: 47-50
66. Kung HT (1987) The warp computer: architecture, implementation and performance. IEEE Trans Comput 36(12): 1523-1528
67. Lee P et al (1997) Advances in the design of the TOTEM neurochip. Nucl Instrum Methods Phys Res Sect A Accel Spectrom Detect Assoc Equip 389: 134-137
68. Liao Y (2001) Neural networks in hardware: a survey. Available in the internet at http://bit.csc.lsu.edu/~jianhua/shiv2.pdf (accessed November 2011)
69. Linares-Barranco B, Sanchez-Sinencio E, Rodriguez-Vazquez A, Huertas J. L (1993) A CMOS analog adaptive BAM with on-chip learning and weight refreshing. IEEE Trans Neural Netw 4(3): 445-455
70. Lindsey C (2002) Neural networks in hardware: architectures, products and applications. Available in the internet at: http://www.particle.kth.se/ ~lindsey/HardwareNNWCourse/home.html (accessed November 2011)
71. Lindsey CS, Denby B, Lindblad T (1998) Neural network hardware. Available in the internet at: http://neuralnets.web.cern.ch/NeuralNets/nnwInHepHard.html (accessed November 2011)
72. Lippmann RP (1987) An Introduction to computing with neural nets. IEEE ASSP Mag 4(2): 4-22
73. Lysaght P (1991) Dynamically reconfigurable logic in undergraduate projects. In: Moore W, Luk W (eds) FPGAs. Abingdon EE&CS Books, England
74. Mahowald M (1994) Analog VLSI chip for stereocorrespondence. In: Proceedings of IEEE International Symposium on Circuits and Systems, vol 6, pp 347-350
75. McCartor H (1991) Back propagation implementation on the adaptive solutions CNAPS neurocomputer chip. In: Lippmann R et al (eds) Proceedings of NIPS-3 advances in neural information processing systems, 3 edn, pp 1028-1031
76. McCulloch W, Pitts V (1943) A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 5: 115-133
77. Mead C, Ismail M (1989) Analog VLSI implementation of neural systems. Kluwer academic publisher, Boston, pp 239-246
78. Melton M et al (1992) The TINMANN VLSI chip. IEEE Trans Neural Netw 3(3): 375-384
79. Menendez A, Paillet G (2008) Fish inspection system using a parallel neural network chip and the image knowledge builder application. Artif Intell Mag 29(1): 21-28
80. Merchant SG, Peterson GD (2010) Evolvable block-based neural network design for applications in dynamic environments. VLSI Design Hindawi Publishing Corporation. doi: 10.1155/2010/251210
81. Morgan N et al. (1990) The RAP: a ring array processor for layered network calculations. In: Proceedings of the Conference on application specific array processors, Princeton, NJ, pp 296-308
82. Moussa M, Areibi S, Nichols K (2006) On the arithmetic precision for implementing back-propagation networks on FPGA: a case study. In: Omondi AR, Rajapakse JC (eds) FPGA implementations of neural networks. Springer, Berlin, pp 37-61
83. Muller UA, Gunzinger A, Guggenb'uhl W (1995) Fast neural net simulation with a DSP processor array. IEEE Transactions on Neural Networks, pp 203-213
84. Munoz AR et al (2008) An IP core and GUI for implementing multilayer perceptron with a fuzzy activation function on configurable logic devices. J Univers Comput Sci 14(10): 1678-1694
85. Murray AF, Smith AVW (1987) A novel computational and signaling method for VLSI neural networks. European solide state circuit conference, pp 19-22
86. Murray AF, Smith AVW (1988) Asynchronous VLSI neural networks using pulse stream arithmetic. IEEE J Solid State Circuits 23(3): 688-697
87. Murray AF et al (1991) Pulse stream VLSI networks mixing analog and digital techniques. IEEE Trans Neural Netwo 2(2): 193-204
88. NC3003-Digital Processor for Neural Networks (2011) Data sheet, Rel. 12/99 available in the internet at: http://www.digchip.com/datasheets/parts/ datasheet/327/NC3003-pdf.php (accessed on November 2011)
89. Nestor Inc. Providence R.I Ni1000 (1994) Recognition Accelerator Datasheet
90. Nichols KR (2004) A reconfigurable computing architecture for implementing artificial neural networks on FPGA. Master Thesis, University of Guelph
91. o TULEA 1991:16, Luleå University of Technology, Sweden
92. Nordstrom T (1995) On line localized learning systems part II: parallel computer implementation. Research report TULEA 1995:02, Division of Computer Science and Engineering, Lulea University of Technology, Sweden
93. Nordstrom T, Svensson B (1992) Using and designing massively parallel computers for artificial neural networks. J Parallel Distrib Comput Special Issue Neural Comput Massively Parallel Process 14(3): 260-285
94. Omandi AR, Rajapakse J (2006) FPGA implementations of neural networks. Kluwer, Dordrecht
95. Onuki J, Maenosono T et al (1993) ANN accelerator by parallel processor based on DSP. In: Proceedings of the IJCNN-93-Nagoya, pp 1913-1916
96. Passos Almeida A, Franca JE (1993) A mixed-mode architecture for implementation of analog neural networks with digital programmability. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN'93), Nagoya, Japan, vol 1, pp 887-890
97. Pearson MJ et al (2005) Design and FPGA implementation of an embedded real-time biologically plausible spiking neural network processor. International Conference on Field Programmable Logic and Applications, pp 582-585
98. Peiris V, Hochet B, Declercq M (1994) Implementation of a fully parallel Kohonen map: a mixed analog/digital approach. IEEE Int Conf Neural Netw 4: 2064-2069
99. Peres-Uribe A (1999) Structure adaptable digital neural networks. PhD thesis 2052, Logic System laboratory, Computer Science department, Swiss federal Institute of Technology- Lausanne
100. Porrmann M (2002) Implementation of artificial neural networks on a reconfigurable hardware accelerator. 10th Euromicro Workshop on Parallel, Distributed and Network-based Processing, pp 243-250. doi: 10.1109/EMPDP.2002. 994279
101. Prange SJ (1991) Architectures for a biology-oriented neuroemulator. VLSI design of neural networks. Kluwer, Dordrecht, pp 83-102
102. Ramacher U, Ruckert U (1991) VLSI design of neural networks. Kluwer, Dordrecht
103. Ramacher U, Wesseling M (1990) Systolic synthesis of neural networks. Int Neural Netw Conf 2: 572-576
104. Ramacher U et al (1991) Design of a first generation neurocomputer. VLSI design of neural networks. Kluwer, Norwell, pp 271-310
105. Ramacher U, Raab W et al (1993) Multiprocessor and memory architecture of the neurocomputer SYNAPSE-1. In: Proceedings of the 3rd International conference on microelectronics for neural networks (Micro Neuro), Edinburgh, pp 227-231
106. Rocke P, McGinley B, Maher J, Morgan F, Harkin J (2008) Investigating the suitability of FPAAs for evolved hardware spiking neural networks. In: ICES 2008, LNCS 5216, pp 118-129
107. Rodriguez A, Dominguez CR, Rueda A, Huertas RL, Sanchez-sinenco E (1990) Non linear switched capacitor neural networks for optimization problems. IEEE Trans Circuits Syst 37(3): 384-398
108. Sanada A, Ishii K, Yagi T (2010) A robot vision system using a silicon retina. Brain Inspir Inf Technol 266: 135-139. doi: 10.1007/978-3-642-04025-2-23
109. Sato Y et al (1993) Development of a highperformance, general purpose neuro-computer composed of 512 digital neurons. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN '93), Nagoya, Japan, vol 2, pp 1967-1970
110. Schoenauer T et al (1998) Digital neurohardware: principles and perspectives. Neural Networks in Applications, Magdeburg, pp 101-106
111. Schmitz T, Hohmann S, Meier K, Schemmel J, Schurmann F (2003) Speeding up hardware evolution: a coprocessor for evolutionary algorithms. Evolv Syst Biol Hardw Lect Notes Comput Sci 2606: 274-285
112. Schneider CR, Card HC (1991) CMOS mean field learning. Electron Lett 27(19): 1702-1704
113. Schrauwen B, Van Campenhout J (2006) Parallel hardware implementation of a broad class of spiking neurons using serial arithmetic. In: Proceedings of ESANN, pp 623-628
114. Sejnowski TJ, Rosenberg CR (1987) Parallel networks that learn to pronounce English text. Complex Syst 1:145-168
115. Shayani H, Bentley PJ, Tyrrell AM (2008) An FPGA-based model suitable for evolution and development of spiking neural networks. European symposium on artificial neural networks advances in computational intelligence and Learning, pp 197-202. ISBN 2-930307-08-0
116. Shima T et al (1992) Neuro chips with on-chip back-propagation and/or hebbian learning. IEEE J Solid State Circuts 27: 1868-1876
117. Silviott MA, Mahowald MA, Mead CA (1987) Real-time visual computations using analog CMOS processing arrays. In: Proceedings Of the stanford advanced research in VLSI conference. MIT Press, Cambridge
118. Skrbek M (1999) Fast neural network implementation. Neural Netw World 9(5): 375-391
119. Speckman H, Thole P, Rosenstiel W (1993) COKOS: a co-processor for Kohonen's self organizing map. In: Proceedings of the ICANN-93-Amsterdam. Springer, London, pp 1040-1045
120. Sudarshan S, Santosh K, Pinjare SL (2010) MDAC Synapse-Neuron for Analog Neural Networks, 2, 523-527
121. Tang T, Ishizuka O, Matsumoto H (1993) Backpropagation learning in analog T-model neural network hardware. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN '93), Nagoya, Japan, vol 1, pp 899-902
122. Tavenik M, Linde A (1995) A reconfigurable SIMD computer for artificial neural networks. Licentiate thesis No 189L. Department of Computer Engineering Chalmers, University of Technology, Goteborg, Sweden
123. Titri S, Boumeridja H, Lazib D, Izeboudjen N (1999) A reuse oriented design methodology for artificial neural networks implementation. Twelfth Annual IEEE International ASIC/SOC conference, pp 409-413
124. Treleaven P, Pacheco M, Vellasco M (1989) VLSI Architectures for Neural Networks. IEEE MICRO, pp 8-27
125. Trealeven PC (1989) Neurocomputers. Int J Neural Comput 1: 4-31
126. Tsividis Y, Anastassiou D (1987) Switched-capacitor neural networks. Electron Lett 23(18): 958-959
127. Uker A, Alkar AZ (2006) HW/SW Code sign of FPGA-based Neural Networks. Fifteenth Turkish Symposium on Artificial Intelligence and Neural Networks
128. Upegui A, Pena-Reyes CA, Sanchez E (2005) An FPGA platform for on-line topology exploration of spiking neural networks. Microprocess Microsyst Elsevier 29: 211-223
129. Viredaz MA, Ienne P (1993) MANTRA- I: a systolic neurocomputer. In: Proceedings of the International Joint Conference on Neural Networks III, pp 3054-3057
130. Watanabe T et al (1989) Neural network simulation on a massively parallel cellular array processor: AAP-2. International Joint Conference on Neural Networks, Washington DC, vol 2, pp 155-161
131. Wawrzynek J, Asanovi K, Morgan N (1993) The design of a neuro microprocessor. IEEE Trans Neural Netw 4: 394-399
132. Wolf DF et al (2001) Using embedded processors in hardware models of artificial neural networks. In: Proceeding of the Brazilian Symposium of Intelligent Automation
133. Xie Y, Jabri MA (1991) Training algorithms for limited precision feed forward neural networks. Technical report SEPAL
134. Yasunaga M et al (1989) A wafer scale integration neural network utilizing completely digital circuits. Int Joint Conf Neural Netw 2: 213-217
135. Zell A, Korb T, Sommer T, Bayer R (1990) Recent developments of the snns neural network simulator. In: Proceedings of the applications of neural networks conference, SPIE 1294, pp 534-544
136. Zhu J, Milne GJ, Gunther BK (1999) Towards an FPGA based reconfigurable computing environment for neural network implementations. In: Ninth International Conference on artificial neural networks, vol 2, pp 661-666