Analog Implementation of ANN with Inherent Quadratic Nonlinearity of the Synapses

IEEE Transactions on Neural Networks

Volumn 14, Issue 5, 2003, Pages 1187-1200

  Milev, Momchil ^a,b   Hristov, Marin ^b  

^a TEXAS INSTRUMENTS   (United States)

^b TECHNICAL UNIVERSITY OF SOFIA   (Bulgaria)

Author keywords

Analog implementations; Artificial neural networks (ANNs); Nonlinear synapse; Very large scale of integration (VLSI)

Indexed keywords

ANALOG STORAGE; CMOS INTEGRATED CIRCUITS; COMPUTER SIMULATION; CONVERGENCE OF NUMERICAL METHODS; FEATURE EXTRACTION; LEARNING SYSTEMS; MATHEMATICAL MODELS; MULTILAYER NEURAL NETWORKS; NONLINEAR SYSTEMS; QUADRATIC PROGRAMMING; VLSI CIRCUITS;

NONLINEAR SYNAPSE;

FEEDFORWARD NEURAL NETWORKS;

EID: 0242611592     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2003.816369     Document Type: Article

References (40)

1. A. A. Blumer, D. H. Ehrenfeucht, and M. K. Warmuth, "Learnability and the Vapnik-Chervonenkis dimension," J. Association for Computing Machinery, vol. 36, pp. 929-965, 1989.
2. R. W. Newcomb and N. El-Leithy, "Perspectives on realizations of neural networks," in CAS, IEEE Int. Symp, Neural Networks, vol. 2, 1989, pp. 818-819.
3. P. E. Allen and D. H. Holberg, CMOS Analog Circuit Design. Oxford, U.K.: Oxford Univ. Press, 1987, pp. 98-101.
4. R. Gregorian and G. C. Temes, Analog MOS Integrated Circuits For Signal Processing. New York: Wiley, 1986, pp. 462-483.
5. S. Haykin, Neural Networks - A Comprehensive Foundation, 2nd ed. Upper Saddle River, NJ: Prentice-Hall, 1999, pp. 3-18.
6. M. Milev, "Hardware model of a neural synapse with nonlinearity," in Tenth National Application-Scientific Conference on Electronic Technology with International Participation ET-2001, Sozopol, Bulgaria, 2001.
7. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations of back-propagation errors," Nature, vol. 323, pp. 533-536.
8. M. Hagiwara, "Theoretical derivation of momentum term in back-propagation," in Proc. Int. Joint Conf. Neural Networks, vol. I, 1992, pp. 682-686.
9. S. Haykin, Neural Networks - A Comprehensive Foundation, 2nd ed. Upper Saddle River, NJ: Prentice-Hall, 1999, pp. 170-171.
10. M. Milev, "LMS training of a PE with nonlinearity with respect to weights as a linear classifier," presented at the Ninth National Conference on Electronic Technology with International Participation, Sozopol, Bulgaria, 2000.
11. P. Liang and N. Jamali, "Artificial neural networks with nonlinear synapses and nonlinear synaptic contacts," in Proc. IEEE Int. Conf. Systems, Man, Cybernetics, vol. 2, 1992, pp. 1043-1048.
12. J. B. Lont and W. Guggenbuhl, "Analog CMOS implementation of a multilayer perceptron with nonlinear synapses," IEEE Trans. Neural Networks, vol. 3, pp. 457-465, May 1992.
13. J. B. Lont, "A large scale neural network with nonlinear synapses," in Proc. 1994 IEEE World Congr. Computational Intelligence, vol. 1, 1994, pp. 350-353.
14. _, "A large scale neural network with nonlinear synapses," in Proc. IEEE World Congr. Computational Intelligence., vol. 1, 1994, pp. 350-353.
15. K. Nakayama, A. Hirano, and M. Fusakawa, "A selective learning algorithm for nonlinear synapses in multilayer neural networks," in Proc. Int. Joint Conf. Neural Networks, vol. 3, 2001, pp. 1704-1709.
16. M.-R. Choi and J.-S. Park, "Implementation of MEBP learning circuitry with simple nonlinear synapse circuits," in Proc. Fuzzy Systems Conf., vol. 1, 1999, pp. 315-320.
17. Y. Horio, M. Ymamamoto, and S. Nakamura, "Active analog memories for neuro-computing," in IEEE Int. Symp. Circuits Systems, vol. 4, May 1-3, 1990, pp. 2986-2989.
18. T. P. Allen, J. B. Cser, and Synaptics, Inc, "Method for Changing the Weight of a Synaptic Element," U.S. Patent no. 6 023 422, Feb. 8, 2003.
19. B. A. Minch, P. E. Hasler, C. J. Diorio, C. A. Mead, and California Institute of Technology, Pasadena, CA, "Autozeroing Floating-Gate Amplifier," U.S. Patent no. 5 986 927, Nov. 16, 1999.
20. C. J. Diorio, P. E. Hasler, B. A. Minch, C. A. Mead, and California Institute of Technology, Pasadena, CA, "Method for Implementing a Learning Function," U.S. Patent no. 5 914 894, June 22, 1999.
21. T. P. Allen, J. B. Cser, and Synaptics Inc., San Jose, CA, "One-Transistor Adaptable Analog Storage Element and Array," United States Patent no. 5 864 242, Jan. 26, 1999.
22. C. J. Diorio, P. E. Hasler, B. A. Minch, C. A. Mead, and California Institute of Technology, San Jose, CA, "Three-Terminal Silicon Synaptic Device," U.S. Patent no. 5 825 063, Oct. 20, 1998.
23. T. Ohmi, T. Shibata, H. Kosaka, and Y. Takeo, "Semiconductor Device," U.S. Patent no. 5 818 081, Oct. 6, 1998.
24. C. J. Diorio, P. E. Hasler, B. A. Minch, C. A. Mead, and California Institute of Technology, San Jose, CA, "Semiconductor Structure for Long Term Learning," U.S. Patent no. 5 627 392, May 6, 1997.
25. T. Shibata and T. Ohmi, "Neuron Circuit," U.S. Patent no. 5 621 336, Apr. 15, 1997.
26. G. Imondi, G. Marotta, E. Pasero, G. Porrovecchio, and G. Savarese, "Integrated Circuit With Non-Volatile, Variable Resistor, for Use in Neuronic Network," U.S. Patent no. 5 457 771, Oct. 10, 1995.
27. T. P. Allen, J. B. Cser, and Synaptics, Incorporated, San Jose, CA, "One-Transistor Adaptable Analog Storage Element and Array," U.S. Patent no. 5 336 936, Aug. 9, 1994.
28. T. P. Allen, J. D. W. Anderson, C. A. Mead, F. Faggin, J. C. Platt, M. F. Wall, and Synaptics, Incorporated, San Jose, CA, "Electrically Adaptable Neural Network With Post-Processing Circuitry," U.S. Patent no. 5 331 215, July 19, 1994.
29. R. L. Shimabukuro, M. E. Stewart, P. A. Shoemaker, and G. A. Garica, "MOS Analog Memory With Injection Capacitors," U.S. Patent no. 5 253 196, Oct. 12, 1993.
30. J. C. Platt, J. D. W. Anderson, and C. A. Mead, "Synaptic Element Including Weight-Storage and Weight-Adjustment Circuit," U.S. Patent no. 5 204 549, Apr. 20, 1993.
31. R. E. Reedy, R. L. Shimabukuro, and G. A. Garcia, "Dual Polarity Floating Gate MOS Analog Memory Device," U.S. Patent no. 5 027 171, June 25, 1991.
32. I. S. Han, Y. J. Choi, and D. H. Kim, "Multiplier and Neural Network Synapse Using Current Mirror Having Low-Power mosfets," U.S. Patent no. 5 914 868, June 22, 1999.
33. C. Y Kwok and H. R. Mehrvarz, "Four Quadrant Square Law Analog Multiplier Using Floating Gate MOS Transitions," U.S. Patent no. 5 864 255, Jan. 26, 1999.
34. I. S. Han, "MOSFET Analog Multiplier," U.S. Patent no. 5 254 889, Oct. 19, 1993.
35. G. Marotta and E. Pasero, "Voltage-Current Conversion Circuit Employing MOS Transistor Cells as Synapses of Neural Network," U.S. Patent no. 5 704 014, Dec. 30, 1997.
36. K. A. Boahen, A. G. Andreou, P. O. Pouliquen, and R. E. Jenkins, "Current-Mode Based Analog Circuits for Synthetic Neural Systems," U.S. PAtent no. 5 206 541, Apr. 27, 1993.
37. H.-S Chung, "Synapse MOS Transistor," U.S. Patent no. 5 442 209, Aug. 15, 1995.
38. S. Isoda and Y. Hanazato, "Information Processing Device Capable Optically Writing Synapse Strength Matrix," U.S. Patent no. 5 546 504, Aug. 13, 1996.
39. M. M. Milev, "Method and Apparatus for Modeling a Neural Synapse Function by Utilizing a Single Conventional MOSFET," U.S. Patent Application no. 09/968 263. Pub. no. US2002/0057 606 A1, May 16, 2002.
40. R. Gregorian and G. C. Temes, Analog MOS Integrated Circuits for Signal Processing. New York: Wiley, 1986, pp. 75-78.