Biologically-Inspired On-Chip Learning in Pulsed Neural Networks

Analog Integrated Circuits and Signal Processing

Volumn 18, Issue 2-3, 1999, Pages 117-131

  Lehmann, Torsten ^a   Woodburn, Robin ^b  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

^b UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

Analog VLSI; Classical conditioning; Hebbian learning; On chip learning

Indexed keywords

ALGORITHMS; LEARNING SYSTEMS; VLSI CIRCUITS;

BIOLOGICALLY-INSPIRED ON-CHIP LEARNING; HEBBIAN LEARNING;

NEURAL NETWORKS;

EID: 0032629465     PISSN: 09251030     EISSN: None     Source Type: Journal    
DOI: 10.1023/a:1008398919379     Document Type: Article

References (56)

1. J. Alspector, G. Bhusan, and R. B. Allen, "Performance of a stochastic learning microchip." In D. S. Touretzky editor, Advances in Neural Information Processing Systems 1, 749-760. Morgan Kaufmann, San Mateo, Ca, 1989.
2. J. Alspector, A. Jayakumar, and S. Luna, "Experimental evaluation of learning in a neural microsystem." In J. E. Moody, S. J. Hanson, and R. P. Lippman, editors, Advances in Neural Information Processing Systems 4, chapter 144, pp. 871-878. Morgan Kaufmann, San Mateo, Ca, 1992.
3. Y. Arima, M. Murasaki, T. Yamada, A. Maeda, and H. Shinohara, "A refreshable analog VLSI neural network chip with 400 neurons and 40k synapses." IEEE Journal of Solid-State Circuits 27(12), pp. 1854-1867, 1992.
4. D. Badoni, S. Bertazzoni, S. Buglioni, G. Salina, D. J. Amit, and S. Fusi, "Electronic implementation of an analog attractor neural-network with stochastic learning." Network-Computation in Neural Systems 6(2), pp. 125-157, 1995.
5. Y. Berg, R. L. Sigvartsen, T. S. Lande, and A. Abusland, "An analog feed-forward neural network with on-chip learning." Analog Integrated Circuits and Signal Processing 9(1), pp. 65- 75, 1996.
6. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford University Press, Oxford, 1995.
7. M. E. Boulton, "Conditioning, remembering, and forgetting." Journal of Experimental Psychology: Animal Behavior Processes 20(3), pp. 219-231, 1994.
8. J. H. Byrne, "Cellular analysis of associative learning." Physiological Reviews 67(2), pp. 329-439, 1987.
9. G. Cairns and L. Tarassenko, "Perturbation techniques for on-chip learning with analog VLSI MLPs." Journal of Circuits, Systems and Computers 6(2), pp. 93-113, 1996.
10. G. Cauwenberghs, "An analog VLSI recurrent neural network learning a continuous-time trajectory." IEEE Transactions in Neural Networks 7(2), pp. 346-361, 1996.
11. M. H. Cohen and A. C. Andreou, "MOS circuit for nonlinear hebbian learning." Electronics Letters 28(6), pp. 591-593, 1992.
12. B. K. Dolenko and H. C. Card, "Neural learning in analog hardware : effects of component variation from fabrication and from noise." Electronics Letters 29(8), pp. 693-694, 1993.
13. B. K. Dolenko and H. C. Card, "Tolerance to analog hardware of on-chip learning in backpropagation networks." IEEE Transactions on Neural Networks 4(5), pp. 1045-1052, 1995.
14. J. Donald and L. Akers L, "A neural processing node with on-chip learning." In Proceedings of IEEE International Symposium on Circuits and Systems, Chicago 4, pp. 2748-2751, 1993.
15. R. Douglas, M. Mahowald, and C. Mead, "Neuromorphic analog VLSI." Annual Review of Neuroscience 18, pp. 255-281, 1995.
16. J. G. Elias and D. P. M. Northmore, "Switched-capacitor neuromorphs with wide-range variable dynamics." IEEE Transactions on Neural Networks 6(6), pp. 1542-1548, 1995.
17. F. Faggin and C. Mead, "VLSI implementation of neural networks." In S. F. Zornetzer, J. L. Davis, and C. Lau, editors, An Introduction to Neural and Electronic Networks chapter 13, pp. 275-292. Academic Press, San Diego, Ca, 1990.
18. I. M. Filanovsky and H. Baltes, "CMOS Schmitt trigger design." IEEE Transactions on Circuits and Systems, pt I 41(1), pp. 46-49, 1994.
19. P. Fromherz and A. Stett, "Silicon-neuron junction: Capacitive stimulation of an individual neuron on a single chip." Physical Review Letters 75(8), pp. 1670-1673, 1995.
20. M. Gluck, D. B. Parker, and E. S. Reifsnider, "Learning with temporal derivatives in pulse-coded neuronal systems." In D. S. Touretzky, editor, 1988 IEEE Conf. on Neural Information Processing Systems. 1, pp. 195-203, Denver, CO, 1989. Morgan Kaufmann Pub. Inc.
21. B. Gupta, R. Goodman, F. Jiang, Y. Tia, S. Tung, and C. Ho, "Analog VLSI system for active drag reduction." IEEE Micro 16(5), pp. 30-39, 1996.
22. J. Hertz, A. Krogh, B. Lautrup, and T. Lehmann, "Nonlinear backpropagation: doing backpropagation without derivatives of the activation function." IEEE Transactions on Neural Networks 8(6), pp. 1321-1327, 1997.
23. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Addison-Wesley Publishing Company, Redwood City, 1991.
24. M. Jabri and B. Flower, "Weight perturbation: An optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks." IEEE Transactions on Neural Networks 3(1), pp. 154-157, 1992.
25. E. Kandel, J. Schwartz, and T. Jessel, Principles of Neural Science. Elsevier, New York, 1991.
26. D. B. Kirk, D. Kerns, K. Fleischer, and A. H. Barr, "Analog VLSI implementation of multi-dimensional gradient descent." In S. J. Hanson, J. D. Cowan, and C. L. Giles, editors, Advances in Neural Information Processing Systems 5, pp. 789-796. Morgan Kaufmann, San Mateo, Ca, 1993.
27. A. H. Klopf, "A neuronal model of classical-conditioning." Psychobiology 16(2), pp. 85-125, 1988.
28. J. Lazzaro, J. Wawrzynek, and A. Kramer, "Systems technologies for silicon auditory models." IEEE Micro 14(3), pp. 7-15, 1994.
29. T. Lehmann, Hardware Learning in Analog VLSI Neural Networks. PhD thesis, Electronics Institute, Technical University of Denmark, Lyngby, 1994.
30. T. Lehmann, "Implementation issues of self-learning pulsed integrated neureal systems." In 13th NORCHIP Conference pp. 145-152, Copenhagen, Denmark, 1995.
31. T. Lehmann, "Eccopunch: Algorithms, circuits and systems; design and evaluation." Technical report, The University of Edinburgh, Integrated Systems Group, Dep. Electrical Engineering, 1996.
32. T. Lehmann, "Teaching pulsed integrated neural systems: a psychobiological approach." In Fifth International Conference on Microelectronics for Neural Networks and Fuzzy Systems pp. 185-190, Lausanne, Switzerland, 1996.
33. T. Lehmann, "Classical conditioning with pulsed integrated neural networks: Circuits and system." Submitted to IEEE Trans. Circuits and Systems II, 1997.
34. T. Lehmann, "Eccopunch: the Edinburgh classical conditioning pulsed neural chip." In Proceedings of the IEEE International Symposium on Circuits and Systems, pp. 33-36, Hong Kong, 1997.
35. T. Lehmann, E. Bruun, and C. Dietrich, "Mixed analog/digital matrix-vector multiplier for neural network synapses." Analog Integrated Circuits and Signal Processing 9(1), pp. 55-63, January 1996.
36. T. Lehmann, R. Woodburn, and A. F. Murray, "On-chip learning in pulsed silicon neural networks." To appear in Proc. IEEE International Symposium on Circuits and Systems, Invited paper, 1997.
37. D. S. Levine, Introduction to Neural and Cognitive Modeling. Lawrence Erlbaum Associates, Inc., London, 1991.
38. I. Levitan and L. Kaczmarek, The Neuron : Cell and Molecular Biology. Oxford University Press, New York, 1991.
39. B. Linares Barranco, E. Sanchez Sinencio, A. Rodriguez Vazquez, and J. Huertas, "A CMOS analog adaptive BAM with on-chip learning and weight refreshing." IEEE Transactions on Neural Networks 4(3), pp. 445-455, 1993.
40. C. Mead, Analog VLSI & Neural Systems. Addison-Wesley Publishing Company, Reading, 1989.
41. A. J. Montalvo, R. S. Gyurcsik, and J. J. Paulos, "Building blocks for a temperature-compensated analog VLSI neural network with on-chip learning." In IEEE International Symposium on Circuits and Systems 6, pp. 329-332, 1994.
42. A. J. Montalvo, P. W. Hollis, and J. J. Paulos, "On-chip learning in the analog domain with limited precision circuits." In International Symposium on Circuits and Systems pp. I-196-I-201, 1992.
43. J. S. Morgan, E. C. Patterson, and A. H. Klopf, "A drive-reinforcement neural network model of simple instrumental conditioning." In International Joint Conf. on Neural Networks, pp. 227-232, San Diego, CA, 1990. IEEE.
44. T. Morie and Y. Amemiya, "An all-analog expandable neural network LSI with on-chip backpropagation learning." IEEE Journal of Solid-State Circuits 29(9), 1086-1093, 1994.
45. D. B. Mundie and L. W. Massengill, "Weight decay and resolution effects in feedforward artificial neural networks." IEEE Transactions on Neural Networks 2(1), pp. 168-170, 1991.
46. A. F. Murray, D. D. Corso, and L. Tarassenko, "Pulse-stream VLSI neural networks mixing analog and digital techniques." IEEE Transactions on Neural Networks 2(2), pp. 193-204, March 1991.
47. A. F. Murray and P. J. Edwards, "Synaptic weight noise during MLP training : Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training." IEEE Transactions on Neural Networks 5(5), pp. 792-802, 1994.
48. I. Pavlov, Conditioned reflexes : an investigation of the physiological activity of the cerebral cortex. Oxford University Press, Oxford, 1927.
49. S. Porsgaard, "Democracy in neural networks." Master's thesis, Electronics Institute, Technical University of Denmark, Lyngby, 1994. In Danish.
50. U. Ramacher and P. Schildberg, "Recent developments in neurodynamics and their impact on the design of neuro-chips." International Journal of Neural Systems 4(4), pp. 309-316, 1993.
51. D. Schwartz, R. Howard, and L. Hubbard, "Adaptive neural networks using MOS charge storage." In D. S. Touretzky, editor, Advances in Neural Information Processing Systems 1, pp. 761-768, Morgan Kaufmann, San Mateo, Ca, 1989.
52. P. Shah, Design of Analog Integrated Circuits for Very Low Frequency Signal Processing. PhD thesis, Electronics Institute, Technical University of Denmark, Lyngby, 1993.
53. T. Shima, T. Kimura, Y. Kamatani, T. Itakura, Y. Fujita, and T. Iida, "Neuro chips with on-chip back-propagation and/or Hebbian learning." IEEE Journal of Solid-State Circuits 27(12), pp. 1868-1876, 1992.
54. P. A. Shoemaker, C. Hutchens, and S. Patil, "A hierarchical clustering network based on a model of olfactory processing." Analog Integrated Circuits and Signal Processing 2(4), pp. 297-311, 1992.
55. R. Woodburn, H. M. Reekie, and A. F. Murray, "Pulse-stream circuits for on-chip learning in analog VLSI neural networks." In Proceedings of the IEEE International Symposium on Circuits and Systems 4, pp. 103-106, 1994.
56. M. Zaghloul, J. L. Meador, and R. W. Newcomb, editors. Silicon Implementation of Pulse Coded Neural Networks. Kluwer Academic Publishers, London, 1994.