Neuromorphic Synapses for Artificial Dendrites

Analog Integrated Circuits and Signal Processing

Volumn 13, Issue 1-2, 1997, Pages 167-184

  Westerman, Wayne C ^a,b   Northmore, David P M ^a,c   Elias, John G ^a,d,e  

^a UNIVERSITY OF DELAWARE   (United States)

^b PURDUE UNIVERSITY   (United States)

^c UNIVERSITY OF OXFORD   (United Kingdom)

^d Electrical Engineering Department   (United Kingdom)

^e YALE UNIVERSITY   (United States)

Author keywords

Dendrites; Neuromorphic; Synapses; VLSI

Indexed keywords

APPROXIMATION THEORY; COMPUTATIONAL METHODS; COMPUTER SOFTWARE; ELECTRIC RESISTANCE; MICROPROCESSOR CHIPS; NEURAL NETWORKS; SPURIOUS SIGNAL NOISE;

ARTIFICIAL DENDRITES; NEUROMORPHIC DENDRITES;

VLSI CIRCUITS;

EID: 0031145312     PISSN: 09251030     EISSN: None     Source Type: Journal    
DOI: 10.1023/a:1008248431434     Document Type: Article

References (94)

1. J. G. Elias, "Artificial dendritic trees." Neural Computation 5, pp. 648-663, 1993.
2. J. G. Elias and D. P. M. Northmore, "Switched-capacitor neuromorphs with wide-range variable dynamics." IEEE Trans. on Neural Networks 6, pp. 1542-1548, 1995.
3. R. Douglas and M. Mahowald, "A constructor set for silicon neurons," in An Introduction to Neural and Electronic Networks (S. F. Zornetzer, J. Davis and T. McKenna, eds.). Academic Press, 1994.
4. T. H. Brown, E. W. Kairiss and C. L. Keenan, "Hebbian synapses: Biophysical mechanisms and algorithms." Annu. Rev. of Neurosci. 13, pp. 475-511, 1990.
5. T. Shima, T. Kimura, Y. Kamatani, T. Itakura, Y. Fujita, and T. Iida, "Neuro chips with on-chip back-propagation and/or Hebbian learning." IEEE J. of Solid-State Circuits 27(12), pp. 1868-1875, 1992.
6. Y. Arima, M. Murasaki, T. Yamada, A. Maeda and H. Shinohara, "A refreshable analog VLSI neural network chip with 400 neurons and 40K synapses." IEEE J. of Solid State Circuits 27, pp. 1854-1861, 1992.
7. W. Gerstner, R. Ritz, and J. Leo van Hemmen, "Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns." Biol. Cybern. 69, pp. 503-515, 1993.
8. D. V. Buonomano and M. M. Merzenich, "Temporal information transformed into a spatial code by a neural network with realistic properties." Science 267, pp. 1028-1030, 1995.
9. C. Koch and T. Poggio, "The biophysical properties of spines as a basis for their electrical function: A comment on Kawato and Tsakahara (1983)." J. Theor. Biol. 113, pp. 225-230, 1985.
10. T. M. Jessel, and E. R. Kandel, "Synaptic transmission: A bidirectional and self-modifiable form of cell-cell comunication." Cell 72, Neuron 10 (Suppl.), pp. 1-30, 1993.
11. B. Hille, Ionic Channels of Excitable Membranes, 2nd. ed. Sinauer Associates Inc., 1992.
12. E. R. Kandel, J. H. Schwartz and T. M. Jessell (eds.), "Part III: Elementary interactions between neurons: Synaptic transmission," in Principles of Neural Science, 3rd ed. Appleton and Lange: Norwalk, Connecticut, 1991, pp. 123-269.
13. G. M. Edelman, W. E. Gall, and W. M. Cowan (eds.), Synaptic Function. John Wiley and Sons, 1987.
14. W. Rall, "Theoretical significance of dendritic trees for neuronal input-output relations," in Neural Theory and Modeling (R. F. Reiss, ed.). Stanford University Press: Palo Alto, 1964.
15. T. Poggio and V. Torre, "A new approach to synaptic interactions," in Lecture Notes in Biomathematics. Theorectical Approaches to Computer Systems, vol. 2 (H. Heim and G. Palm, eds.). Heidelberg, NewYork, 1977, pp. 89-115.
16. G. M. Shepherd and C. Koch, "Introduction to synaptic circuits," in The Synaptic Organization of the Brain (G. M. Shepherd, ed.). Oxford University Press: NewYork, 1990, pp. 3-31.
17. K. Y. Tsai, N. T. Carnevale and T. H. Brown, "Hebbian learning is jointly controlled by electronic and input structure." Network 5, pp. 1-19, 1994.
18. B. W. Mel, "Information processing in dendritic trees." Neural Computation 6, pp. 1031-1085, 1994.
19. C. Koch and T. Poggio, "Biophysics of computation: Neurons, synapses, and membranes," in Synaptic Function (G. M. Edelman, W. E. Gall, W. M. Cowan, eds.). John Wiley and Sons, 1987, pp. 637-698.
20. C. Koch and T. Poggio, "Multiplying with synapses and neurons," in Single Neuron Computation, (T. McKenna, J. Davis, S. F. Zornetzer, eds.). Academic Press, Inc., 1992, pp. 315-346.
21. H. Korn and A. Mallet, "Transformation of binomial input by the postsynaptic membrane at a central synapse." Science 225, pp. 1157-1159, 1984.
22. D. P. M. Northmore and J. G. Elias, "Spike train processing by a silicon neuromorph: The role of sublinear summation in dendrites." Neural Computation 8, pp. 1245-1265, 1996.
23. W. Rall, "Distinguishing theoretical synaptic potentials computed for different soma-dendritic distributions of synaptic inputs." J. of Neurophys. 30, pp. 1138-1168, 1967.
24. J. L. Martinez and B. E. Derrick, "Long-term potentiation and learning." Annu. Rev. of Psychology 47, pp. 173-203, 1996.
25. F. A. Edwards, "LTP - A structural model to explain the inconsistencies." Trends in Neurosci. 18, pp. 250-255, 1995.
26. G. L. Collingridge and T. V. P. Bliss, "Memories of NMDA receptors and LTP." Trends in Neurosci. 18, pp. 54-56, 1995.
27. M. Bawdry and J. L. Davis, Long-Term Potentiation. Vol. 2. The MIT Press: Cambridge, MA, 1994.
28. P. O. Anderson, "Properties of Hippocampal Synapses of Importance for Integration and Memory," in Synaptic Function (G. M. Edelman, W. E. Gall, W. M. Cowan, eds.). John Wiley and Sons, 1987, pp. 403-430.
29. E. R. Kandel, M. Klein, B. Hochner, M. Shuster, S. A. Siegelbaum, R. D. Hawkins, D. L. Glanzman, V. F. Castellucci, and T. W. Abrams, "Synaptic modulation and learning: New insights into synaptic transmission from the study of behavior," in Synaptic Function (G. M. Edelman, W. E. Gall, W. M. Cowan, eds.). John Wiley and Sons, 1987, pp. 471-518.
30. D. M. Kullmann and R. A. Nicoll, "Long-term potentiation is associated with increases in quantal content and quantal amplitude." Nature 357, pp. 240-244, 1992.
31. A. Larkman, T. Hannay, K. Stratford and J. Jack, "Presynaptic release probability influences the locus of long-term potentiation." Nature 360, pp. 70-73, 1992.
32. T. V. P. Bliss, and G. L. Collingridge, "A synaptic model of memory: Long-term potentiation in the Hippocampus." Nature 361, pp. 31-39, 1993.
33. H. Korn and D. S. Faber, "Regulation and significance of probablistic release mechanisms at central synapses," in Synaptic Function (G. M. Edelman, W. E. Gall, W. M. Cowan, eds.). John Wiley and Sons, 1987, pp. 57-108.
34. N. A. Hessler, A. M. Shirke, and R. Malinow, "The probability of transmitter release at a mammalian central synapse." Nature 366, pp. 569-573, 1993.
35. T. H. Murphy, J. M. Baraban, W. G. Weir, L. A. Blatter, "Visualization of quantal synaptic transmission by dendritic calcium imaging." Science 263, pp. 529-532, 1994.
36. N. Otmakhov, A. M. Shrike and R. Malinow, "Measuring the impact of probabilistic transmission on neuronal output." Neuron 10, pp. 1101-1111, 1993.
37. A. Malgaroli, A. E. Ting, B. Wendland, A. Bergamaschi, A. Villa, R. W. Tsien, and R. H. Scheller, "Presynaptic component of long-term potentiation visualized at individual hippocampal synapses." Science 268, pp. 1624-1628, 1995.
38. D. Liao, N. A. Hessler, and R. Malinow, "Activation of post synaptically silent synapses during pairing-induced LTP in CA1 region of hippocampal slice." Nature 375, pp. 400-404, 1995.
39. J. M. Bekkers, "Quantal analysis of synaptic transmission in the central nervous system." Current Opinion in Neurobiology 4, pp. 360-365, 1994.
40. 2+ imaging of CNS axons in culture indicates reliable coupling between single action potentials and distal functional release sites." Neuron 16, pp. 783-795, 1996.
41. C. Allen and C. F. Stevens, "An evaluation of causes for unreliability of synaptic transmission." Proc. Natl. Acad. Sci. USA 91, pp. 10380-10383, 1994.
42. C. F. Stevens and Y. Wang, "Changes in reliability of synaptic function as a mechanism for plasticity." Nature 371, pp. 704-707, 1994.
43. A. F. Murray and P. J. Edwards, "Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training." IEEE Trans. on Neural Networks 5, pp. 792-802, 1994.
44. R. C. Malenka, J. A. Kauer, R. S. Zucker and R. A. Nicoll, "Postsynaptic calcium is sufficient for potentiation of Hippocampal synaptic transmission." Science 242, pp. 81-84, 1988.
45. R. C. Malenka, J. A. Kauer, D. J. Perkel, M. D. Mauk, P. T. Kelly, R. A. Nicoll and M. N. Waxham, "An essential role for postsynaptic calmodulin and protein kinase activity in long-term potentiation." Nature 340, pp. 554-557, 1989.
46. A. Zador, C. Koch, and T. H. Brown, "Biophysical model of a Hebbian synapse." Proc. Natl. Acad. Sci. U.S.A. 87, pp. 6718-6722, 1990.
47. D. E. Rummelhart, G. E. Hinton and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Explorations in the Microstructures of Cognition. Volume I: Foundations (D. E. Rumelhart, J. L. McClelland and the PDP Research Group, eds.). MIT Press/Bradford Books: Cambridge, MA.
48. G. M. Shepherd, "Canonical neurons and their computational organization," in Single Neuron Computation, (T. McKenna, J. Davis, S. F. Zornetzer, eds.). Academic Press, Inc., 1992, pp. 27-60.
49. Y. Hirai, "Recent VLSI neural networks in Japan." J. of VLSI Signal Processing 6, pp. 7-18, 1993.
50. H. P. Graf, E. Sackinger and L. D. Jackel, "Recent developments of electronic neural nets in North America." J. of VLSI Signal Processing 5, pp. 19-31, 1993.
51. S. Eberhardt, T. Duong and A. Thakoor, "Design of parallel hardware neural network systems from custom analog VLSI 'building blocks' chips." Int. Jt. Conf. Neural Network 2, pp. 183-190, 1988.
52. M. H. Cohen and A. G. Andreou, "Current-node subthreshold MOS implementation of the Herault-Jutten autoadaptive network." IEEE J. of Solid State Circuits 27, pp. 714-727, 1992.
53. B. W. Lee and B. J. Sheu, "General-purpose neural chips with electrically programmable synapses and gain-adjustable neurons." IEEE J. of Solid-State Circuits 27, pp. 1299-1302, 1992.
54. B. Linares-Barranco, E. Sanchez-Sinencio, A. Rodriguez-Vazquez and J. L. Huertas, "A CMOS analog adaptive BAM with on-chip learning and weight refreshing." IEEE Trans. on Neural Networks 4, pp. 445-457, 1993.
55. P. Mueller, J. V. D. Spiegel, D. Blackman, T. Chiu, T. Clare, J. Dao, C. Donham, T. Hsieh and M. Loinaz, "A general purpose analog neural computer." Int. Jt. Conf. Neural Network 2, pp. 177-182, 1989.
56. B. W. Lee, B. J. Sheu, and H. Yang, "Analog floating-gate synapses for general-purpose VLSI neural computation." IEEE Trans. on Circuits and Systems 38, pp. 654-658, 1991.
57. H. A. Castro, S. M. Tam, and M. A. Holler, "Implementation and preformance of an analog nonvolatile neural network." Analog Integrated Circuits and Signal Processing 4, pp. 97-113, 1993.
58. P. Hasler, C. Diorio, B. A. Minch and C. Mead, "Single transistor learning synapses with long-term storage." Proc. IEEE Int. Symp. on Circuits and Systems 3, pp. 1660-1663, 1995.
59. A. F. Murray and L. Tarassenko, Analogue Neural VLSI: A Pulse Stream Approach. Chapman and Hall, 1994.
60. G. Moon, M. E. Zaghloul, and R. W. Newcomb, "VLSI implementation of synaptic weighting and summing in pulse coded neural-type cells." IEEE Trans. Neural Networks 3, pp. 394-403, 1992.
61. M. E. Zaghloul, J. L. Meador, and R. W. Newcomb (eds.), Silicon Implementation of Pulse Coded Neural Networks. Kluwer Academic Publishers, 1994.
62. C. A. Mead and M. A. Mahowald, "Silicon model of early visual processing." Neural Networks 1, pp. 91-97, 1988.
63. T. Delbruck, "Silicon retina with correlation-based velocity-tuned pixels." IEEE Trans. on Neural Networks 4, pp. 529-541, 1993.
64. K. A. Boahen and A. G. Andreou, "A contrast sensitive silicon retina with reciprocal synapses." Advances in Neural Information Processing Systems 4, 1992.
65. L. Watts, D. A. Kerns, R. F. Lyon, and C. A. Mead, "Improved implementation of the silicon cochlea." IEEE J. of Solid-State Circuits 27(5), pp. 692-700, 1992.
66. J. P. Lazzaro and C. Mead, "Circuit models of sensory transduction in the cochlea," in Analog VLSI Implementations of Neural Networks (Mead and Ismail, eds.). Kluwer Academic Publishers: Norwell, MA, 1989, pp. 85-101.
67. M. Mahowald and R. Douglas, "A silicon neuron." Nature 354, pp. 515-518, 1991.
68. B. Linares-Barranco, E. Sanchez-Sinencio, A. Rodriguez-Vazquez and J. L. Huertas, "A CMOS implementation of FitzHugh-Nagumo neuron model." IEEE Solid-State Circuits 26(7), pp. 956-965, 1991.
69. A. Mortara and E. A. Vittoz, "A communication architecture tailored for analog VLSI artificial neural networks: Intrinsic performance and limitations." IEEE Trans. On Neural Networks 5(3), pp. 459-466, 1994.
70. J. Van der Spiegel, P. Mueller, D. Blackman, P. Chance, C. Donham, R. Etienne-Cummings, and P. Kinget, "An analog neural computer with modular architecture for real-time dynamic computations." IEEE J. of Solid-State Circuits 27, pp. 82-92, 1992.
71. M. A. Mahowald, "VLSI analogs of neuronal visual processing: A synthesis of form and function," Ph.D. dissertation, California Institute of Technology, Dept. of Computation and Neural Systems, Pasadena, CA, 1992.
72. C. A. Mead, Analog VLSI and Neural Systems. Addison-Wesley: New York, 1989.
73. M. Banu and Y. Tsividis, "Floating voltage-controlled resistors in CMOS technology." Electron. Lett. 18, pp. 678-679, 1982.
74. G. Moon, M. E. Zaghloul, and R. W. Newcomb, "An enhancement-mode MOS voltage-controlled linear resistor with large dynamic range." IEEE Trans. on Circuits and Systems 37, pp. 1284-1288, 1990.
75. Z. Wang, "Novel electronically-controlled floating resistors using MOS transistors operating in saturation." Electron. Lett. 27, pp. 188-189, 1991.
76. C. Acar and M. S. Ghausi, "Fully integrated active-RC filters using MOS and non-balanced structure," Int. J. of Circuit Theory and Applications 15, pp. 105-121, 1987.
77. T. Shibata, and T. Ohmi, "A functional MOS transistor featuring gate-level weighted sum and threshold operations." IEEE Trans. on Electron Devices 39, pp. 1444-1455, 1992.
78. Y. P. Tsividis and D. Anastassiou, "Switched-capacitor neural networks." Electron. Lett. 23, pp. 958-959, 1988.
79. J. Tomberg, "Synchronous pulse density modulation in neural network implementation," in Silicon Implementation of Pulse Coded Neural Networks (M. E. Zaghloul, J. L. Meador, R. W. Newcomb, eds.). Kluwer Academic Publishers, 1994, pp. 165-198.
80. I. Segev, J. W. Fleshman, and R. E. Burke, "Compartmental models of complex neurons," in Methods in Neuronal Modeling: From Synapses to Networks (C. Koch and I. Segev, eds.). The MIT Press, 1989, pp. 63-96.
81. G. Major, A. U. Larkman, P. Jones, B. Sakmann, and J. J. B. Jack, "Detailed passive cable models of whole-cell recorded CA3 pyramidal neurons in rat hippocampal slices." J. of Neurosci. 14(8), pp. 4613-4638, 1994.
82. D. P. Shelton, "Membrane resistivity estimated for the Purkinje neuron by means of a passive computer model." Neuroscience 14, pp. 111-131, 1985.
83. K. J. Stratford, A. J. R. Mason, A. U. Larkman, G. Major, J. J. B. Jack, "The modelling of pyramidal neurones in the visual cortex," in The Computing Neurone (R. Durbing, C. Miall, G. Mitchison, eds.). Addison-Wesley, 1989, pp. 296-321.
84. M. Rapp, I. Segev, Y. Yarom, "Physiology, morphology and detailed passived models of cerebellar Purkinje cell." J. of Physiol. 474, pp. 101-118, 1994.
85. D. Thurbon, A. Field, S. J. Redman, "Electrotonic profiles of interneurones in stratum pyramidale of the CA1 region of rat hippocampus." J. of Neurophys. 71, pp. 1948-1958, 1994.
86. J. G. Elias, D. P. M. Northmore, and W. Westerman, "An analog memory device for spiking silicon neurons." Neural Computation 9, pp. 419-440, 1997.
87. M. J. Rose, J. Hajto, P. G. Lecomber, S. M. Gage, W. K. Choi, A. J. Snell and A. E. Owen, "Amorphous silicon analogue memory devices." J. of Non-Crystalline Solids 115, pp. 168-170, 1989. 1986.
88. A. J. Holmes, R. A. G. Gibson, J. Hajto, A. F. Murray, A. E. Owen, M. J. Rose and A. J. Snell, "Use of a-Si:H memory devices for non-volatile weight storage in artificial neural networks." J. of Non-Crystalline Solids 164-166, pp. 817-820, 1993.
89. J. N. Barrett and W. E. Crill, "Specific membrane properties of cat motoneurones." J. of Physiol. 239, pp. 301-324, 1974.
90. W. Rall, The Theoretical Foundation of Dendritic Function: Selected Papers of Wilfrid Rall with Commentaries (I. Segev, J. Rinzel, and G. M. Shepherd, eds.). The MIT Press: Cambridge, MA, 1995.
91. K. L. Magleby, "Short-term changes in synaptic efficacy," in Synaptic Function (G. M. Edelman, W. E. Gall, W. M. Cowan, eds.). John Wiley and Sons, 1987, pp. 21-56.
92. J. J. B. Jack, D. Noble, and R. W. Tsien, Electrical Current Flow in Excitable Cells, 2nd ed. Clarendon Press: Oxford, 1975.
93. J. D. Clements and S. J. Redman, "Cable properties of cat spinal motoneurons measured by combining voltage clamp, current clamp and intracellular staining." J. of Physiol. 409, pp. 63-87, 1989.
94. P. J. B. Hancock, L. S. Smith, and W. A. Phillips, "A biologically supported error-correcting learning rule." Neural Computation 3, pp. 201-212, 1991.