An analog VLSI recurrent neural network learning a continuous-time trajectory

IEEE Transactions on Neural Networks

Volumn 7, Issue 2, 1996, Pages 346-361

  Cauwenberghs, Gert ^a,b  

^a Johns Hopkins University   (United States)

^b IEEE

Author keywords

[No Author keywords available]

Indexed keywords

CMOS INTEGRATED CIRCUITS; COMPUTATIONAL COMPLEXITY; ERRORS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; REAL TIME SYSTEMS; VLSI CIRCUITS;

INTEGRATED LEARNING SYSTEM; LEARNING NEURAL NETWORKS; REAL TIME ALGORITHMS; RECURRENT NEURAL NETWORK; STOCHASTIC PERTURBATIVE ALGORITHMS;

NEURAL NETWORKS;

EID: 0030106428     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.485671     Document Type: Article

References (47)

1. J. Alspector, B. Gupta, and R. B. Allen, "Performance of a stochastic learning microchip," in Advances in Neural Information Processing Systems, vol. 1. San Mateo, CA: Morgan Kaufman, 1989, pp. 748-760.
2. E. Vittoz and X. Arreguit, "CMOS integration of Herault-Jutten cells for separation of sources," in Analog VLSI Implementation of Neural Systems. Norwell, MA: Kluwer, 1989, pp. 57-83.
3. S. Bibyk and M. Ismall, "Issues in analog VLSI and MOS techniques for neural computing," in Analog VLSI Implementation of Neural Systems. Norwell, MA: Kluwer, 1989, pp. 103-133.
4. B. Hochet, V. Peiris, S. Abdot, and M. J. Declercq, "Implementation of a learning Kohonen neuron based on a new multilevel storage technique," IEEE J. Solid-State Circuits, vol. 26, no. 3, pp. 262-267, 1991.
5. M. H. Cohen and A. G. Andreou, "Current-mode subthreshold MOS implementation of the Jutten-Herault autoadaptive network," IEEE J. Solid-State Circuits, vol. 27, no. 5, pp. 714-727, 1992.
6. G. Cauwenberghs, C. F. Neugebauer, and A. Yariv, "Analysis and verification of an analog VLSI outer-product incremental learning system," IEEE Trans. Neural Networks, vol. 3, no. 3, pp. 488-497, 1992.
7. R. G. Benson and D. A. Kerns, "UV-activated conductances allow for multiple time scale learning," IEEE Trans. Neural Networks, vol. 4, no. 3, pp. 434-440, 1993.
8. J. Donald and L. Akers, "An adaptive neural processor node," IEEE Trans. Neural Networks, vol. 4, no. 3, pp. 413-426, 1993.
9. B. Linares-Barranco, B. Sanchez-Sinencio, A. Rodriguez-Vazquez, and J. L. Huertas, "A CMOS analog adaptive BAM with on-chip learning and weight refreshing," IEEE Trans. Neural Networks, vol. 4, no. 3, pp. 445-455, 1993.
10. H. P. Graf and L. D. Jackel, "Analog electronic neural network circuits," IEEE Circuits Devices Mag., vol. 5, no. 4, pp. 44-49, 1989.
11. C. A. Mead, Analog VLSI and Neural Systems. Reading, MA. Addison-Wesley, 1989.
12. A. G. Andreou, K. A. Boahen, P. O. Pouliquen, A. Pavasovic, R. E. Jenkins, and K. Strohbehn, "Current-mode subthreshold MOS circuits for analog VLSI neural systems," IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 205-213, 1991.
13. D. E. Rumelhart and J. L. McClelland, Eds., Parallel Distributed Processing, Explorations in the Microstructure of Cognition, vol. 1. Cambridge, MA: MIT Press, 1986.
14. M. J. S. Smith, "An analog integrated neural network capable of learning the Feigenbaum logistic map." IEEE Trans Circuits Syst., vol. 37, no. 6, pp. 841-844, 1990.
15. R. C. Frye, E. A. Rietman, and C. C. Wong, "Backpropagation learning and nonidealities in analog neural network hardware," IEEE Trans. Neural Networks, vol. 2, no. 1, pp. 110-117, 1991.
16. M. Jabri and B. Flower, "Weight perturbation: An optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayered networks," IEEE Trans. Neural Networks, vol. 3, no. 1, pp. 154-157, 1992.
17. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Networks, vol. 1, no. 1, pp. 4-27, 1990.
18. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Computa., vol. 1, no. 2, pp. 270-280, 1989.
19. B. A. Pearlmutter, "Learning state-space trajectories in recurrent neural networks," Neural Computa., vol. 1, no. 2, pp. 263-269, 1989.
20. N. B. Toomarian and I. Barben, "Learning a trajectory using adjoint functions and teacher forcing," Neural Networks, vol. 5, no. 3, pp. 473-484, 1992.
21. 3) time complexity learning algorithm for fully recurrent continually running networks," Neural Computa., vol. 4, no. 2, pp. 243-248, 1992.
22. G.-Z. Sun, H.-H. Chen, and Y.-C. Lee, "Green's function method for fast on-line learning algorithm of recurrent neural networks," in Advances in Neural Information Processing Systems, vol. 4. San Mateo, CA: Morgan Kaufman, 1992, pp. 333-340.
23. P. Baldi, "Gradient descent learning algorithms: A general dynamical systems perspective," IEEE Trans. Neural Networks, vol. 6, no. 1, pp. 182-195, Jan. 1995.
24. P. Mueller, J. Van Der Spiegel, D. Blackman, T. Chiu, T. Clare, C. Dunham, T. P. Hsieh, and M. Loinez, "Design and fabrication of VLSI components for a general purpose analog neural computer," in Analog VLSI Implementation of Neural Systems. Norwell, MA: Kluwer, 1989, pp. 135-169.
25. G. Cauwenberghs, "A fast stochastic error-descent algorithm for supervised learning and optimization," in Advances in Neural Information Processing Systems, vol. 5. San Mateo, CA: Morgan Kaufman, 1993, pp. 244-251.
26. H. Robins and S. Monro, "A stochastic approximation method," Annals Math Statist., vol. 22, pp. 400-407, 1951.
27. H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Cconstrained and Unconstrained Systems. New York: Springer-Verlag, 1978.
28. [281 M. A. Styblinski and T.-S. Tang, "Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing," Neural Networks, vol. 3, no. 4, pp. 467-483, 1990.
29. J. C. Spall, "Multivariate stochastic approximation using a simultaneous perturbation gradient approximation," IEEE Trans. Automat. Contr., vol. 37, no. 3, pp. 332-341, 1992.
30. A. Dembo and T. Kailath, "Model-free distributed learning," IEEE Trans. Neural Networks, vol. 1, no. 1, pp. 58-70, 1990.
31. D. Kirk, D. Kerns, K. Fleischer, and A. Barr, "Analog VLSI implementation of gradient descent," in Advances in Neural Information Processing Systems, vol. 5. San Mateo, CA. Morgan Kaufman, 1993, pp. 789-796.
32. J. Alspector, R. Meir, B. Yuhas, and A. Jayakumar, "A parallel gradient descent method for learning in analog VLSI neural networks," in Advances in Neural Information Processing Systems, vol. 5. San Mateo, CA: Morgan Kaufman, 1993, pp. 836-844.
33. B. Flower and M. Jabri, "Summed weight neuron perturbation: An O(n) improvement over weight perturbation," in Advances in Neural Information Processing Systems, vol. 5. San Mateo, CA: Morgan Kaufman, 1993, pp. 212-219.
34. G. Cauwenberghs, C. F. Neugebauer, A. Agranat, and A. Yariv, "Large scale optoelectronic integration of asynchronous analog neural networks," in Proc. Dig. Int. Neural Network Conf. (INNC-90 Paris), vol. 2. pp. 551-554, 1990.
35. S. Satyanarayana, Y. P. Tsividis, and H. P. Graf, "A reconfigurable VLSI neural network," IEEE J. Solid-Slate Circuits, vol. 27, no 1, pp. 67-81, 1992.
36. J. W. Fattaruso, S. Kiriaki, G. Warwar, and M. de Wit, "Self-calibration techniques for a second-order multibit sigma-delta modulator," in ISSCC Tech. Dig., vol. 36, pp. 228-229, 1993.
37. E. Säckinger and W. Guggenbuhl, "A high-swing, high-impedance MOS cascode circuit," IEEE J. Solid-State Circuits, vol. 25, no. 1, pp. 289-298, 1990.
38. Z. Czarnul, "Modification of the Banu-Tsividis continuous-time integrator structure," IEEE Trans. Circuits Syst., vol. CAS-33, no. 7, pp. 714-716, 1986.
39. L. Watts, D. A. Kerns, and R. F. Lyon, "Improved implementation of the silicon cochlea," IEEE J. Solid-State Circuits, vol. 27, no. 5, pp. 692-700, 1992.
40. S. Satyanarayana, Y. P. Tsividis, and H. P. Graf, "Analogue neural networks with distributed neurons," Electron. Lett., vol. 25, no. 5, pp. 302-303, 1989.
41. B. Sheu and C. Hu, "Switched-induced error voltage on a switched capacitor," IEEE J. Solid-State Circuits, vol. SSC-19, no. 4, pp. 519-525, 1984.
42. E. Vittoz and J. Fellrath, "CMOS analog integrated circuits based on weak inversion operation," IEEE J. Solid-State Circuits, vol. SSC-12, no. 3, pp. 224-231, 1977.
43. S. W. Golomb, Shift Register Sequences. San Francisco, CA: Holden-Day, 1967.
44. J. Alspector, J. W. Gannett, S. Haber, M. B. Parker, and R. Chu, "A VLSI-efficient technique for generating multiple uncorrelated noise sources and its application to stochastic neural networks," IEEE Trans. Circuits Syst., vol. 38, no. 1, pp. 109-123, 1991.
45. G. Cauwenberghs and A. Yariv, "Fault-tolerant dynamic multilevel storage in analog VLSI," IEEE Trans. Circuits Syst. II vol. 41, no. 12, pp. 827-829, 1994.
46. _, "Method and apparatus for monotonic algorithmic digital-to-analog and analog-to-digital conversion," U.S. Patent 5 258 759, 1993.
47. P. Baldi, "Learning in dynamical systems: Gradient descent, random descent, and modular approaches," California Institute Technol., JPL Tech. Rep., 1992.