A Study of Parallel Perturbative Gradient Descent

NIPS 1994: Proceedings of the 7th International Conference on Neural Information Processing Systems

Volumn , Issue , 1994, Pages 803-810

  Lippe, D ^a   Alspector, J ^a  

^a TELCORDIA TECHNOLOGIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANALOG VLSI; FUNCTION PARAMETERS; GRADIENT-DESCENT; LEARNING METHODS; NEURAL-NETWORKS; NEW RESULTS;

EID: 0002702540     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. Alspector, J., Gannett, J. W., Haber, S., Parker, M. B., and Chu, R. (1991). A VLSI-efficient technique for generating multiple uncorrected noise sources and its application to stochastic neural networks. IEEE Transactions on Circuits and Systems, 38:109-123.
2. Alspector, J., Meir, R., Yuhas, B., Jayakumar, A., and Lippe, D. (1993). A parallel gradient descent method for learning in analog VLSI neural networks. In Hanson, S. J., Cowan, J. D., and Giles, C. L., editors, Advances in Neural Information Processing Systems S, pages 836-844, San Mateo, California. Morgan Kaufmann Publishers.
3. Biggs, N. L. (1989). Discrete Math. Oxford University Press.
4. Brown, T. X., Tran, M. D., Duong, T., and Thakoor, A. P. (1992). Cascaded VLSI neural network chips: Hardware learning for pattern recognition and classification. Simulation, 58 (5):340-347.
5. Cauwenberghs, G. (1993). A fast stochastic error-descent algorithm for supervised learning and optimization. In Hanson, S. J., Cowan, J. D., and Giles, C. L., editors, Advances in Neural Information Processing Systems 5, pages 244-261, San Mateo, California. Morgan Kaufmann Publishers.
6. Cauwenberghs, G. (1994). Analog VLSI Autonomous Systems for Learning and Optimization. PhD thesis, California Institute of Technology-
7. Flower, B. and Jabri, M. (1993). Summed weight neuron perturbation: An o (n) improvement over weight perturbation. In Hanson, S. J., Cowan, J. D., and Giles, C. L., editors, Advances in Neural Information Processing Systems 5, pages 212-219, San Mateo, California. Morgan Kaufmann Publishers.
8. Jabri, M. and Flower, B. (1991). Weight perturbation: An optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks. In Neural Computation 3, pages 546-565.
9. Kirk, D., Kerns, D., Fleischer, K., and Barr, A. (1993). Analog VLSI implementation of gradient descent. In Hanson, S. J., Cowan, J. D., and Giles, C. L., editors, Advances in Neural Information Processing Systems 5, pages 789-796, San Mateo, California. Morgan Kaufmann Publishers.
10. Kushner, H. and Clark, D. (1978). Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer-Verlag, New York.
11. Lippe, D. A. (1994). Parallel, perturbative gradient descent methods for learning in analog VLSI neural networks. Master's thesis, Massachusetts Institute of Technology.
12. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning internal representations by error propogation. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, page 318. MIT Press, Cambridge, MA.
13. Spall, J. C. (1992). Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control, 37 (3):332-341.