Learning curves for stochastic gradient descent in linear feedforward networks

Advances in Neural Information Processing Systems

Volumn , Issue , 2004, Pages

  Werfel, Justin ^a   Xie, Xiaohui ^b   Sebastian Seung, H ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b PRINCETON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FEED-FORWARD NETWORK; GRADIENT DESCENT; LEARNING CURVES; LEARNING METHODS; ONLINE TRAINING; STOCHASTIC GRADIENT DESCENT; STOCHASTIC METHODS; WEIGHT PERTURBATION;

LEARNING ALGORITHMS;

EID: 0346554799     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (9)

1. Widrow, B. & Lehr, M. A. 30 years of adaptive neural networks: Perceptron, Madaline, and backpropagation. Proc. IEEE 78(9):1415-1442, 1990.
2. Jabri, M. & Flower, B. Weight perturbation: An optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayered networks. IEEE Transactions on Neural Networks 3(1):154-157, 1992.
3. Flower, B. & Jabri, M. Summed weight neuron perturbation: An O(n) improvement over weight perturbation. In Advances in Neural Information Processing Systems 5, San Mateo, CA: Morgan Kaufman Publishers: 212-219, 1993.
4. Cauwenberghs, G. A fast stochastic error-descent algorithm for supervised learning and optimization. In Advances in Neural Information Processing Systems 5, San Mateo, CA: Morgan Kaufman Publishers: 244-251, 1993.
5. Cauwenberghs, G. An analog VLSI recurrent neural network learning a continuous-time trajectory. IEEE Transactions on Neural Networks 7(2):346-361, 1996.
6. Fiete, I. Private communication.
7. Bartlett, P. & Baxter, J. Hebbian synaptic modifications in spiking neurons that learn. Technical report, November 27 1999.
8. Baldi, P. & Hornik, K. Learning in linear neural networks: A survey. IEEE Transactions on Neural Networks 6(4):837-858, 1995.
9. Williams, R.J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8:229-256, 1992.