Statistical Analysis of the Single-Layer Backpropagation Algorithm: Part I-Mean Weight Behavior

IEEE Transactions on Signal Processing

Volumn 41, Issue 2, 1993, Pages 573-582

  Feintuch, Paul L ^c   Shynk, John J ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c RAYTHEON COMPANY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DIFFERENTIABLE NONLINEARITY; GAUSSIAN INPUT SIGNALS; GRADIENT DESCENT METHOD; HYPERPLANE DECISION BOUNDARY; MEAN-SQUARE ERROR MINIMIZATION; NONLINEAR SYSTEM IDENTIFICATION; SINGLE-LAYER BACKPROPAGATION ALGORITHM;

ALGORITHMS; DECISION THEORY; VECTORS;

LEARNING SYSTEMS;

EID: 0027542458     PISSN: 1053587X     EISSN: 19410476     Source Type: Journal    
DOI: 10.1109/78.193199     Document Type: Article

References (14)

1. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Exploration in the Microstructure of Cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: M.I.T. Press, 1986, pp. 318-362.
2. R. P. Lippmann, “An introduction to computing with neural nets,” IEEEASSP Mag., vol. 4, pp. 4-22, Apr. 1987.
3. B. Widrow and M. A. Lehr, “30 years of adaptive neural networks: Perceptron, madaline, and backpropagation,” Proc. IEEE, vol. 78, pp. 1415-1442, Sept. 1990.
4. B. Widrow and S. D. Stearns, Adaptive Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1985.
5. J.J. Shynk and S. Roy, “Convergence properties and stationary points of a perceptron learning algorithm,” Proc. IEEE, vol. 78, pp. 1599-1604, Oct. 1990.
6. J. J. Shynk, “Performance surfaces of a single-layer perceptron,” IEEE Trans. Neural Networks, vol. 1, pp. 268-274, Sept. 1990.
7. J. J. Shynk and N. J. Bershad, “Steady-state analysis of a single-layer perceptron based on a system identification model with bias terms,” IEEE Trans. Circuits Syst., vol. 38, pp. 1030-1042, Sept. 1991.
8. N. J. Bershad, “On error saturation nonlinearities in LMS adaptation,” IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-36, pp. 440-452, Apr. 1988.
9. N. J. Bershad, “Nonlinear quantization effects in the LMS and block LMS adaptive algorithms-a comparison,” IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, pp. 1504-1512, Oct. 1989.
10. N. J. Bershad, “On weight update saturation nonlinearities in LMS adaptation,” IEEE Trans. Acoust., Speech, Signal Processing, vol.38, pp. 623-630, Apr. 1990.
11. R. Price, “A useful theorem for nonlinear devices having Gaussian inputs,” IRE Trans. Inform. Theory, vol. IT-4, pp. 69-72, June 1958.
12. R. F. Pawula, “A modified version of Price's theorem,” IEEE Trans. Inform. Theory, vol. IT-13, pp. 285-288, Apr. 1967.
13. J. J. Bussgang, “Cross-correlation functions of amplitude-distorted Gaussian signals,” Tech. Rep. 216, Res. Lab. Electron., M.I.T., Cambridge, MA, Mar. 1952.
14. H. B. Dwight, Tables of Integrals and Other Mathematical Data, 3rd ed. New York: MacMillan, 1960.