Regression Modeling in Back-propagation and Projection Pursuit Learning

IEEE Transactions on Neural Networks

Volumn 5, Issue 3, 1994, Pages 342-353

  Hwang, Jeng Neng ^a   Lay, Shyh Rong ^a   Maechler, Martin ^b   Martin, R Douglas ^a   Schimert, James ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

^b ETH ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; ITERATIVE METHODS; MATHEMATICAL MODELS; NEURAL NETWORKS; OPTIMIZATION; POLYNOMIALS; REGRESSION ANALYSIS; STATISTICAL METHODS; VECTORS;

BACK PROPAGATION LEARNING; GAUSS NEWTON OPTIMIZATION ALGORITHMS; PARSIMONY; PROJECTION PRUSUIT LEARNING; REGRESSION MODELING;

LEARNING SYSTEMS;

EID: 0028428443     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.286906     Document Type: Article

References (34)

1. H. Akaike, “Information theory and an extension of the maximum likelihood principle,” 2nd Int' I Symposium on Information Theory, Akademia Kiado, Budapest, pp. 267-281, 1973.
2. A. R. Barron, “Predicted squared error: A criterion for automatic model selection,” Self-Organizing Methods in Modeling, S. Farlow and Marcel Dekker, Eds. 1984, chap. 4.
3. G. Cybenko. “Approximation by superpositions of a sigmoidal function,” University of Illinois, Dept. of Elect, and Computer Engr., Technical Report 856, 1988.
4. P. J. Davis, Interpolation and Approximation, New York: Dover Publication Inc., 1975, chap. 6.
5. D. L. Donoho and I. M. Johnstone, “Projection-based approximation and a duality with kernel methods,” The Annals of Statistics, vol. 17, no. 1, pp. 58-106, 1989.
6. J. H. Friedman and W. Stuetzle, “Projection pursuit regression,” Journal of the American Statistical Association, vol. 76, no. 376, pp. 817-823, 1981.
7. J. H. Friedman, “A variable span smoother,” Department of Statistics, Stanford University, Technical Report no. 5, November 1984.
8. J. H. Friedman, “Classification and multiple regression through projection pursuit,” Department of Statistics, Stanford University, Technical Report no. 12, January 1985.
9. R. Hecht-Nielsen, “Theory of Backpropagation neural networks,” in Proc. Int'I Joint Conf on Neural Networks. pp. 593-608, 1989.
10. K. Hornik, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, no. 5, pp. 359-366, 1989.
11. T. Hrycej, “A modular architecture for efficient learning,” in Proc. Int'l Joint Conf. on Neural Networks, pp. I557-567, 1990.
12. P. J. Huber, “Projection pursuit,” The Annals of Statistics, vol. 13, no. 2, 435-475. 1985.
13. J. N. Hwang, J. A. Vlontzos, and S. Y. Kung, “A systolic neural network architecture for hidden Markov model,” IEEE Trans, on Acoustics, Speech, and Signal Processing, vol. 37. no. 12, pp. 1967-1979, 1989.
14. J. N. Hwang and P. S. Lewis, “From nonlinear optimization to neural network learning,” in Proc. 24th Asilomar Conf. on Signals. Systems. & Computers, Pacific Grove, CA, November 1990, pp. 985-989.
15. J. N. Hwang, H. Li, D. Martin, and J. Schimert, “The learning parsimony of projection pursuit and Backpropagation networks,” in 25th Asilomar Conf. on Signals, Systems. & Computers. Pacific Grove, CA, November 1991, pp. 491-495.
16. J. N. Hwang, H. Li, M. Maechler. D. Martin, and J. Schimert, “A comparison of projection pursuit and neural network regression modeling,” in Neural Information Processing Systems IV. Denver, CO. November 1991, pp. 1159-1166.
17. J N. Hwang, H. Li, M. Maechler, D. Martin, and J. Schimert, “Projection pursuit learning network for regression,” Engineering Applic. Artif. Intell., New York: Pergamon Press Ltd., vol. 5, no. 3, pp. 193-204, 1992.
18. P. Hall, “On polynomial-based projection indices for exploratory projection pursuit,” The Annals of Statistics, vol. 17, no. 2, pp. 589-605, 1989.
19. L. K. Jones, “On a conjecture of Huber concerning the convergence of projection pursuit regression,” The Annals of Statistics, vol. 15, no. 2, pp. 880-882, 1987.
20. R. D. Jones, Y. C. Lee, C. W. Barnes, G. W. Flake, K. Lee, P. S. Lewis, and S. Qian, “Function approximation and time series prediction with neural networks,” in Proc. Int'l Joint Conf on Neural Networks, pp. I 649-665, 1990.
21. S. Kollias and D. Anastassiou, “An adaptive least squares algorithm for the efficient training of artificial neural networks,” IEEE Trans. Circuits and Systems, vol. 36, no. 8, pp. 1092-1101, 1989.
22. B. Kosko, Neural Networks and Fuzzy Systems: A Dynamical Systems Approach to Machine Intelligence. Englewood Cliffs, NJ: Prentice Hall, 1991.
23. J. B. Kruskal, “Nonmetric multidimensional scaling: a numerical method,” Psychometrika vol. 29, pp. 115-129. 1964.
24. M. Maechler, D. Martin. J. Schimert, M. Csoppenszky and J. N. Hwang, “Projection pursuit learning networks for regression.” in Proc. 2nd Int'l Conf. Tools for Al, Washington D.C., November 1990, pp. 350-358.
25. J. Moody and C. J. Darken, “Fast learning in networks of locally tuned processing units,” Neural Computations, vol. 1, no. 1, pp. 281-294, 1989.
26. D. B. Parker, Learning logic. Center for Computational Research in Economics and Management Science. MIT, Technical Report no. 47, April 1985.
27. J. Rissanen, “A universal prior for integers and estimation by minimum description length,” Ann. of Stat., vol. 11, no. 2, pp. 416-431, 1983.
28. H. Robbins and S. Monro, “A stochastic approximation method.” Annals of Mathematical Statistics, vol. 22, pp. 400-407, 1951.
29. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. “Learning internal representation by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Volume 1: Foundations D. E. Rumelhart and J. L. McClelland, Eds., Cambridge, MA: MIT Press, 1986.
30. D. F. Shannon, “Recent advances in numerical techniques for large-scale optimization,” in Neural Networks for Robotics and Control, E. W. T. Miller, R. Sutton, and P. J. Werbos, Eds. Cambridge. MA: MIT Press 1990.
31. Statistical Science Inc., S-Plus Users Manual, (Version 3.0), Seattle, WA.
32. R. L. Watrous, “Learning algorithms for connectionist networks: Applied gradient methods of nonlinear optimization.” in Proc. 1987 IEEE Int. Conf. Neural Networks: vol. II San Diego, CA. June 1987, pp. 619-627.
33. S. M. Weiss and C. A, Kulikowski, Computer Systems that Learn. San Mateo, CA: Morgan Kauffman Inc., 1992.
34. H. White, “Learning in artificial networks: A statistical perspective.” Neural Computation, vol. 1, no. 4, pp. 425-169, 1989.