Limitations of nonlinear PCA as performed with generic neural networks

IEEE Transactions on Neural Networks

Volumn 9, Issue 1, 1998, Pages 165-173

  Malthouse, Edward C ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

Author keywords

Data compression; Feature extraction; Nonlinear principal components analysis; Principal components; Principal curves; Principal surfaces

Indexed keywords

APPROXIMATION THEORY; DATA COMPRESSION; ERROR ANALYSIS; FEATURE EXTRACTION; FUNCTIONS; LEARNING SYSTEMS; MATHEMATICAL MODELS;

GENERIC NEURAL NETWORKS; NONLINEAR PRINCIPAL COMPONENTS ANALYSIS (NLPCA);

FEEDFORWARD NEURAL NETWORKS;

EID: 0031646493     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.655038     Document Type: Article

References (26)

1. B. Baldi and K. Hornik, "Neural networks and principal component analysis: Learning from examples without local minima," Neural Networks, vol. 2, pp. 53-58, 1989.
2. D. J. Bartholomew, Latent Variable Models and Factor Analysis. London: Charles Griffin & Company LTD, 1987.
3. C. Bishop, M. Svensen, and C. Williams, "GTM: The generative topographic mapping," Neural Computa., to be published.
4. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Contr. Signals Syst., vol. 2, pp. 303-314, 1989.
5. D. DeMers and G. Cottrell, "Nonlinear dimensionality reduction," Neural Inform. Processing Syst., vol. 5, pp. 580-587, 1993.
6. D. Dong and T. McAvoy, "Nonlinear principal component analysis - Based on principal curves and neural networks," Computers and Chem. Eng., vol. 20, no. 1, pp. 65-78, 1996.
7. D. Fotheringhame and R. Baddeley, "Nonlinear principal components analysis of neuronal spike train data," Dept. Physiology, Univ. Oxford, U.K., Tech. Rep., 1997.
8. J. H. Friedman, "Exploratory projection pursuit," J. Amer. Statist. Assoc., vol. 82, no. 397, pp. 249-267, 1987.
9. C. Fyfe and R. Baddeley, "Nonlinear data structure extraction using simple Hebbian networks," Biol. Cybern., vol. 72, pp. 533-541, 1995.
10. T. Hastie, "Principal curves and surfaces," Ph.D. dissertation, Stanford Univ., CA, Nov. 1984.
11. T. Hastie and W. Stuetzle, "Principal curves," J. Amer. Statist. Assoc., vol. 84, no. 406, pp. 502-516, 1989.
12. T. Hastie and R. Tibshirani, Generalized Additive Models. London: Chapman and Hall, 1990.
13. J. Karhunen and J. Joutsensalo, "Representation and separation of signals using nonlinear PCA type learning," Neural Networks, vol. 7, pp. 113-127, 1994.
14. _, "Generalizations of PCA, optimization problems, and neural networks," Neural Networks, vol. 8, pp. 549-562, 1995.
15. M. A. Kramer, "Nonlinear principal component analysis using autoassociative neural networks," AIChE J., vol. 37, no. 2, pp. 233-243, 1991.
16. M. LeBlanc and R. Tibshirani, "Adaptive principal surfaces," J. Amer. Statist. Assoc., vol. 89, no. 425, pp. 53-64, 1994.
17. D. Liu and J. Nocedal, "On the limited memory {BFGS} method for large scale optimization," Math. Prog., vol. 45, pp. 503-528, 1989.
18. E. Malthouse, R. Mah, and A. Tamhane, "Some theoretical results on nonlinear principal components analysis," in Proc. Amer. Contr. Conf., June 1995.
19. K. V. Mardia, J. T. Kent, and J. M. Bibby, Multivariate Analysis. London: Academic, 1979.
20. T. Sanger, "Optimal unsupervised learning in a single-layer linear feedforward neural network," Neural Networks, vol. 2, pp. 459-473, 1989.
21. _, "Optimal hidden units for two-layer nonlinear feedforward neural networks," Int. J. Pattern Recognition and Artificial Intell., vol. 5, no. 4, pp. 545-561, 1991.
22. E. Saund, "Dimensionality-reduction using connectionist networks," IEEE Trans. Pattern Anal. Machine Intell., vol. 11, no. 3, pp. 304-314, 1989.
23. D. J. Struik, Lectures on Classical Differential Geometry, 2nd ed. New York: Dover, 1961.
24. S. Tan and M. Mavrovouniotis, "Reducing data dimensionality through optimizing neural-network inputs," AIChE J., vol. 41, no. 6, pp. 1471-1480, 1995.
25. J. A. Thorpe, Elementary Topics in Differential Geometry. New York: Springer-Verlag, 1979.
26. L. Xu, E. Oja, and C. Suen, "Modified Hebbian learning for curve and surface fitting," Neural Networks, vol. 5, pp. 441-457, 1992.