Robust Principal Component Analysis by Self-Organizing Rules Based on Statistical Physics Approach

IEEE Transactions on Neural Networks

Volumn 6, Issue 1, 1995, Pages 131-143

  Xu, Lei ^a   Yuille, Alan L ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BINARY SEQUENCES; DECISION THEORY; DESCRIBING FUNCTIONS; GIBBS FREE ENERGY; NEURAL NETWORKS; ROBUSTNESS (CONTROL SYSTEMS); STATISTICAL METHODS; VECTORS;

ENERGY FUNCTIONS; ROBUST PRINCIPAL COMPONENT ANALYSIS; STATISTICAL PHYSICS;

LEARNING ALGORITHMS;

EID: 0029184173     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.363442     Document Type: Article

References (43)

1. P. Baldi and K. Hornik, “Neural networks and principal component analysis: Learning from examples without local minima,” Neural Networks, vol. 2, pp. 52-58, 1989.
2. “Back-propagation and unsupervised learning in linear networks,” in Y. Chauvin and D. E. Rumelhart, Eds., Back-Propagation: Theory, Architecture, and Applications. Hillsdale, NJ: Erlbaum Associates, 1991.
3. H. G. Barrow, “Learning receptive fields,” in Proc. 1987 IEEE Ist Annu. Conf. Neural Networks, vol. 4, 1987, pp. 115-121.
4. W. Bialek, D. L. Ruderman, and A. Zee, “Optimal sampling of natural images: A design principle for the visual system,” in Advances in Neural Information Processing System 3, R. P. Lippmann, J. E. Moody, and D. S. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, 1991, pp. 363-369.
5. H. Bourlard and Y. Kamp, “Auto-association by multilayer perceptrons and singular value decomposition,” Biol. Cybern., vol. 59, pp. 291-294, 1988.
6. R. Brockett, “Dynamical systems that learn subspaces,” in Mathematical System Theory: The Influence of R. E. Kalman. Berlin: Springer-Verlag, 1991, pp. 410-420.
7. Y. Chauvin, “Principal component analysis by gradient descent on a constrained linear Hebbian cell,” in Proc. IEEE Int. Conf. Neural Networks, Vol. 1 (Washington, DC). New York: IEEE Press, 1989, pp. 373-380.
8. G. W. Cottrell, P. W. Munro, and D. Zipser, “Image compression by back-propagation: A demonstration of extentional programming,” Tech. Rep. 8702, Inst. Sci., UCSD, 1987.
9. F. Critchley, “Influence in principal components analysis,” Biometrika, vol. 72, pp. 627-636, 1985.
10. S. J. Devlin, R. Gnanadesikan, and J. R. Ketternrig, “Robust estimation of dispersion matrices and principal components,” J. Amer. Statist. Ass., vol. 76, pp. 354-362, 1981.
11. P. Foldiak, “Adaptive network for optimal linear feature extraction,” in Proc. IEEE Int. Conf. Neural Networks, Vol. I (Washington, DC). New York: IEEE Press, 1989, 401-405.
12. D. Geiger and A. L. Yuille, “A common framework for image segmen- Int. J. Vision, vol. 6, 227-233, 1991.
13. K. Hornik and C. M. Kuan, “Convergence analysis of local feature extraction Neural Networks, vol. 5, 229-260, 1992.
14. P. J. Huber, Robust Statistics. New York: Wiley, 1981.
15. R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive mixtures of local experts,” Neural Computation, vol. 3, pp. 79-87, 1991.
16. D. M. Kammen and A. L. Yuille, “Spontaneous symmetry-breaking energy functions and the emergence of orientation selective cortical cells,” Biol. Cybern., vol. 59, 23-31, 1988.
17. S. Y. Kung, “Adaptive principal component analysis via a new neural learning network,” in Proc. 1990 IEEE Int. Symp. Circuits Syst. (New Orleans, LA). New York: IEEE Press, 1990, pp. 138-140.
18. E. Linsker, “From basic network principles to neural architecture,” Proc. Nat. Acad. Sci. USA, vol. 83, pp. 7508-7512, 8390-8394, 1986.
19. “Self-organization in a perceptual network,” IEEE Computer, pp. 105-117, Mar. 1988.
20. “Design a sensory processing system: What can be learned from principal components analysis?,” in Proc. Int. Joint Conf Neural Networks, Vol. II, (Washington, DC). 1990, pp. 291-297.
21. D. J. C. Mackey and K. D. Miller, “Analysis of Linsker's simulation of Hebbian rules,” Neural Computation, vol. 2, 173-187, 1990.
22. E. Oja, “A simplified neuron model as a principal component analyzer,” J. Math. Biol., vol. 16, pp. 267-273, 1982.
23. Subspace Methods of Pattern Recognition. Letch worth, England: Research Studies Press, 1983.
24. E. Oja and J. Karhunen, “On stochastic approximation of eigenvectors and eigenvalues of the expectation of a random matrix,” J. Math. Anal. Appl., vol. 106, pp. 69-84, 1985.
25. E. Oja, “Neural networks, principal components, and subspaces,” Int. J. Neural Syst., vol. 1, pp. 61-68, 1989.
26. E. Oja, H. Ogawa, and J. Wangviwattana, “Learning in nonlinear constrained Hebbian networks,” in Proc. ICANN-91, Helsinki, Finland, 1991, 385-390.
27. E. Oja, “Principal components, minor components, and linear neural networks,” Neural Networks, vol. 5, pp. 927-935, 1992.
28. J. Rubner and P. Tavan, “A self-organizing network for principal- Lett., vol. 10, 693-689, 1989.
29. F. Palmieri and J. Zhu, “A comparison of two eigen-networks,” in Proc. Int. Joint Conf. Neural Networks, Vol. II, (Seattle, WA). 1991, pp. 193-199.
30. G. Parisi, Statistical Field Theory. Reading, MA: Addison-Wesley, 1988.
31. R. A. Redner and H. F. Walker, “Mixture densities, maximum likelihood, and the EM SIAM Rev., vol. 195-239, 1984.
32. J. Rubner and K. Schulten, “Development of feature detectors by self- Biol. vol. 62, 193-199, 1990.
33. F. H. Ruymagaart, “A robust principal component analysis,” J. Multivariate Anal., vol. 1981.
34. T. D. Sanger, “Optimal unsupervised learning in a single-layer linear feed forward neural network,” Neural Networks, vol. 2, pp. 459-473, 1989.
35. R. Williams, “Feature discovery through error-correcting learning,” Tech. 8501, Inst. Sci., UCSD, 1985.
36. L. Xu, “Least MSE reconstruction for self-organization: (II) Further theoretical and experimental studies on one layer nets,” in Proc. Int. Joint Neural Networks, Nov. 1991, 2368-2373.
37. L. Xu, E. Oja, and C. Y. Suen, “Modified Hebbian learning for curve and surface fitting,” Neural Networks, vol. 5, pp. 441-457, 1992.
38. L. Xu, A. Krzyzak, and E. Oja, “Neural nets for dual subspace pattern recognition,” Int. J. Neural Syst., vol. 2, pp. 169-187, 1991.
39. L. Xu, “Least mean square error reconstruction for self-organizing neural nets,” Neural Networks, vol. 6, pp. 627-648, 1993.
40. A. L. Yuille, D. M. Kammen, and D. S. Cohen, “Quadrature and the development of orientation selective cortical cells by Hebb rules,” Biol. vol. 60, 183-194, 1989.
41. A. L. Yuille, D. Geiger, and H. H. Bulthoff, “Stereo integration, mean field and vol. 2, 423-442, 1991.
42. A. L. Yuille, “Generalized deformable models, statistical physics, and Neural vol. 2, 1-24, 1990.
43. A. L. Yuille, T. Yang, and D. Geiger, “Robust statistics, transparency and correspondence,” Harvard Robotics Lab., Tech. Rep. 90-7, 1990.