Sparse variable PCA using geodesic steepest descent

IEEE Transactions on Signal Processing

Volumn 56, Issue 12, 2008, Pages 5823-5832

  Ulfarsson, Magnus O ^a,b   Solo, Victor ^c  

^a UNIVERSITY OF MICHIGAN   (United States)

^b UNIVERSITY OF ICELAND   (Iceland)

^c UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Geodesic; Geometric; L1; Lasso; Principal component analysis (PCA); Regularization; Sparse

Indexed keywords

GEODESIC; GEOMETRIC; L1; LASSO; PRINCIPAL COMPONENT ANALYSIS (PCA); REGULARIZATION; SPARSE;

ALGORITHMS; GEODESY; LOADING; MAGNETIC RESONANCE IMAGING; RESONANCE;

PRINCIPAL COMPONENT ANALYSIS;

EID: 77951130963     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2008.2006587     Document Type: Article

References (46)

1. I. Jolliffe, Principal Component Analysis, 2nd ed. New York: Springer, 2002.
2. J. Jackson, A Users Guide to Principal Components. New York: Wiley, 1991.
3. R. Preisendorfer, Principal Component Analysis in Meteorlogy and Oceangraphy. Amsterdam, The Netherlands: Elsevier, 1988.
4. D. Lawley, "A modified method of estimation in factor analysis and some large sample results", in Proc. Uppsala Symp. Psychological Factor Analysis, Uppsala, Sweden, 1953, pp. 35-42.
5. M. Tipping and C. Bishop, "Probabilistic principal component analysis", J. Royal Stat. Soc, Series B, vol. 61, no. 3, pp. 611-622, 1999.
6. C. Theobald, "An inequality with application to multivariate analysis", Biometrika, vol. 62, no. 2, pp. 461-466, 1975.
7. 2 norms with application to Bayesian estimation", IEEE Trans. Signal Process., vol. 42, no. 3, pp. 618-627, Mar. 1994.
8. D. Donoho and I. Johnstone, "Ideal spatial adaptation by wavelet shrinkage", Biometrika, vol. 81, no. 3, pp. 425-455, 1994.
9. R. Tibshirani, "Regression shrinkage and selection via the LASSO", J. Royal Stat. Soc., Series B, vol. 58, no. 1, pp. 267-288, 1996.
10. I. Jolliffe and M. Uddin, "A modified principal component technique based on the LASSO", J. Comput. Graphic. Stat., vol. 12, no. 3, pp. 531-547, 2003. (Pubitemid 37239916)
11. H. Zou, T. Hastie, and R. Tibshirani, "Sparse principal component analysis", J. Comput. Graphic. Stat., vol. 15, no. 2, pp. 265-286, 2006. (Pubitemid 43935978)
12. A. D'Aspremont, L. E. Ghaoui, M. Jordan, and G. Lanckriet, "A direct formulation for sparse PCA using semidefinite programming", SIAM Rev., to be published.
13. I. Johnstone and A. Lu, Sparse "Principal component analysis", Statistics Dept., Stanford Univ., Stanford, CA, Tech. Rep., 2004.
14. G. Schwartz, "Estimating the dimension of a model", Ann. Stat., vol. 6, no. 2, pp. 461-464, 1978.
15. M. Ulfarsson and V. Solo, "Sparse variable principal component analysis with application to fMRI", in Proc. IEEE Int. Symp. Biomedical Imaging (ISBI), Washington DC, 2007, pp. 460-463. (Pubitemid 350141125)
16. H. Schneeweiss and H. Mathes, "Factor analysis and principal components", J. Multivar. Anal., vol. 55, pp. 105-124, 1995.
17. H. Schneeweiss, "Factors and principal components in the near spherical case", Multivar. Behav. Res., vol. 32, no. 4, pp. 375-401, 1997. (Pubitemid 127341625)
18. T. Anderson, "Asymptotic theory for principal component analysis", Ann. J. Math. Stat., vol. 34, pp. 122-148, 1963.
19. G. Robinson, "That BLUP is a good thing: The estimation of random effects", Stat. Sci., vol. 6, no. 1, pp. 15-32, 1991.
20. D. Donoho, "De-noising by soft-thresholding", IEEE Trans. Inf. Theory, vol. 41, pp. 613-627, 1995.
21. L. Rudin, S. Osher, and E. Fatemi, "Nonlinear total variation based noise removal algorithms", Physica D, vol. 60, pp. 259-268, 1992.
22. C. Vogel and M. Oman, "Iterative methods for total variation denoising", SIAM J. Sci. Comp., vol. 17, no. 1, pp. 227-238, 1996. (Pubitemid 126621162)
23. B. Turlach, W. Venables, and S. Wright, "Simultaneous variable selection", Technometrics, vol. 47, no. 3, pp. 349-363, 2005. (Pubitemid 41164070)
24. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables", J. Roy. Stat. Soc. B, vol. 68, pp. 49-67, 2006. (Pubitemid 43415335)
25. V. Solo, "Selection of regularization parameters for total variation denoising", in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), Phoenix, AZ, 1999, vol. 3, pp. 1653-1655.
26. R. Stevenson, B. Schmitz, and E. Delp, "Discontinuity preserving regularization of inverse visual problems", IEEE Trans. Syst., Man, Cybern., vol. 24, no. 3, pp. 455-469, 1994.
27. S. Z. Li, "On discontinuity-adaptive smoothness priors in computer vision", IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 6, pp. 576-586, 1995.
28. D. Luenberger, "The gradient projection method along geodesic", Management Sci., Theory Series, vol. 18, no. 11, pp. 620-631, 1972.
29. D. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA: Athena Scientific, 1999.
30. W. Zangwill, Nonlinear Programming: A Unified Approach. Englewood Cliffs, NJ: Prentice-Hall, 1969.
31. A. Edelman, T. Aries, and S. Smith, "The geometry of algorithms with orthogonality constraints", SIAM J. Matrix Anal. Appl., vol. 20, no. 2, pp. 303-353, 1998. (Pubitemid 129333771)
32. L. Elden and H. Park, "A Procrustes problem on the Stiefel manifold", Numer. Math., vol. 82, no. 4, pp. 599-619, 1999.
33. S. Douglas, S. Amari, and S. Kung, "On gradient adaptation with unit norm constraints", IEEE Trans. Signal Process., vol. 48, no. 6, pp. 1843-1847, Jun. 2000.
34. A. Cichocki and P. Georgiev, "Blind source separation algorithms with matrix constraints", IEICE Trans. Fundamentals, vol. E86-A, no. 1, pp. 1-9, 2003.
35. S. Fiori, "A theory for learning by weight flow on Stiefel-Grassman manifold", Neural Comput., vol. 13, pp. 1625-1637, 2001.
36. Y. Nishimori, "Learning algorithm for ICA by geodesic flows and orthogonal groups", in Proc. Int. Joint Conf. Neural Networks (IJCNN), Washington DC, 1999, vol. 2, pp. 933-938. (Pubitemid 30593084)
37. J. Manton, "Optimization algorithms exploiting unitary constraints", IEEE Trans. Signal Process., vol. 50, no. 3, pp. 635-650, Mar. 2002. (Pubitemid 34295105)
38. C. Moler and C. Loan, "Nineteen dubious ways to compute the exponential of a matrix", SIAM Rev., vol. 20, no. 4, pp. 801-831, 1978.
39. G. Golub and C. V. Loan, Matrix Computation, 3rd ed. Baltimore, MD: The Johns Hopkins Univ. Press, 1996.
40. D. G. Luenberger, Introduction to Linear and Nonlinear Programming. New York: Addison-Wesley, 1973.
41. V. Solo and S. Ratcliffe, Rotated "Functional principal components analysis with regression", Univ. of New South Wales, Sydney, Australia, Tech. Rep., 2002.
42. R. Meyer, "Sufficient conditions for the convergence of monotonic mathematical programming algorithms", J. Comput. Syst. Sci., vol. 12, pp. 108-121, 1976.
43. K. Lange, "A gradient algorithm locally equivalent to the EM algorithm", J. Roy. Stat. Soc. Series B, vol. 57, no. 2, pp. 425-437, 1995.
44. C. Loader, Local Regression and Likelihood. New York: Springer, 1999.
45. H. Akaike, "A new look at the statistical model identification" , IEEE Trans. Autom. Control, vol. 19, no. 6, pp. 716-723, 1974.
46. A. Ostrowski, Solutions of Equations in Euclidian and Banach Spaces. New York: Academic, 1973.