A scalable two-stage approach for a class of dimensionality reduction techniques

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2010, Pages 313-322

  Sun, Liang ^a   Ceran, Betul ^a   Ye, Jieping ^a  

^a ARIZONA STATE UNIVERSITY   (United States)

Author keywords

Dimensionality reduction; Generalized eigenvalue problem; Least squares; Regularization; Scalability

Indexed keywords

CANONICAL CORRELATION ANALYSIS; DATA DIMENSIONALITY; DATA MINING APPLICATIONS; DIMENSIONALITY REDUCTION; DIMENSIONALITY REDUCTION TECHNIQUES; EQUIVALENCE RELATIONSHIP; GENERALIZED EIGENVALUE PROBLEMS; HIGH DIMENSIONAL DATA; HYPERGRAPH; LARGE SIZES; LEAST SQUARE; LEAST-SQUARES FORMULATION; LINEAR DISCRIMINANT ANALYSIS; ORTHONORMAL; PARTIAL LEAST SQUARES; REAL WORLD DATA; REGULARIZATION; SAMPLE SIZES; SPECTRAL LEARNING; TWO STAGE;

DATA MINING; DISCRIMINANT ANALYSIS; EIGENVALUES AND EIGENFUNCTIONS; SCALABILITY; SWITCHING SYSTEMS;

DATA REDUCTION;

EID: 77956202761     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1835804.1835846     Document Type: Conference Paper

References (34)

1. S. Agarwal, K. Branson, and S. Belongie. Higher order learning with graphs. In Proceedings of the 23rd International Conference on Machine Learning (ICML), pages 17-24, 2006.
2. J. Arenas-Garcia and G. Camps-Valls. Efficient kernel orthonormalized PLS for remote sensing applications. IEEE Transactions on Geoscience and Remote Sensing, 46(10):2872-2881, 2008.
3. A. Asuncion and D. J. Newman. UCI machine learning repository, 2007.
4. F. R. Bach and M. I. Jordan. Kernel independent component analysis. Journal of Machine Learning Research, 3:1-48, 2003.
5. R. E. Bellman. Adaptive Control Processes: A Guided Tour. Princeton Universsity Press, Princeton, NJ, 1961.
6. C. M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, NY, 2006.
7. M. R. Boutell, J. Luo, X. Shen, and C. M. Brown. Learning multi-label scene classification. Pattern Recognition, 37(9):1757-1771, 2004.
8. D. Cai. Spectral Regression: A Regression Framework for Efficient Regularized Subspace Learning. PhD thesis, Department of Computer Science, University of Illinois at Urbana-Champaign, 2009.
9. D. Cai, X. He, and J. Han. SRDA: An efficient algorithm for large-scale discriminant analysis. IEEE Transactions on Knowledge and Data Engineering, 20(1):1-12, 2008.
10. A. Elisseeff and J. Weston. A kernel method for multi-labelled classification. In Advances in Neural Information Processing Systems 13 (NIPS), pages 681-687, 2001.
11. K. Fukunaga. Introduction to Statistical Pattern Recognition. Academic Press, New York, NY, 1990.
12. G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins Press, Baltimore, MD, 3rd edition, 1996.
13. D. R. Hardoon, S. R. Szedmak, and J. R. Shawe-taylor. Canonical correlation analysis: An overview with application to learning methods. Neural Computation, 16(12):2639-2664, 2004.
14. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York, NY, 2nd edition, 2009.
15. H. Hotelling. Relations between two sets of variables. Biometrika, 28:312-377, 1936.
16. P. Howland and H. Park. Two-stage methods for linear discriminant analysis: Equivalent results at a lower cost. Technical report, Georgia Institute of Technology, 2009.
17. K. Lang. Newsweeder: Learning to filter netnews. In Proceedings of the 12th International Conference on Machine Learning (ICML), pages 331-339, 1995.
18. D. Lewis, Y. Yang, T. Rose, and F. Li. Rcv1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361-397, 2004.
19. C. C. Paige and M. A. Saunders. Algorithm 583: LSQR: Sparse linear equations and least squares problems. ACM Transactions on Mathematical Software, 8(2):195-209, 1982.
20. C. C. Paige and M. A. Saunders. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software, 8(1):43-71, 1982.
21. R. Rosipal and N. Krämer. Overview and recent advances in partial least squares. In Subspace, Latent Structure and Feature Selection Techniques, Lecture Notes in Computer Science, pages 34-51, 2006.
22. Y. Saad. Numerical Methods for Large Eigenvalue Problems. Halsted Press, New York, NY, 1992.
23. B. Schölkopf and A. J. Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge, MA, 2002.
24. L. Sun, S. Ji, and J. Ye. Hypergraph spectral learning for multi-label classification. In Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD), pages 668-676, 2008.
25. L. Sun, S. Ji, and J. Ye. A least squares formulation for canonical correlation analysis. In Proceedings of the 25th International Conference on Machine Learning (ICML), pages 1024-1031, 2008.
26. L. Sun, S. Ji, and J. Ye. A least squares formulation for a class of generalized eigenvalue problems in machine learning. In Proceedings of the 26th International Conference on Machine Learning (ICML), pages 977-984, 2009.
27. L. Sun, S. Ji, S. Yu, and J. Ye. On the equivalence between canonical correlation analysis and orthonormalized partial least squares. In Proceedings of the 21st International Jont Conference on Artifical Intelligence (IJCAI), pages 1230-1235, 2009.
28. D.S. Watkins. Fundamentals of matrix computations. John Wiley & Sons, Inc., New York, NY, 1991.
29. S. Wold and et al. Chemometrics, mathematics and statistics in chemistry. Reidel Publishing Company, Dordrecht, Holland, 1984.
30. K. Worsley, J.-B. Poline, K. J. Friston, and A.C. Evans. Characterizing the response of PET and fMRI data using multivariate linear models. Neuroimage, 6(4):305-319, 1997.
31. J. Ye. Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems. Journal of Machine Learning Research, 6:483-502, 2005.
32. J. Ye and Q. Li. A two-stage linear discriminant analysis via QR-decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27:929-941, 2005.
33. Z. Zhang, G. Dai, and M. Jordan. A flexible and efficient algorithm for regularized fisher discriminant analysis. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), pages 632-647. Springer, 2009.
34. D. Zhou, J. Huang, and B. Schölkopf. Learning with hypergraphs: Clustering, classification, and embedding. In Advances in Neural Information Processing Systems 18 (NIPS), pages 1601-1608, 2006.