Effective and efficient dimensionality reduction for large-scale and streaming data preprocessing

IEEE Transactions on Knowledge and Data Engineering

Volumn 18, Issue 3, 2006, Pages 320-332

  Yan, Jun ^a,f   Zhang, Benyu ^b   Liu, Ning ^b,c   Yan, Shuicheng ^a   Cheng, Qiansheng ^a,g   Fan, Weiguo ^d   Yang, Qiang ^e   Xi, Wensi ^d,h   Chen, Zheng ^b,i  

^a PEKING UNIVERSITY   (China)

^b MICROSOFT RESEARCH ASIA   (China)

^c TSINGHUA UNIVERSITY   (China)

^d VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

^e HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^f HMS   (United States)

^g Chinese Signal Processing Society   (China)

^h State University   (United States)

ⁱ MICROSOFT RESEARCH   (United States)

Author keywords

Feature extraction; Feature selection; Orthogonal centroid algorithm

Indexed keywords

DIMENSIONALITY REDUCTION; FEATURE SELECTION; ORTHOGONAL CENTROID ALGORITHM; STREAMING DATA CLASSIFICATION;

ALGORITHMS; DATA REDUCTION; FEATURE EXTRACTION; OPTIMIZATION; SET THEORY;

DATA PROCESSING;

EID: 31644433092     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2006.45     Document Type: Article

References (36)

1. M. Artae, M. Jogan, and A. Leonardis, "Incremental PCA for On-Line Visual Learning and Recognition," Proc. 16th Int'l Conf. Pattern Recognition, pp. 781-784, 2002.
2. A.L. Slum and P. Langley, "Selection of Relevant Features and Examples in Machine Learning," Artificial Intelligence, vol. 97, nos. 1-2, pp. 245-271, 1997.
3. M. Belkin and P. Niyogi, "Using Manifold Structure for Partially Labelled Classification," Proc. Conf. Advances in Neural Information Processing, pp. 929-936, 2002.
4. S.E. Brian and G. Dunn, Applied Multivariate Data Analysis. Edward Arnold, 2001.
5. K. Daphne and M. Sahami, "Toward Optimal Feature Selection," Proc. 13th Int'l Conf. Machine Learning, pp. 284-292, 1996.
6. W. Fan, M.D. Gordon, and P. Pathak, "Effective Profiling Of Consumer Information Retrieval Needs: A Unified Framework And Empirical Comparison," Decision Support Systems, vol. 40, pp. 213-233, 2004.
7. J.E. Gentle, Numerical Linear Algebra for Applications in Statistics. Springer-Verlag, 1998.
8. K. Hiraoka, K. Hidai, M. Hamahira, H. Mizoguchi, T. Mishima, and S. Yoshizawa, "Successive Learning of Linear Discriminant Analysis: Sanger-Type Algorithm," Proc. 14th Int'l Conf. Pattern Recognition, pp. 2664-2667, 2000.
9. R. Hoch, "Using IR Techniques for Text Classification in Document Analysis," Proc. 17th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 31-40, 1994.
10. P. Howland and H. Park, "Generalizing Discriminant Analysis Using the Generalized Singular Value Decomposition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, pp. 995-1006, 2004.
11. M. Jeon, H. Park, and J.B. Rosen, "Dimension Reduction Based on Centroids and Least Squares for Efficient Processing of Text Data," Technical Report MN TR 01-010, Univ. of Minnesota, Minneapolis, Feb. 2001.
12. I.T. Jolliffe, Principal Component Analysis. Springer-Verlag, 1986.
13. J.F. Hair, R.L. Tatham, R.E. Anderson, and W. Black, Multivariate Data Analysis, fifth ed. Prentice Hall, Mar. 1998.
14. R. Kohavi and G. John, "Wrappers for Feature Subset Selection," Artificial Intelligence, vol. 97, nos. 1-2, pp. 273-324, 1997.
15. H.J. Kushner and D.S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems. New York: Springer-Verlag, 1978.
16. D. Lewis, Y. Yang, T. Rose, and F. Li, "RCV1: A New Benchmark Collection for Text Categorization Research," J. Machine Learning Research, pp. 361-397, 2003.
17. D.D. Lewis, "Feature Selection and Feature Extraction for Text Categorization," Proc. Workshop Speech and Natural Language, pp. 212-217, 1992.
18. H. Li, T. Jiang, and K. Zhang, "Efficient and Robust Feature Extraction by Maximum Margin Criterion," Proc. Conf. Advances in Neural Information Processing Systems, pp. 97-104, 2004.
19. Y. Li, L. Xu, J. Morphett, and R. Jacobs, "An Integrated Algorithm of Incremental and Robust PCA," Proc. Int'l Conf. Image Processing, pp. 245-248, 2003.
20. R.-L. Liu and Y.-L. Lu, "Incremental Context Mining for Adaptive Document Classification," Proc. Eighth ACM Int'l Conf. Knowledge Discovery and Data Mining, pp. 599-604, 2002.
21. A.M. Martinez and A.C. Kak, "PCA versus LDA," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, pp. 228-233, 2001.
22. E. Oja, "Subspace Methods of Pattern Recognition," Pattern Recognition and Image Processing Series, vol. 6, 1983.
23. H. Park, M. Jeon, and J. Rosen, "Lower Dimensional Representation of Text Data Based on Centroids and Least Squares," BIT Numerical Math., vol. 43, pp. 427-448, 2003.
24. J. Platt, "Fast Training of Support Vector Machines Using Sequential Minimal Optimization," Advances in Kernel Methods: Support Vector Learning, pp. 185-208, 1999.
25. B.-Y. Ricardo and R.-N. Berthier, Modern Information Retrieval. Addison Wesley Longman, 1999.
26. R.O. Duda, P.E. Hart, and D.G Stork, Pattern Classification, second ed. John Wiley, 2001.
27. S.T. Roweis and L.K. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, pp. 2323-2326, 2000.
28. G. Salton and C. Buckley, "Term Weighting Approaches in Automatic Text Retrieval," Information Processing and Management, vol. 24, pp. 513-523, 1988.
29. M. Spitters, "Comparing Feature Sets for Learning Text Categorization," Proc. Int'l Conf. Computer-Assisted Information Retrieval, pp. 233-251, 2000.
30. J.B. Tenenbaum, V. de Silva, and J.C. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, pp. 2319-2323, 2000.
31. R.J. Vaccaro, SVD and Signal Processing II: Algorithms, Analysis and Applications. Elsevier Science, 1991.
32. A.R. Webb, Statistical Pattern Recognition, second ed. John Wiley, 2002.
33. J. Weng, Y. Zhang, and W.-S. Hwang, "Candid Covariance-Free Incremental Principal Component Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, pp. 1034-1040, 2003.
34. J. Yan, B.Y. Zhang, S.C. Yan, Z. Chen, W.G. Fan, Q. Yang, W.Y. Ma, and Q.S. Cheng, "IMMC: Incremental Maximum, Marginal Criterion," Proc. 10th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 725-730, 2004.
35. J. Yan, N. Liu, B.Y. Zhang, S.C. Yan, Q.S. Cheng, W.G. Fan, Z. Chen, W.S. Xi, and W.Y. Ma, "OCFS: Orthogonal Centroid Feature Selection," Proc. 28th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, 2005.
36. Y. Yang and J.O. Pedersen, "A Comparative Study on Feature Selection in Text Categorization," Proc. 14th Int'l Conf. Machine Learning, pp. 412-420, 1997.