IDR/QR: An incremental dimension reduction algorithm via QR decomposition

IEEE Transactions on Knowledge and Data Engineering

Volumn 17, Issue 9, 2005, Pages 1208-1221

  Ye, Jieping ^a,b   Li, Qi ^a,c   Xiong, Hui ^a,b   Park, Haesun ^b   Janardan, Ravi ^a,b   Kumar, Vipin ^a,b  

^a IEEE

^b University of Minnesota   (United States)

^c UNIVERSITY OF DELAWARE   (United States)

Author keywords

Dimension reduction; Incremental learning; Linear discriminant analysis; QR decomposition; Singular value decomposition (SVD)

Indexed keywords

DIMENSION REDUCTION; INCREMENTAL LEARNING; LINEAR DISCRIMINANT ANALYSIS; QR DECOMPOSITION; SINGULAR VALUE DECOMPOSITION (SVD);

ALGORITHMS; DATA MINING; DATA STORAGE EQUIPMENT; DATABASE SYSTEMS; EIGENVALUES AND EIGENFUNCTIONS; INFORMATION RETRIEVAL;

DATA PROCESSING;

EID: 33947660936     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2005.148     Document Type: Article

References (37)

1. C.C. Aggarwal, "On the Effects of Dimensionality Reduction on High Dimensional Similarity Search," Proc. ACM Symp. Principles of Database Systems Conf., 2001.
2. P.N. Belhumeour, J.P. Hespanha, and D.J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
3. M.W. Berry, S.T. Dumais, and G.W. O'Brie, "Using Linear Algebra for Intelligent Information Retrieval," SIAM Rev., vol. 37, pp. 573-595, 1995.
4. C. Bohm, S. Berchtold, and D.A. Keim, "Searching in High-Dimensional Spaces: Index Structures for Improving the Performance of Multimedia Databases," ACM Computing Surveys, vol. 33, no. 3, pp. 322-373, 2001.
5. V. Castelli, A. Thomasian, and C.-S. Li, "Csvd: Clustering and Singular Value Decomposition for Approximate Similarity Searches in High Dimensional Space," IEEE Trans. Knowledge Discovery and Data Eng., vol. 15, no. 3, pp. 671-685, May/June 2003.
6. S. Chakrabarti, S. Roy, and M. Soundalgekar, "Fast and Accurate Text Classification via Multiple Linear Discriminant Projections," Proc. Very Large Data Bases Conf., pp. 658-669, 2002.
7. S. Chandrasekaran, B.S. Manjunath, Y.F. Wang, J. Winkeler, and H. Zhang, "An Eigenspace Update Algorithm for Image Analysis," Graphical Models and Image Processing: GMIP, vol. 59, no. 5, pp. 321-332, 1997.
8. C. Chatterjee and V.P. Roychowdhury, "On Self-Organizing Algorithms and Networks for Class-Separability Features," IEEE Trans. Neural Networks, vol. 8, no. 3, pp. 663-678, 1997.
9. J.W. Daniel, W.D. Gragg, L. Kaufman, and G.W. Stewart, "Reorthogonalization and Stable Algorithms for Updating the Gram-Schmidt Gr Factorization," Math. Computation, vol. 30, pp. 772-795, 1976.
10. S. Deerwestcr, S.T. Dumais, G.wrpurnas, T.K. Landauer, and R. Harshman, "Indexing by Latent Semantic Analysis," J. Soc. Information Science, vol. 41, pp. 391-407, 1990.
11. I.S. Dhillon and D.S. Modha, "Concept Decompositions for Large Sparse Text Data Using Clustering," Machine Learning, vol. 42, pp. 143-175, 2001.
12. R.O. Duda, P.E. Hart, and D. Stork, Pattern Classification. Wiley, 2000.
13. S. Dudoit, J. Fridlyand, and T.P. Speed, "Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data," J. Am. Statistical Assoc., vol. 97, no. 457, pp. 77-87, 2002.
14. J. Van et al. "IMMC: Incremental Maximum Margin Criterion," Proc. Int'l Conf. Knowledge Discovery and Data Mining, pp. 725-730, 2004.
15. K. Hiraoka et al. "Fast Algorithm for Online Linear Discriminant Analysis," IEICE Trans. Fundamentals, vol. E84-A, no. 6, pp. 1431-1441, 2001.
16. J.H. Friedman, "Regularized Discriminant Analysis," J. Am. Statistical Assoc., vol. 84, no. 405, pp. 165-175, 1989.
17. K. Fukunaga, Introduction to Statistical Pattern Classification. Academic Press, 1990.
18. G.H. Golub and C.F. Van Loan, Matrix Computations, third ed. Baltimore, M.D.: The Johns Hopkins Univ. Press, 1996.
19. P. Hall, D. Marshall, and R. Martin, "Merging and Splitting Eigenspace Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 9, pp. 1042-1049, Sept. 2000.
20. T. Hastie and R. Tibshirani, "Discriminant Analysis by Gaussian Mixtures," J. Royal Statistical Soc. series B, vol. 58, pp. 158-176, 1996.
21. T. Hastie, R. Tibshirani, and J.H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2001.
22. P. Howland, M. Jeon, and H. Park, "Structure Preserving Dimension Reduction for Clustered Text Data Based on the Generalized Singular Value Decomposition," SIAM J. Matrix Analysis and Applications, vol. 25, no. 1, pp. 165-179, 2003.
23. I.T. Jolliffe, Principal Component Analysis. New York: Springer-Verlag, 1986.
24. K.V. Ravi Kanth, D. Agrawal, A. El Abbadi, and A. Singh, "Dimensionality Reduction for Similarity Searching in Dynamic Databases," Computer Vision and Image Understanding: CVIU, vol. 75, nos. 1-2, pp. 59-72, 1999.
25. W.J. Krzanowski, P. Jonathan, W.V McCarthy, and M.R. Thomas, "Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data," Applied Statistics, vol. 44, pp. 101-115, 1995.
26. H. Li, J. Tao, and K. Zhang, "Efficient and Robust Feature Extraction by Maximum Margin Criterion," Advances in Neural Information Processing Systems 16, Cambridge, Mass.: MIT Press, 2004.
27. J. Mao and K. Jain, "Artificial Neural Networks for Feature Extraction and Multivariate Data Projection," IEEE Trans. Neural Networks, vol. 6, no. 2, pp. 296-317, 1995.
28. A. Martinez and A. Kak, "PCA versus LDA," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, pp. 228-233, 2001.
29. A.M. Martinez and R. Benavente, "The AR Face Database," Technical Report No. 24, 1998.
30. H. Park, M. Jeon, and J.B. Rosen, "Lower Dimensional Representation of Text Data Based on Centroids and Least Squares," BIT Numerical Math., vol. 43, no. 2, pp. 1-22, 2003.
31. R. Polikar, L. Udpa, S. Udpa, and V. Honavar, "Learn++: An Incremental Learning Algorithm for Supervised Neural Networks," IEEE Trans. Systems, Man, and Cybernetics, vol. 31, pp. 497-508, 2001.
32. M.F. Porter, "An Algorithm for Suffix Stripping Program," Program, vol. 14, no. 3, pp. 130-137, 1980.
33. G. Salton, Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, 1989.
34. D.L. Swets and J. Weng, "Using Discriminant Eigenfeatures for Image Retrieval," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 831-836, Aug. 1996.
35. F.D.L. Torre and M. Black, "Robust Principal Component Analysis for Computer Vision," Proc. Int'l Conf. Computer Vision, pp. 362-369, 2001.
36. J. Ye, R. Janardan, C.H. Park, and H. Park, "An Optimization Criterion for Generalized Discriminant Analysis on Undersampled Problems," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 982-994, Aug. 2004.
37. Y. Zhao and G. Karypis, "Empirical and Theoretical Comparisons of Selected Criterion Functions for Document Clustering," Machine Learning, vol. 55, no. 3, pp. 311-331, 2004.