Adaptive nonlinear discriminant analysis by regularized minimum squared errors

IEEE Transactions on Knowledge and Data Engineering

Volumn 18, Issue 5, 2006, Pages 603-612

  Kim, Hyunsoo ^a,b   Drake, Barry L ^a,b,c   Park, Haesun ^b,d  

^a IEEE   (United States)

^b Georgia Institute of Technology   (United States)

^c Analytic Solutions Inc   (United States)

^d NATIONAL SCIENCE FOUNDATION   (United States)

Author keywords

Adaptive classifier; Kernel methods; Leave one out cross validation; Linear discriminant analysis; Qr decomposition updating and downdating; Regularization

Indexed keywords

ADAPTIVE CLASSIFIERS; KERNEL METHODS; LEAVE-ONE-OUT CROSS VALIDATION; LINEAR DISCRIMINANT ANALYSIS; QR DECOMPOSITION UPDATING AND DOWNDATING; REGULARIZATION; SINGULAR VALUE DECOMPOSITION (SVD);

ADAPTIVE ALGORITHMS; ALGORITHMS; COMPUTATIONAL METHODS; EIGENVALUES AND EIGENFUNCTIONS; MATHEMATICAL TECHNIQUES; PROBLEM SOLVING;

NONLINEAR SYSTEMS;

EID: 33645663296     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2006.72     Document Type: Article

References (33)

1. S.A. Billings and K.L. Lee, "Nonlinear Fisher Discriminant Analysis Using a Minimum Squared Error Cost Function and the Orthogonal Least Squares Algorithm," Neural Networks, vol. 15, pp. 263-270, 2002.
2. A. Björck, "Numerical Methods for Least Square Problems," Proc. SIAM Conf., 1996.
3. A. Björck, H. Park, and L. Eldén, "Accurate Downdating of Least Squares Solutions," SIAM J. Matrix Analysis Applications, vol. 15, pp. 549-568, 1994.
4. A.W. Bojanczyk, R.P. Brent, P. van Dooren, and F.R. de Hoog, "A Note on Downdating the Cholesky Factorization," SIAM J. Science and Statistical Computation, vol. 8, pp. 210-221, 1987.
5. P. Businger, "Updating a Singular Value Decomposition," BIT, vol. 10, pp. 376-385, 1970.
6. L. Chen, H.M. Liao, M. Ko, J. Lin, and G. Yu, "A New LDA-Based Face Recognition System which Can Solve the Small Sample Size Problem," Pattern Recognition, vol. 33, pp. 1713-1726, 2000.
7. N. Cristianini and J. Shawe-Taylor, Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge: Univ. Press, 2000.
8. R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification. Wiley-Interscience, 2001.
9. L. Eldén and H. Park, "Block Downdating of Least Squares Solutions," SIAM J. Matrix Analysis Applications, vol. 15, pp. 1018-1034, 1994.
10. L. Eldén and H. Park, "Perturbation Analysis for Block Down-dating of a Cholesky Decomposition," Numerical Math., vol. 68, pp. 457-468, 1994.
11. W.R. Ferng, G.H. Golub, and R.J. Plemmons, "Adaptive Lanczos Methods for Recursive Condition Estimation," Numerical Algorithms, vol. 1, pp. 1-20, 1991.
12. K. Fukunaga, Introduction to Statistical Pattern Recognition, second ed. Academic Press, 1990.
13. P.E. Gill, G.H. Golub, W. Murray, and M.A. Saunders, "Methods for Modifying Matrix Factorizations," Math. Computation, vol. 28, pp. 505-535, 1974.
14. G.H. Golub and C.F. van Loan, Matrix Computations, third ed. Johns Hopkins Univ. Press, 1996.
15. P. Howland, M. Jeon, and H. Park, "Structure Preserving Dimension Reduction for Clustered Text Data Based on the Generalized Singular Value Decomposition," SIAM J. Matrix Analysis Application, vol. 25, pp. 165-179, 2003.
16. H. Kim, "Machine Learning and Bioinformatics," PhD thesis, Univ. of Minnesota, Twin Cities, 2004.
17. MATLAB User's Guide, The MathWorks, Inc., 1992.
18. S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K.R. Müller, "Fisher Discriminant Analysis with Kernels," Proc. Conf. Neural Networks for Signal Processing, pp. 41-48, 1999.
19. M. Moonen, P. van Dooren, and J. Vandewalle, "A Singular Value Decomposition Updating Algorithm," SIAM J. Matrix Analysis Applications, vol. 13, pp. 1015-1038, 1992.
20. C.C. Paige and M.A. Saunders, "Towards a Generalized Singular Value Decomposition," SIAM J. Numerical Analysis, vol. 18, pp. 398-405, 1981.
21. C.H. Park and H. Park, "Nonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition," SIAM J. Matrix Analysis Application, to appear.
22. C.H. Park and H. Park, "A Relationship between LDA and the Generalized Minimum Squared Error Solution," J. Matrix Analysis Application, to appear.
23. D.J. Pierce and R.J. Plemmons, "Fast Adaptive Condition Estimation," SIAM J. Matrix Analysis Applications, vol. 13, pp. 274-291, 1992.
24. G. Shroff and C.H. Bischof, "Adaptive Condition Estimation for Rank-One Updates of QR Factorizations," SIAM J. Matrix Analysis Application, vol. 13, pp. 1264-1278, 1992.
25. G.W. Stewart, "The Effects of Rounding Error on an Algorithm for Downdating a Cholesky Factorization," The Inst. Math. and Its Applications J., vol. 23, pp. 203-213, 1979.
26. G.W. Stewart, "Updating a Rank-Revealing ULV Decomposition," SIAM J. Matrix Analysis Application, vol. 14, pp. 494-499, 1993.
27. G.W. Stewart, "Updating URV Decompositions in Parallel," Parallel Computing, vol. 20, pp. 151-172, 1994.
28. M. Stewart and P. van Dooren, "Updating a Generalized URV Decomposition," SIAM J. Matrix Analysis Application, vol. 22, pp. 479-500, 2000.
29. N.A. Syed, H. Liu, and K.K. Sung, "Incremental Learning with Support Vector Machines," Proc. Workshop Support Vector Machines at the Int'l Joint Conf. Articial Intelligence (IJCAI-99), 1999.
30. V. Vapnik, The Nature of Statistical Learning Theory. Springer-Verlag, 1995.
31. V. Vapnik, Statistical Learning Theory. John Wiley and Sons, 1998.
32. J. Yang and J.Y. Yang, "Why Can LDA Be Performed in PCA Transformed Space?" Pattern Recognition, vol. 36, pp. 563-566, 2003.
33. H. Yu and J. Yang, "A Direct LDA Algorithm for High-Dimensional Data with Application to Face Recognition," Pattern Recognition, vol. 34, pp. 2067-2070, 2001.