Low-rank kernel learning with bregman matrix divergences

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 341-376

  Kulis, Brian ^a   Sustik, Mátyás A ^a   Dhillon, Inderjit S ^a  

^a University of Texas at Austin   (United States)

Author keywords

Bregman divergences; Convex optimization; Kernel learning; Kernel methods; Matrix nearness

Indexed keywords

CONVEX OPTIMIZATION; EDUCATION; LEARNING SYSTEMS;

BREGMAN DIVERGENCES; EFFICIENT ALGORITHMS; INPUT MATRICES; KERNEL LEARNING; KERNEL MATRICES; KERNEL METHODS; LEARNING KERNELS; LEARNING PROBLEMS; LOW-RANK MATRICES; MACHINE LEARNING APPLICATIONS; MATRIX NEARNESS; NUMBER OF DATUM; POSITIVE SEMIDEFINITENESS; RUNNING TIME;

LEARNING ALGORITHMS;

EID: 61749094150     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (40)

1. F. Bach and M. Jordan. Predictive low-rank decomposition for kernel methods. In Proc. 22nd International Conference on Machine Learning (ICML), 2005.
2. J. Barnes and P. Hut. A hierarchical O(n log n) force calculation algorithm. Nature, 324:446-449, 1986.
3. M. Bilenko, S. Basu, and R. Mooney. Integrating constraints and metric learning in semi-supervised clustering. In Proceedings of the 21st International Conference on Machine Learning, 2004.
4. L. Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comp. Mathematics and Mathematical Physics, 7:200-217, 1967.
5. R.P. Brent. Algorithms for Minimization without Derivatives. Prentice-Hall, Englewood Cliffs, NJ., 1973. ISBN 0-13-022335-2.
6. Y. Censor and S. Zenios. Parallel Optimization. Oxford University Press, 1997.
7. T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley-Interscience, 1991.
8. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola. On kernel-target alignment. In Advances in Neural Information Processing Systems (NIPS) 14, 2002.
9. J. Davis, B. Kulis, S. Sra, and I. S. Dhillon. Information-theoretic metric learning. In Proc. 24th International Conference on Machine Learning (ICML), 2007.
10. J. D. Demmel. Applied Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997.
11. I. S. Dhillon and J. A. Tropp. Matrix nearness problems with Bregman divergences. SIAM Journal on Matrix Analysis and Applications, 29(4): 1120-1146, 2007.
12. I. S. Dhillon, Y. Guan, and B. Kulis. Kernel k-means, spectral clustering and normalized cuts. In Proc. 10th ACM SIGKDD Conference, 2004.
13. S. Fine and K. Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2:243-264, 2001.
14. R. Fletcher. A new variational result for quasi-Newton formulae. SIAM Journal on Optimziation, 1 (1), February 1991.
15. A. Globerson and S. Roweis. Metric learning by collapsing classes. In Advances in Neural Information Processing Systems (NIPS) 18, 2005.
16. G. Golub. Some modified matrix eigenvalue problems. SIAM Review, 15:318-334, 1973.
17. G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins University Press, 1996.
18. L. Greengard and V. Rokhlin. A fast algorithm for particle simulations. J. Comput. Phys., 73: 325-348, 1987.
19. M. Gu and S. Eisenstat. A stable and efficient algorithm for the rank-one modification of the symmetric eigenproblem. SIAM J. Matrix Anal. Appl., 15:1266-1276, 1994.
20. N. Higham. Computing the nearest correlation matrix - a problem from finance. IMA J. Numerical Analysis, 22(3):329-343, 2002.
21. P. Jain, B. Kulis, and K. Grauman. Fast image search for learned metrics. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2008.
22. W. James and C. Stein. Estimation with quadratic loss. In Proc. Fourth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 361-379. Univ. of California Press, 1961.
23. B. Kulis, M. A. Sustik, and I. S. Dhillon. Learning low-rank kernel matrices. In Proc. 23rd International Conference on Machine Learning (ICML), 2006.
24. B. Kulis, S. Sra, S. Jegelka, and I. S. Dhillon. Scalable semidefinite programming using convex perturbations. Technical Report TR-07-47, University of Texas at Austin, September 2007a.
25. B. Kulis, A. Surendran, and J. Piatt. Fast low-rank semidefinite programming for embedding and clustering. In Proc. 11th International Conference on Al and Statistics (AISTATS), 2007b.
26. J. Kwok and I. Tsang. Learning with idealized kernels. In Proc. 20th International Conference on Machine Learning (ICML), 2003.
27. G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, and M. Jordan. Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research, 5:27-72, 2004.
28. K. Lang. Fixing two weaknesses of the spectral method. In Advances in Neural Information Processing Systems (NIPS) 18, 2005.
29. M.A. Nielsen and I.L. Chuang. Quantum Computation and Quantum Information. Cambridge University Press, 2000.
30. M. Schultz and T. Joachims. Learning a distance metric from relative comparisons. In Advances in Neural Information Processing Systems (NIPS) 16, 2003.
31. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
32. J. Sherman and W. J. Morrison. Adjustment of an inverse matrix corresponding to changes in the elements of a given column or a given row of the original matrix. Annals of Mathematical Statistics, 20, 1949.
33. A. Strehl, J. Ghosh, and R. Mooney. Impact of similarity measures on web-page clustering. In Workshop on Artificial Intelligence for Web Search (AAAI), 2000.
34. M. A. Sustik and I. S. Dhillon. On some modified root-finding problems. Working manuscript, 2008.
35. L. Torresani and K. Lee. Large margin component analysis. In Advances in Neural Information Processing Systems (NIPS) 19, 2006.
36. K. Tsuda, G. Rátsch, and M. Warmuth. Matrix exponentiated gradient updates for online learning and Bregman projection. Journal of Machine Learning Research, 6:995-1018, 2005.
37. M. K. Warmuth and D. Kuzmin. Randomized PCA algorithms with regret bounds that are logarithmic in the dimension. In Advances in Neural Information Processing Systems (NIPS) 20, 2006.
38. K. Weinberger, F. Sha, and L. Saul. Learning a kernel matrix for nonlinear dimensionality reduction. In Proc. 21st International Conference on Machine Learning (ICML), 2004.
39. K. Weinberger, J. Blitzer, and L. Saul. Distance metric learning for large margin nearest neighbor classification. In Advances in Neural Information Processing Systems (NIPS) 18, 2005.
40. K. Weinberger, F. Sha, Q. Zhu, and L. Saul. Graph Laplacian methods for large-scale semidefinite programming, with an application to sensor localization. In Advances in Neural Information Processing Systems (NIPS) 19, 2006.