Metric and kernel learning using a linear transformation

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 519-547

  Jain, Prateek ^a   Kulis, Brian ^b   Davis, Jason V ^c   Dhillon, Inderjit S ^d  

^a MICROSOFT RESEARCH   (United States)

^b OHIO STATE UNIVERSITY   (United States)

^c ETSY INC   (United States)

^d UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Kernel learning; Linear transformation; Logdet divergence; Matrix divergences; Metric learning

Indexed keywords

CONSTRUCTIVE METHODS; DATA POINTS; DIMENSIONALITY REDUCTION; KERNEL FUNCTION; KERNEL LEARNING; KERNEL MATRICES; LEARNING APPROACH; LEARNING METRICS; LEARNING PROBLEM; LINEAR CONSTRAINTS; LOGDET DIVERGENCE; MACHINE LEARNING APPLICATIONS; MAHALANOBIS METRIC; METRIC LEARNING; OPTIMIZATION FRAMEWORK; REAL-WORLD PROBLEM; SEMI-SUPERVISED; TEXT MINING;

COMPUTER VISION; LEARNING ALGORITHMS; LINEAR TRANSFORMATIONS;

MATRIX ALGEBRA;

EID: 84859478057     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (48)

1. A. Argyriou, C. A. Micchelli, and M. Pontil. On spectral learning. Journal of Machine Learning Research (JMLR), 11:935-953, 2010.
2. S. Arora and S. Kale. A combinatorial, primal-dual approach to semidefinite programs. In Proceedings of the ACM Symposium on Theory of Computing (STOC), pages 227-236, 2007. (Pubitemid 47630739)
3. A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall. Learning a mahalanobis metric from equivalence constraints. Journal of Machine Learning Research (JMLR), 6:937-965, 2005.
4. S. Basu, M. Bilenko, and R. J. Mooney. A probabilistic framework for semi-supervised clustering. In Proceedings of the ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD), 2004.
5. Y. Bengio, O. Delalleau, N. Le Roux, J. Paiement, P. Vincent, and M. Ouimet. Learning eigen-functions links spectral embedding and kernel PCA. Neural Computation, 16(10):2197-2219, 2004. (Pubitemid 39137820)
6. A. C. Berg and J. Malik. Geometric blur for template matching. In Proccedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pages 607-614, 2001.
7. J. Cai, E. J. Candes, and Z. Shen. A singular value thresholding algorithm for matrix completion. Arxiv:0810.3286, 2008.
8. Caltech-101 Data Set. http://www.vision.caltech.edu/ImageDatasets/ Caltech101/, 2004.
9. R. Chatpatanasiri, T. Korsrilabutr, P. Tangchanachaianan, and B. Kijsirikul. A new kernelization framework for Mahalanobis distance learning algorithms. Neurocomputing, 73(10-12):1570-1579, 2010.
10. S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity metric discriminatively, with application to face verification. In Proccedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2005.
11. Classic3 Data Set. ftp.cs.cornell.edu/pub/smart, 2008.
12. CMU 20-Newsgroups Data Set. http://www.cs.cmu.edu/afs/cs.cmu.edu/project/ theo-20/www/data/news20.html, 2008.
13. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola. On kernel-target alignment. In Proc-cedings of Advances in Neural Information Processing Systems (NIPS), 2001.
14. J. V. Davis and I. S. Dhillon. Structured metric learning for high dimensional problems. In Proceedings of the ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD), pages 195-203, 2008.
15. J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon. Information-theoretic metric learning. In Proccedings of the International Conference on Machine Learning (ICML), pages 209-216, 2007.
16. S. C. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman. Indexing by latent semantic analysis. Journal of the American Society of Information Science, 41(6):391-407, 1990.
17. R. Fletcher. A new variational result for quasi-Newton formulae. SIAM Journal on Optimization, 1 (1), 1991.
18. A. Globerson and S. T. Roweis. Metric learning by collapsing classes. In Proccedings of Advances in Neural Information Processing Systems (NIPS), 2005.
19. J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov. Neighbourhood component analysis. In Proccedings of Advances in Neural Information Processing Systems (NIPS), 2004.
20. K. Grauman and T. Darrell. The Pyramid Match Kernel: Efficient learning with sets of features. Journal of Machine Learning Research (JMLR), 8:725-760, April 2007. (Pubitemid 46677049)
21. M. Groschel, L. Lovasz, and A. Schrijver. Geometric Algorithms and Combinatorial Optimization. Springer-Verlag, 1988.
22. J. Ha, C. J. Rossbach, J. V. Davis, I. Roy, H. E. Ramadan, D. E. Porter, D. L. Chen, and E. Witchel. Improved error reporting for software that uses black-box components. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 101-111, 2007. (Pubitemid 47630679)
23. T. Hastie and R. Tibshirani. Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 18:607-616, 1996. (Pubitemid 126773896)
24. P. Jain, B. Kulis, I. S. Dhillon, and K. Grauman. Online metric learning and fast similarity search. In Proccedings of Advances in Neural Information Processing Systems (NIPS), pages 761-768, 2008.
25. P. Jain, B. Kulis, and I. S. Dhillon. Inductive regularized learning of kernel functions. In Proccedings of Advances in Neural Information Processing Systems (NIPS), 2010.
26. W. James and C. Stein. Estimation with quadratic loss. In Fourth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 361-379. Univ. of California Press, 1961.
27. B. Kulis, M. Sustik, and I. S. Dhillon. Learning low-rank kernel matrices. In Proccedings of the International Conference on Machine Learning (ICML), pages 505-512, 2006.
28. B. Kulis, M. Sustik, and I. Dhillon. Low-rank kernel learning with Bregman matrix divergences. Journal of Machine Learning Research, 2008.
29. B. Kulis, P. Jain, and K. Grauman. Fast similarity search for learned metrics. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 31(12):2143-2157, 2009a.
30. B. Kulis, S. Sra, and I. S. Dhillon. Convex perturbations for scalable semidefinite programming. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), 2009b.
31. J. T. Kwok and I. W. Tsang. Learning with idealized kernels. In Proccedings of the International Conference on Machine Learning (ICML), 2003.
32. G. R. G. Lanckriet, N. Cristianini, P. L. Bartlett, L. El Ghaoui, and M. I. Jordan. Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research (JMLR), 5:27-72, 2004.
33. S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In Proccedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pages 2169-2178, 2006. (Pubitemid 44931582)
34. G. Lebanon. Metric learning for text documents. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 28(4):497-508, 2006. ISSN 0162-8828. (Pubitemid 43264649)
35. M. A. Nielsen and I. L. Chuang. Quantum Computation and Quantum Information. Cambridge University Press, 2000.
36. C. S. Ong, A. J. Smola, and R. C. Williamson. Learning the kernel with hyperkernels. Journal of Machine Learning Research (JMLR), 6:1043-1071, 2005.
37. B. Recht, M. Fazel, and P. A. Parrilo. Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization. SIAM Review, 52(3):471-501, 2010.
38. M. Schultz and T. Joachims. Learning a distance metric from relative comparisons. In Proccedings of Advances in Neural Information Processing Systems (NIPS), 2003.
39. M. Seeger. Cross-validation optimization for large scale hierarchical classification kernel methods. In Proccedings of Advances in Neural Information Processing Systems (NIPS), pages 1233-1240, 2006.
40. S. Shalev-Shwartz, Y. Singer, and A. Y. Ng. Online and batch learning of pseudo-metrics. In Proccedings of the International Conference on Machine Learning (ICML), 2004.
41. V. Sindhwani, P. Niyogi, and M. Belkin. Beyond the point cloud: from transductive to semi-supervised learning. In Proccedings of the International Conference on Machine Learning (ICML), pages 824-831, 2005.
42. K. Tsuda, G. Rätsch, and M. K. Warmuth. Matrix exponentiated gradient updates for on-line learning and Bregman projection. Journal of Machine Learning Research (JMLR), 6:995-1018, 2005.
43. M. K. Warmuth and D. Kuzmin. Randomized online PCA algorithms with regret bounds that are logarithmic in the dimension. Journal of Machine Learning Research, 9:2287-2320, 2008.
44. K. Q. Weinberger and L. K. Saul. Fast solvers and efficient implementations for distance metric learning. In Proccedings of the International Conference on Machine Learning (ICML), 2008.
45. K. Q. Weinberger, J. Blitzer, and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. In Proccedings of Advances in Neural Information Processing Systems (NIPS), 2005.
46. E. P. Xing, A. Y. Ng, M. I. Jordan, and S. J. Russell. Distance metric learning with application to clustering with side-information. In Proccedings of Advances in Neural Information Processing Systems (NIPS), pages 505-512, 2002.
47. H. Zhang, A. C. Berg, M. Maire, and J. Malik. SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. In Proccedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pages 2126-2136, 2006. (Pubitemid 44931577)
48. X. Zhu, J. Kandola, Z. Ghahramani, and J. Lafferty. Nonparametric transforms of graph kernels for semi-supervised learning. In Lawrence K. Saul, Yair Weiss, and Léon Bottou, editors, Proc-cedings of Advances in Neural Information Processing Systems (NIPS), volume 17, pages 1641-1648, 2005.