Metric learning: A survey

Foundations and Trends in Machine Learning

Volumn 5, Issue 4, 2012, Pages 287-364

  Kulis, Brian ^a  

^a OHIO STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DISTANCE FUNCTIONS; HIGH-DIMENSIONAL FEATURE SPACE; LEARNING METHODS; MAHALANOBIS DISTANCES; NEAREST NEIGHBOR METHOD; NUMBER OF DATUM; PROGRAM ANALYSIS; RECENT PROGRESS;

DISTANCE EDUCATION;

SURVEYS;

EID: 84887890691     PISSN: 19358237     EISSN: 19358245     Source Type: Journal    
DOI: 10.1561/2200000019     Document Type: Review

References (84)

1. A. Beck and M. Teboulle, "Mirror descent and nonlinear projected subgradient methods for convex optimization", Operations Research Letters, vol. 31, pp. 167-175, 2003.
2. D. P. Bertsekas, Nonlinear Programming. Athena Scientific, 1999.
3. M. Bilenko, S. Basu, and R. Mooney, "Integrating constraints and metric learning in semi-supervised clustering", in Proceedings of International Conference on Machine Learning (ICML), 2004.
4. C. M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
5. L. Bottou, "Online algorithms and stochastic approximations", in Online Learning and Neural Networks, (D. Saad, ed.), Cambridge University Press, 1998.
6. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, 2004.
7. J. P. Boyle and R. L. Dykstra, "A method for finding projections onto the intersection of convex sets in Hilbert spaces", Lecture Notes in Statistics, vol. 37, pp. 28-47, 1986.
8. L. M. Bregman, "The relxation method of finding the common points of convex sets and its application to the solutionof problems in convex programming", USSR Computational Mathematics and Mathematical Physics, vol. 7, no. 3, pp. 200-217, 1967.
9. Q. Cao, Z. C. Guo, and Y. Ying, "Generalization bounds for metric and similarity learning", arXiv: 1207. 5437, 2012.
10. N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games. Cambridge University Press, 2006.
11. R. Chatpatanasiri, T. Korsrilabutr, P. Tangchanachaianan, and B. Kijsirikul, "A new kernelization framework for Mahalanobis distance learning algorithms", Neurocomputing, vol. 73, no. 10-12, pp. 1570-1579, 2010.
12. G. Chechik, V. Sharma, U. Shalit, and S. Bengio, "An online algorithm for lrage scale image similarity learning", in Advances in Neural Information Processing Systems (NIPS), 2009.
13. S. Chopra, R. Hadsell, and Y. LeCun, "Learning a similarity metric discriminatively, with application to face verification", in Proceedin gs of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2005.
14. P. Ciaccia, M. Patella, and P. Zezula, "M-Tree: An efficient access method for similarity search in metric spaces", in Proceedings of International Conference on Very Large Data Bases (VLDB), 1997.
15. K. Crammer, O. Dekel, S. Shalev-Shwartz, and Y. Singer, "Online passiveaggressive algorithms", in Advances in Neural Information Processing Systems, 2004.
16. O. G. Cula and K. J. Dana, "3D texture recognition using bidirectional feature histograms", International Journal of Computer Vision (IJCV), vol. 59, no. 1, pp. 33-60, 2004.
17. Curious Labs, Inc., Poser 5 - Reference Manual. Santa Cruz, CA, 2002.
18. H. Daume, "Frustratingly easy domain adaptation", in Conference of the Association for Computational Linguistics (ACL), 2007.
19. J. Davis and I. S. Dhillon, "Structured metric learning for high-dimensional problems", in Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2008.
20. J. Davis, B. Kulis, P. Jain, S. Sra, and I. Dhillon, "Information- theoretic metric learning", in Proceedings of International Conference on Machine Learning (ICML), 2007.
21. R. Fletcher, "A new variational result for quasi-Newton formulae", SIAM Journal on Optimization, vol. 1, no. 1, 1991.
22. J. Friedman, J. Bentley, and A. Finkel, "An algorithm for finding best matches in logarithmic expected time", ACM Transactions on Mathematics Software, vol. 3, no. 3, pp. 209-226, 1977.
23. A. Frome, Y. Singer, F. Sha, and J. Malik, "Learning globally consistent local distance functions for shape-based image retrieval and classification", in Proceedings of IEEE International Conference on Computer Vision (ICCV), 2007.
24. T. Gaertner, "A survey of kernels for structured data", ACM SIGKDD Explorations Newsletter, vol. 5, no. 1, 2003.
25. A. Globerson and S. Roweis, "Metric learning by collapsing classes", in Advances in Neural Information Processing Systems (NIPS), 2005.
26. J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, "Neighbourhood components analysis", in Advances in Neural Information Processing Systems (NIPS), 2004.
27. A. A. Goldstein, "Convex programming in Hilbert space", Bulletin of the American Mathematical Society, vol. 70, pp. 709-710, 1964.
28. G. H. Golub and C. F. Van Loan, Matrix Computations. Johns Hopkins University Press, 1996.
29. K. Grauman and T. Darrell, "The pyramid match kernel: Efficient learning with sets of features", Journal of Machine Learning Research, vol. 8, pp. 725-760, 2007.
30. M. Guillaumin, J. Verbeek, and C. Schmid, "Is that you? Metric learning approaches for face identification", in P roceedin gs of IEEE I nternational Conference on Computer Vision (ICCV), 2009.
31. J. Ha, C. Rossbach, J. Davis, I. Roy, D. Chen, H. Ramadan, and E. Witchel, "Improved error reporting for software that uses black box components", in Proceedings of Programming Language Design and Implementation (PLDI), 2007.
32. S. C. H. Hoi, W. Liu, M. R. Lyu, and W. Y. Ma, "Learning distance metrics with contextual constraints for image retrieval", in Proceedin gs of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2006.
33. G. Hua, M. Brown, and S. Winder, "Discriminant embedding for local image descriptors", in Proceedings of IEEE International Conference on Computer Vision (ICCV), 2007.
34. P. Indyk and R. Motwani, "Approximate nearest neighbors: Towards removing the curse of dimensionality", in Proceedings of Symposium on Theory of Computing (STOC), 1998.
35. P. Jain, B. Kulis, J. Davis, and I. Dhillon, "Metric and kernel learning using a linear transformation", Journal of Machine Learning Research, vol. 13, pp. 519-547, 2012.
36. P. Jain, B. Kulis, and I. Dhillon, "Inductive regularized learning of kernel functions", in Advances in Neural Information Processing Systems (NIPS), 2010.
37. P. Jain, B. Kulis, I. Dhillon, and K. Grauman, "Online metric learning and fast similarity search", in Advances in Neural Information Processing Systems (NIPS), 2008.
38. P. Jain, B. Kulis, and K. Grauman, "Fast image search for learned metrics", in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2008.
39. W. James and C. Stein, "Estimation with quadratic loss", Proceedin gs of Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 361-379, 1961.
40. R. Jin, S. Wang, and Y. Zhou, "Regularized distance metric learning: Theory and algorithm", in Advances in Neural Information Processing Systems (NIPS), 2009.
41. S. Kaski and J. Peltonen, "Informative discriminant analysis", in Proceedin gs of International Conference on Machine Learning (ICML), 2003.
42. D. Kedem, S. Tyree, K. Q. Weinberger, F. Sha, and G. Lanckriet, "Nonlinear metric learning", in Advances in Neural Information Processing Systems (NIPS), 2012.
43. B. Kulis, P. Jain, and K. Grauman, "Fast similarity search for learned metrics", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 12, pp. 2143-2157, 2009.
44. B. Kulis, K. Saenko, and T. Darrell, "What you saw is not what you get: Domain adaptation using asymmetric kernel transforms", in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011.
45. B. Kulis, M. Sustik, and I. Dhillon, "Learning low-rank kernel matrices", in Proceedings of International Conference on Machine Learning (ICML), 2006.
46. B. Kulis, M. Sustik, and I. Dhillon, "Low-rank kernel learning with Bregman matrix divergences", Journal of Machine Learning Research, vol. 10, pp. 341-376, 2009.
47. J. Kwok and I. Tsang, "Learning with idealized kernels", in Proceedin gs of International Conference on Machine Learning (ICML), 2003.
48. G. Lebanon, "Metric learning for text documents", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 497-508, 2006.
49. E. S. Levitin and B. T. Polyak, "Constrained minimization problems", USSR Computational Mathematics and Mathematical Physics, vol. 6, pp. 1-50, 1966.
50. H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristanini, and C. Watkins, "Text classification using string kernels", Journal of Machine Learning Research, vol. 2, pp. 419-444, 2002.
51. P. C. Mahalanobis, "On the generalized distance in statistics", Proceedigs of the National Institute of Sciences of India, vol. 2, no. 1, pp. 49-55, 1936.
52. B. McFee and G. Lanckriet, "Metric learning to rank", in Proceedings of International Conference on Machine Learning ICML, 2010.
53. G. J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition. Wiley Interscience, 2004.
54. S. Parameswaran and K. Q. Weinberger, "Large margin multi-task metric learning", in Advances in Neural Information Processing Systems (NIPS), 2010.
55. R. Rosales and G. Fung, "Learning sparse metric via linear programming", in Proceedings of SIGKDD Conference, 2006.
56. S. Roweis, "EM algorithms for PCA and SPCA", in Advances in Neural Information Processing Systems, 1998.
57. K. Saenko, B. Kulis, M. Fritz, and T. Darrell, "Adapting visual category models to new domains", in Proceedings of European Conference on Computer Vision (ECCV), 2010.
58. R. Salakhutdinov and G. Hinton, "Learning a nonlinear embedding by preserving class neighbourhood structure", in Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), 2007.
59. B. Schoelkopf and A. Smola, Learning with Kernels. MIT Press, 2002.
60. B. Schoelkopf, A. Smola, and K.-R. Mueller, "Nonlinear component analysis as a kernel eigenvalue problem", Neural Computation, vol. 10, no. 5, pp. 1299-1319, 1998.
61. M. Schultz and T. Joachims, "Learning a distance metric from relative comparisons", in Advances in Neural Information Processing Systems (NIPS), 2003.
62. G. Shakhnarovich, P. Viola, and T. Darrell, "Fast pose estimation with parameter-sensitive hashing", in Proceedings of IEEE International Conference on Computer Vision (ICCV), 2003.
63. S. Shalev-Shwartz, Y. Singer, and A. Y. Ng, "Online learning of pseudometrics", in Proceedings of International Conference on Machine Learning (ICML), 2004.
64. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
65. M. Slaney, K. Q. Weinberger, and W. White, "Learning a metric for music similarity", in International Symposium on Music Information Retrieval (ISMIR), 2008.
66. N. Snavely, S. Seitz, and R. Szeliski, "Photo tourism: Exploring photo collections in 3D", in Proceedings of ACM SIGGRAPH, 2006.
67. I. Takeuchi, M. Nakagawa, and M. Seto, "Metric learning for DNA microarray data analysis", in Proceedingso of International Workshop on Statistical-Mechanical Informatics (IW-SMI), 2009.
68. M. Taylor, B. Kulis, and F. Sha, "Metric learning for reinforcement learning agents", in Proceedings of International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2011.
69. R. Tibshirani, "Regression shrinkage and selection via the lasso", Journal of the Royal Statistical Society, Series B, vol. 58, no. 1, pp. 267-288, 1996.
70. M. E. Tipping and C. M. Bishop, "Probabilistic principal component analysis", Journal of Royal Statistical Society, Series B, vol. 21, no. 3, pp. 611-622, 1999.
71. L. Torresani and K. Lee, "Large margin component analysis", in Advances in Neural Information Processing Systems (NIPS), 2007.
72. D. Tran and A. Sorokin, "Human activity recognition with metric learning", in Proceedings of European Conference on Computer Vision (ECCV), 2008.
73. T. Tuytelaars and K. Mikolajczyk, "Local invariant feature detectors: A survey", Foundations and Trends in Computer Graphics and Vision, vol. 3, no. 3, pp. 177-280, 2008.
74. J. Uhlmann, "Satisfying general proximity/similarity queries with metric trees", Information Processing Letters, vol. 40, pp. 175-179, 1991.
75. M. Varma and A. Zisserman, "A statistical approach to material classification using image patch exemplars", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 11, pp. 2032-2047, 2009.
76. K. Q. Weinberger, J. Blitzer, and L. K. Saul, "Distance metric learning for large margin nearest neighbor classification", in Advances in Neural Information Processing Systems (NIPS), 2005.
77. K. Q. Weinberger and L. K. Saul, "Fast solvers and efficient implementations for distance metric learning", in Proceedings of International Conference on Machine Learning (ICML), 2008.
78. K. Q. Weinberger and L. K. Saul, "Distance metric learning for large margin nearest neighbor classification", Journal of Machine Learning Research, vol. 10, pp. 207-244, 2009.
79. K. Q. Weinberger and G. Tesauro, "Metric learning for kernel regression", in Proceedings of International Conference on Artificial Intel ligence and Statistics (AISTATS), 2007.
80. E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell, "Distance metric learning, with application to clustering with side-information", in Advances in Neural Information Processing Systems (NIPS), 2002.
81. H. Xiong and X. Chen, "Kernel-based distance metric learning for microarray data classification", BMC Bioinformatics, vol. 7, p. 299, 2006.
82. Y. Ying, K. Huang, and C. Campbell, "Sparse metric learning via smooth optimization", in Advances in Neural Information Processing Systems (NIPS), 2009.
83. Y. Ying and P. Li, "Distance metric learning with eigenvalue optimization", Journal of Machine Learning Research, vol. 13, pp. 1-26, 2012.
84. M. Zinkevich, "Online convex programming and generalized infinitesimal gradient ascent", in Proceedings of International Conference on Machine Learning (ICML), 2003.