Constrained empirical risk minimization framework for distance metric learning

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 8, 2012, Pages 1194-1205

  Bian, Wei ^a   Tao, Dacheng ^a  

^a UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

Data classification; distance metric learning; empirical risk minimization; first order method; generalization; image retrieval; optimal convergence rate

Indexed keywords

DATA CLASSIFICATION; DISTANCE METRIC LEARNING; EMPIRICAL RISK MINIMIZATION; FIRST-ORDER METHODS; GENERALIZATION; OPTIMAL CONVERGENCE;

ALGORITHMS; IMAGE RETRIEVAL; OPTIMIZATION;

CLASSIFICATION (OF INFORMATION);

EID: 84873598142     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2198075     Document Type: Article

References (50)

1. M. S. Baghshah and S. B. Shouraki, "Semi-supervised metric learning using pairwise constraints," in Proc. Int. Joint Conf. Artif. Intell., Pasadena, CA, 2009, pp. 1217-1222.
2. D.-Y. Yeung and H. Chang, "A kernel approach for semisupervised metric learning," IEEE Trans. Neural Netw., vol. 18, no. 1, pp. 141-149, Jan. 2007.
3. C. Shen, J. Kim, and L. Wang, "Scalable large-margin Mahalanobis distance metric learning," IEEE Trans. Neural Netw., vol. 21, no. 9, pp. 1524-1530, Sep. 2010.
4. E. Hu, S. Chen, D. Zhang, and X. Yin, "Semisupervised kernel matrix learning by kernel propagation," IEEE Trans. Neural Netw., vol. 21, no. 11, pp. 1831-1841, Nov. 2010.
5. D. Wang, D. Yeung, and E. Tsang, "Weighted Mahalanobis distance kernels for support vector machines," IEEE Trans. Neural Netw., vol. 18, no. 5, pp. 1453-1462, Sep. 2007.
6. X. Liang and Z. Ni, "Hyperellipsoidal statistical classifications in a reproducing kernel Hilbert space," IEEE Trans. Neural Netw., vol. 22, no. 6, pp. 968-975, Jun. 2011.
7. E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell, "Distance metric learning, with application to clustering with side-information," in Proc. Adv. Neural Inf. Process. Syst., 2002, pp. 505-512.
8. S. Chopra, R. Hadsell, and Y. LeCun, "Learning a similarity metric discriminatively, with application to face verification," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Washington, DC, Jun. 2005, pp. 539-546.
9. G. Lebanon, "Metric learning for text documents," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 4, pp. 497-508, Apr. 2006.
10. A. Frome, F. Sha, Y. Singer, and J. Malik, "Learning globally-consistent local distance functions for shape-based image retrieval and classification," in Proc. IEEE 11th Int. Conf. Comput. Vis., Oct. 2007, pp. 1-8.
11. A. Frome, Y. Singer, and J. Malik, "Image retrieval and classification using local distance functions," in Proc. Adv. Neural Inf. Process. Syst., 2006, pp. 417-424.
12. T. Hertz, A. Bar-Hillel, and D. Weinshall, "Learning distance functions for image retrieval," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Washington, DC, Jul. 2004, pp. 570-577.
13. M. Slaney, K. Q. Weinberger, and W. White, "Learning a metric for music similarity," in Proc. Int. Conf. Music Inf. Retr., Philadelphia, PA, 2008, pp. 313-318.
14. C. D. Manning, P. Raghavan, and H. Schtze, Introduction to Information Retrieval. Cambridge, U.K.: Cambridge Univ. Press, 2008.
15. L. Yang, R. Jin, L. Mummert, R. Sukthankar, A. Goode, B. Zheng, S. C. Hoi, and M. Satyanarayanan, "A boosting framework for visualitypreserving distance metric learning and its application to medical image retrieval," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 1, pp. 30-44, Jan. 2010.
16. A. Frome, Y. Singer, and J. Malik, "Image retrieval and classification using local distance functions," in Proc. Adv. Neural Inf. Process. Syst., 2007, pp. 417-424.
17. P. Jain, B. Kulis, and K. Grauman, "Fast image search for learned metrics," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
18. D. Tran and A. Sorokin, "Human activity recognition with metric learning," in Proc. 10th Eur. Conf. Comput. Vis., 2008, pp. 548-561.
19. L. Yang and R. Jin, "Distance metric learning: A comprehensive survey," Dept. Comput. Sci. Eng., Michigan State Univ., East Lansing, Tech. Rep., May 2006.
20. A. Globerson and S. Roweis, "Metric learning by collapsing classes," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, 2005, pp. 1-8.
21. M. Schultz and T. Joachims, "Learning a distance metric from relative comparisons," in Proc. Adv. Neural Inf. Process. Syst., 2004, pp. 1-8.
22. J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon, "Informationtheoretic metric learning," in Proc. 24th Int. Conf. Mach. Learn., 2007, pp. 209-216.
23. S. Wang and R. Jin, "An information geometry approach for distance metric learning," in Proc. 12th Int. Conf. Artif. Intell. Stat., 2009, pp. 591-598.
24. N. Cristianini, J. Kandola, A. Elisseeff, and J. Shawe-Taylor, "On kerneltarget alignment," in Proc. Adv. Neural Inf. Process. Syst. 14, 2002, pp. 367-373.
25. J. T. Kwok and I. W. Tsang, "Learning with idealized kernels," in Proc. Int. Conf. Mach. Learn., Aug. 2003, pp. 400-407.
26. A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall, "Learning a Mahalanobis metric from equivalence constraints," J.Mach. Learn. Res., vol. 6, pp. 937-965, Dec. 2005.
27. J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, "Neighbourhood components analysis," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, 2004, pp. 513-520.
28. L. Yang, R. Sukthankar, and R. Jin, "Bayesian active distance metric learning," in Proc. 23rd Conf. Uncertainty Artif. Intell., Vancouver, BC, Canada, 2007, pp. 1-8.
29. K. Q. Weinberger, J. Blitzer, and L. K. Saul, "Distance netric learning for large margin nearest neighbor classification," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, 2005, pp. 1473-1480.
30. R. Jin, S. Wang, and Y. Zhou, "Regularized distance metric learning: Theory and algorithm," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, 2010, pp. 1-9.
31. M. Loog, R. P. W. Duin, and R. Haeb-Umbach, "Multiclass linear dimension reduction by weighted pairwise fisher criteria," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 7, pp. 762-766, Jul. 2001.
32. M. Loog and R. P. W. Duin, "Linear dimensionality reduction via a heteroscedastic extension of LDA: The chernoff criterion," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 6, pp. 732-739, Jun. 2004.
33. O. C. Hamsici and A. M. Martinez, "Bayes optimality in linear discriminant analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 4, pp. 647-657, Apr. 2008.
34. D. Tao, X. Li, X. Wu, and S. J. Maybank, "Geometric mean for subspace selection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 260-274, Feb. 2009.
35. W. Bian and D. Tao, "Biased discriminant euclidean embedding for content-based image retrieval," IEEE Trans. Image Process., vol. 19, no. 2, pp. 545-554, Feb. 2010.
36. W. Bian and D. Tao, "Max-min distance analysis by using sequential SDP relaxation for dimension reduction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 1037-1050, May 2011.
37. W. Bian and D. Tao, "Learning a distance metric by empirical loss minimization," in Proc. Int. Joint Conf. Artif. Intell., Barcelona, Spain, 2011, pp. 1186-1191.
38. J. Dattorro, Convex Optimization & Euclidean Distance Geometry. Palo Alto, CA: Meboo Publishing, 2011.
39. A. Maurer, "Generalization bounds for subspace selection and hyperbolic PCA," in Proc. Subspace Latent Struct. Feature Select. Stat. Optim. Perspect. Workshop, Bohinj, Slovenia, 2006, pp. 185-197.
40. O. Bousquet and A. Elisseeff, "Stability and generalization," J. Mach. Learn. Res., vol. 2, pp. 499-526, Mar. 2002.
41. F. Cucker and S. Smale, "On the mathematical foundations of learning," Bull. Amer. Math. Soc., vol. 39, no. 1, pp. 1-49, 2002.
42. A. S. Nemirovsky and D. B. Yudin, Problem Complexity and Method Efficiency in Optimization (Discrete Mathematics). New York: Wiley, 1983.
43. N. Guan, D. Tao, Z. Luo, and B. Yuan, "NeNMF: An optimal gradient method for solving non-negative matrix factorization and its variants," IEEE Trans. Signal Process., vol. 60, no. 6, pp. 2882-2898, Jun. 2012.
44. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Non-negative patch alignment framework," IEEE Trans. Neural Netw., vol. 22, no. 8, pp. 1218-1230, Aug. 2011.
45. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Online non-negative matrix factorization with robust stochastic approximation," IEEE Trans. Neural Netw., DOI: 10.1109/TNNLS.2012.2197827, 2012.
46. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course (Applied Optimization). Boston, MA: Kluwer, 2004.
47. J. D. M. Rennie. (2005, Feb.). Smooth Hinge Classification [Online]. Available: http://people.csail.mit.edu/jrennie/writing
48. D. Wolpert, "The lack of a priori distinctions between learning algorithms," Neural Comput., vol. 8, no. 7, pp. 1341-1390, Oct. 1996.
49. A. Asuncion and D. Newman, "UCI machine learning repository," School Inf. Comput. Sci., Univ. California, Irvine, Tech. Rep., 2007.
50. A. Oliva and A. Torralba, "Modeling the shape of the scene: A holistic representation of the spatial envelope," Int. J. Comput. Vis., vol. 42, pp. 145-175, May 2001.