Learning low-rank kernel matrices for constrained clustering

Neurocomputing

Volumn 74, Issue 12-13, 2011, Pages 2201-2211

  Baghshah, Mahdieh Soleymani ^a   Shouraki, Saeed Bagheri ^a  

^a SHARIF UNIVERSITY OF TECHNOLOGY   (Iran)

Author keywords

Constrained clustering; Distance metric; Kernel learning; Low rank kernel; Semi supervised; Spectral

Indexed keywords

CONSTRAINED CLUSTERING; DISTANCE METRIC; KERNEL LEARNING; LOW-RANK KERNEL; SEMI-SUPERVISED; SPECTRAL;

COMPUTATIONAL COMPLEXITY; LEARNING SYSTEMS; OPTIMIZATION;

MATRIX ALGEBRA;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; INFORMATION PROCESSING; INTERMETHOD COMPARISON; KERNEL METHOD; LEARNING; MACHINE LEARNING; MATHEMATICAL COMPUTING; MATHEMATICS; NONLINEAR SYSTEM; PRIORITY JOURNAL; PROBLEM SOLVING; PROCESS OPTIMIZATION;

EID: 79956106285     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2011.02.009     Document Type: Article

References (52)

1. Bar-Hillel A., Hertz T., Shental N., Weinshall D. Learning a Mahalanobis metric from equivalence constraints. Journal of Machine Learning Research 2005, 6:937-965.
2. S. Basu, M. Bilenko, R.J. Mooney, A probabilistic framework for semi-supervised clustering, in: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004, pp. 59-68.
3. S. Basu, Semi-supervised clustering: probabilistic models, algorithms and experiments, Ph.D. Dissertation, University of Texas at Austin, 2005.
4. Belkin M., Niyogi P. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 2003, 15(6):1373-1396.
5. Boyd S., Vandenberghe L. Convex optimization 2004, Cambridge University Press.
6. D. Cai, H. Xiaofei, H. Jiawei, Semi-supervised discriminant analysis, in: Proceedings of the 11th IEEE International Conference on Computer Vision (ICCV), 2007, pp. 1-7.
7. Chan P., Schlag M., Zien J. Spectral k-way ratio cut partitioning. IEEE Transactions CAD Integrated Circuits and Systems 1994, 13:1088-1096.
8. Chang H., Yeung D.Y. Locally linear metric adaptation with application to semi-supervised clustering and image retrieval. Pattern Recognition 2006, 39:1253-1264.
9. K. Crammer, J. Keshet, Y. Singer, Kernel design using boosting, in: Proceedings of Advances in Neural Information Processing Systems, vol. 15, MIT Press, 2003, pp. 537-544.
10. N. Cristianini, J. Kandola, A. Elisseeff, J. Shawe-Taylor, On kernel target alignment, in: Proceedings of Advances in Neural Information Processing Systems, vol. 14, MIT Press, 2002, pp. 367-373.
11. I.S. Dhillon, Y. Guan, B. Kulis, Kernel k-means, spectral clustering and normalized cuts, in: Proceedings of the Tenth ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2004, pp. 551-556.
12. Domeniconi C., Peng J., Gunopulos D. Locally adaptive metric nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(9):1281-1285.
13. Domeniconi C., Gunopulos D., Peng J. Large margin nearest neighbor classifiers. IEEE Transactions on Neural Networks 2005, 16(4):899-909.
14. Fred A.L.N., Jain A.K. Combining multiple clusterings using evidence accumulation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005, 27(6):835-850.
15. J.H. Friedman, Flexible metric nearest neighbor classification, Technical Report, Statistics Department, Stanford University, 1994.
16. Fukunaga K., Flick T.E. An optimal global nearest neighbor metric. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(3):314-318.
17. Hastie T., Tibshirani R. Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 1996, 18(6):607-616.
18. S.C.H. Hoi, W. Liu, M.R. Lyu, W.-Y. Ma, Learning distance metrics with contextual constraints for image retrieval, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Oregon State University, Corvallis, USA, 2006, pp. 2072-2078.
19. S.C.H. Hoi, R. Jin, M.R. Lyu, Learning nonparametric kernel matrices from pairwise constraints, in: Proceedings of the 24th International Conference on Machine Learning (ICML), New York, USA, 2007, pp. 361-368.
20. S.C.H. Hoi, W. Liu, S.-F. Chang, Semi-supervised distance metric learning for collaborative image retrieval, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2008, pp. 1-7.
21. Hu G., Zhou S., Guan J., Hu X. Toward effective document clustering: a constrained k-means based approach. Information processing and management 2008, 44:1397-1409.
22. Jia Y., Nie F., Zhang C. Trace ratio problem revisited. IEEE Transactions on Neural Networks 2009, 20(4):729-735.
23. S. Kamvar, D. Klein, C.D. Manning, Spectral learning, in: Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI), 2003, pp. 561-566.
24. G. Karypis, CLUTO-a clustering toolkit, Technical Report 02-017, Department of Computer Science, University of Minnesota, 2002.
25. D. Klein, S.D. Kamvar, C. Manning, From instance-level constraints to space-level constraints: making the most of prior knowledge in data clustering, in: Proceedings of the 19th International Conference on Machine Learning (ICML), Sydney, Australia, 2002, pp. 307-314.
26. B. Kulis, M. Sustik, I. Dhillon, Learning low-rank kernel matrices, In: Proceedings of the 23th International Conference on Machine Learning (ICML), Pittsburg, PA, 2006, pp. 505-512.
27. Kulis B., Basu S., Dhillon I. Semi-supervised graph clustering: a kernel approach. Machine Learning 2009, 74(1):1-22.
28. Kulis B., Basu S., Dhillon I. Low-rank kernel learning with Bregman matrix divergences. Journal of Machine Learning Research 2009, 10:341-376.
29. Lanckriet G.R.G., Cristianini N., Bartlett P., Ghaoui L.E., Jordan M.I. Learning the kernel matrix with semi-definite programming. Journal of Machine Leaning Research 2004, 5(1):27-72.
30. M.H.C. Law, Clustering, dimensionality reduction, and side information, Ph.D. Dissertation, Michigan University, 2006.
31. H. Voss, Numerical methods for sparse nonlinear eigenvalue problem, Technical Report, Department of Mathematics, Hamburg University of Technology, 2003.
32. F. Li, J. Yang, J. Wang, A transductive framework of distance metric learning by spectral dimensionality reduction, in: Proceedings of the 24th International Conference on Machine Learning (ICML), Corvallis, OR, USA, 2007, pp. 513-520.
33. Z. Li, J. Liu, X. Tang, Pairwise constraint propagation by semidefinite programming for semi-supervised classification, in: Proceedings of the 25th International Conference on Machine Learning (ICML), 2008, pp. 576-583.
34. Z. Li, J. Liu, Constrained clustering by spectral kernel learning, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2009.
35. Z. Li, J. Liu, X. Tang, Constrained clustering via spectral regularization, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 421-428.
36. Lowe D.G. Similarity metric learning for a variable-kernel classifier. Neural Computation 1995, 7(1):72-85.
37. Shi J., Malik J. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(8):888-905.
38. Short R.D., Fukunaga K. The optimal distance measure for nearest neighbor classification. IEEE Transactions on Information Theory 1981, 27(5):622-627.
39. M. Soleymani Baghshah, S. Bagheri Shouraki, Semi-supervised metric learning using pairwise constraints, in: Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI), 2009, pp. 1217-1225.
40. Soleymani Baghshah M., Bagheri Shouraki S. Metric learning for semi-supervised clustering using pairwise constraints and the geometrical structure of data. Intelligent Data Analysis 2009, 13(6):887-899.
41. Soleymani Baghshah M., Bagheri Shouraki S. Kernel-based metric learning for semi-supervised clustering. Neurocomputing 2010, 73:1352-1361.
42. K.Q. Weinberger, Metric learning with convex optimization, Ph.D. Dissertation, University of Pennsylvania, 2007.
43. Wu L., Jin R., Hoi S.C.H., Zhu J., Yu N. Learning Bregman distance functions and its application for semi-supervised clustering. Advances in Neural Information Processing Systems 2009, MIT Press, Cambridge, MA, USA.
44. Xiang S., Nie F., Zhang C. Learning a Mahalanobis distance metric for data clustering and classification. Pattern Recognition 2008, 41(12):3600-3612.
45. E.P. Xing, A.Y. Ng, M.I. Jordan, S. Russell, Distance metric learning with application to clustering with side information, in: Proceedings of Advances in Neural Information Processing Systems, vol. 15, MIT Press, Cambridge, MA, USA, 2003, pp. 505-512.
46. L. Yang, R. Jin, Distance metric learning: a comprehensive survey, Technical Report, Michigan State University, 2006.
47. Yeung D.Y., Chang H. Extending the relevant component analysis algorithm for metric learning using both positive and negative equivalence constraints. Pattern Recognition 2006, 39:1007-1010.
48. Yeung D.Y., Chang H. A Kernel approach for semi-supervised metric learning. IEEE Transactions on Neural Networks 2007, 18(1):141-149.
49. Yeung D.Y., Chang H., Dai G. A scalable kernel-based semi-supervised metric learning algorithm with out-of-sample generation ability. Neural Computation 2008, 20(11):2839-2861.
50. Yan B., Domeniconi C. Kernel optimization using pairwise constraints for semi-supervised clustering. Advances in Neural Information Processing Systems 2009, MIT Press, Cambridge, MA, USA.
51. Yin X., Chen S., Hu E., Zhang D. Semi-supervised clustering with metric learning: an adaptive kernel method. Pattern Recognition 2010, 43:1320-1333.
52. J. Zhuang, I.W. Tsang, S.C.H. Hoi, Simple NPKL: simple non-parametric kernel learning, in: Proceedings of the 26th International Conference on Machine Learning (ICML), Montreal, Canada, 2009.