Learning on Big Graph: Label Inference and Regularization with Anchor Hierarchy

IEEE Transactions on Knowledge and Data Engineering

Volumn 29, Issue 5, 2017, Pages 1101-1114

  Wang, Meng ^a   Fu, Weijie ^a   Hao, Shijie ^a   Liu, Hengchang ^b   Wu, Xindong ^a,c  

^a HEFEI UNIVERSITY OF TECHNOLOGY   (China)

^b UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^c Informatics Research Institute   (United States)

Author keywords

graph based learning; label inference; label smoothness regularization; Semi supervised learning

Indexed keywords

ANCHORS; MATRIX ALGEBRA; NEAREST NEIGHBOR SEARCH; SUPERVISED LEARNING;

CLASSIFICATION TASKS; COARSE TO FINE; COMPUTATIONAL COSTS; EFFECTIVENESS AND EFFICIENCIES; GRAPH-BASED LEARNING; LAPLACIAN MATRICES; MULTIPLE LAYERS; SEMI- SUPERVISED LEARNING;

GRAPHIC METHODS;

EID: 85018473108     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2017.2654445     Document Type: Article

References (52)

1. M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples," J. Mach. Learn. Res., vol. 7, no. 11, pp. 2399-2434, 2006.
2. A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in Proc. Conf. Comput. Learn. Theory, 1998, pp. 92-100.
3. D. Cai and X. Chen, "Large scale spectral clustering with landmark-based representation," IEEE Trans. Cybern., vol. 45, no. 8, pp. 1669-1680, Aug. 2015.
4. V. Castelli and T. M. Cover, "On the exponential value of labeled samples," Pattern Recog. Lett., vol. 16, no. 1, pp. 105-111, 1995.
5. L. Chen, I. W. Tsang, and D. Xu, "Laplacian embedded regression for scalable manifold regularization," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 6, pp. 902-915, Jun. 2012.
6. B. Cheng, J. Yang, S. Yan, Y. Fu, and T. S. Huang, "Learning with l1-graph for image analysis," IEEE Trans. Image Process., vol. 19, no. 4, pp. 858-866, Apr. 2010.
7. C. Deng, R. Ji, W. Liu, D. Tao, and X. Gao, "Visual reranking through weakly supervised multi-graph learning," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 2600-2607.
8. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, "Liblinear: A library for large linear classification," J. Mach. Learn. Res., vol. 9, no. 9, pp. 1871-1874, 2008.
9. R. Fergus, Y. Weiss, and A. Torralba, "Semi-supervised learning in gigantic image collections," in Proc. Conf. Advances Neural Inf. Proc. Syst., 2009, pp. 522-530.
10. P. Foggia, G. Percannella, and M. Vento, "Graph matching and learning in pattern recognition in the last 10 years," Int. J. Pattern Recog. Artif. Intell., vol. 28, no. 1, pp. 178-215, 2014.
11. P. W. Frey and D. J. Slate, "Letter recognition using holland-style adaptive classifiers," Mach. Learn., vol. 6, no. 2, pp. 161-182, 1991.
12. L. Gao, J. Song, F. Nie, Y. Yan, N. Sebe, and H. T. Shen, "Optimal graph learning with partial tags and multiple features for image and video annotation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2015, pp. 4371-4379.
13. M. Guillaumin, J. Verbeek, and C. Schmid, "Multimodal semisupervised learning for image classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 902-909.
14. T. J. Hastie, R. J. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Berlin, Germany: Springer, 2010.
15. M. Hein and S. Setzer, "Beyond spectral clustering-tight relaxations of balanced graph cuts," in Proc. Conf. Advances Neural Inf. Process. Syst., 2011, pp. 2366-2374.
16. C.-J. Hsieh, S. Si, and I. S. Dhillon, "A divide-and-conquer solver for kernel support vector machines." in Proc. Int. Conf. Mach. Learn., 2014, pp. 566-574.
17. S. Huang, M. Elhoseiny, A. Elgammal, and D. Yang, "Learning hypergraph-regularized attribute predictors," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2015, pp. 409-417.
18. T. Joachims, "Transductive inference for text classification using support vector machines," in Proc. Int. Conf. Mach. Learn., 1999, pp. 200-209.
19. M. Karlen, J. Weston, A. Erkan, and R. Collobert, "Large scale manifold transduction," in Proc. Int. Conf. Mach. Learn., 2008, pp. 448-455.
20. S. Kim and S. Choi, "Multi-view anchor graph hashing," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2013, pp. 3123-3127.
21. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
22. M. Li, J. T.-Y. Kwok, and B. Lü, "Making large-scale nyström approximation possible," in Proc. Int. Conf. Mach. Learn., 2010, pp. 631-638.
23. W. Liu, J. He, and S.-F. Chang, "Large graph construction for scalable semi-supervised learning," in Proc. Int. Conf. Mach. Learn., 2010, pp. 679-686.
24. W. Liu, J. Wang, and S.-F. Chang, "Robust and scalable graphbased semisupervised learning," Proc. IEEE, vol. 100, no. 9, pp. 2624-2638, Sep. 2012.
25. W. Liu, J. Wang, S. Kumar, and S.-F. Chang, "Hashing with graphs," in Proc. Int. Conf. Mach. Learn., 2011, pp. 1-8.
26. L. Bottou, "Large-scale kernel machines," MIT press, 2007.
27. D. Machin, Introduction to Multimodal Analysis. London, U.K.: Bloomsbury Publishing, 2016.
28. S. Melacci and M. Belkin, "Laplacian support vector machines trained in the primal," J. Mach. Learn. Res., vol. 12, no. 5, pp. 1149-1184, 2009.
29. M. Muja and D. G. Lowe, "Scalable nearest neighbor algorithms for high dimensional data," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 11, pp. 2227-2240, Nov. 2014.
30. M. Norouzi, A. Punjani, and D. J. Fleet, "Fast exact search in hamming space with multi-index hashing," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 6, pp. 1107-1119, Jun. 2014.
31. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
32. Z. Tian and R. Kuang, "Global linear neighborhoods for efficient label propagation," in Proc. SIAM Int. Conf. Data Mining, 2012, pp. 863-872.
33. I. W. Tsang, J. T. Kwok, and P.-M. Cheung, "Core vector machines: Fast SVM training on very large data sets," J. Mach. Learn. Res., vol. 6, no. 1, pp. 363-392, 2005.
34. F. Wang and C. Zhang, "Label propagation through linear neighborhoods," IEEE Trans. Knowl. Data Eng., vol. 20, no. 1, pp. 55-67, Jan. 2008.
35. J. Wang, J. Wang, G. Zeng, Z. Tu, R. Gan, and S. Li, "Scalable k-NN graph construction for visual descriptors," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 1106-1113.
36. M. Wang, W. Fu, S. Hao, D. Tao, and X. Wu, "Scalable semi-supervised learning by efficient anchor graph regularization," IEEE Trans. Knowl. Data Eng., vol. 28, no. 7, pp. 1864-1877, Jul. 2016.
37. M. Wang, X.-S. Hua, R. Hong, J. Tang, G.-J. Qi, and Y. Song, "Unified video annotation via multigraph learning," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 5, pp. 733-746, May 2009.
38. B. Xu, J. Bu, C. Chen, C. Wang, D. Cai, and X. He, "EMR: A scalable graph-based ranking model for content-based image retrieval," IEEE Trans. Knowl. Data Eng., vol. 27, no. 1, pp. 102-114, Jan. 2015.
39. Y. Yang, Y. Luo, W. Chen, F. Shen, J. Shao, and H. T. Shen, "Zeroshot hashing via transferring supervised knowledge," in Proc. ACM Conf. Multimedia, 2016, pp. 1286-1295.
40. Y. Yang, F. Shen, H. T. Shen, H. Li, and X. Li, "Robust discrete spectral hashing for large-scale image semantic indexing," IEEE Trans. Big Data, vol. 1, no. 4, pp. 162-171, Dec. 2015.
41. Y. Yang, Y. Yang, H. T. Shen, Y. Zhang, X. Du, and X. Zhou, "Discriminative nonnegative spectral clustering with out-of-sample extension," IEEE Trans. Knowl. Data Eng., vol. 25, no. 8, pp. 1760-1771, Aug. 2013.
42. G. Yu, G. Zhang, Z. Zhang, Z. Yu, and L. Deng, "Semi-supervised classification based on subspace sparse representation," Knowl. Inf. Syst., vol. 43, no. 1, pp. 81-101, 2015.
43. R. Yuster and U. Zwick, "Fast sparse matrix multiplication," ACM Trans. Algorithms, vol. 1, no. 1, pp. 2-13, 2005.
44. L. Zelnik-Manor and P. Perona, "Self-tuning spectral clustering," in Proc. Conf. Advances Neural Inf. Process. Syst., 2005, pp. 1601-1608.
45. K. Zhang, J. T. Kwok, and B. Parvin, "Prototype vector machine for large scale semi-supervised learning," in Proc. Int. Conf. Mach. Learn., 2009, pp. 1233-1240.
46. K. Zhang, L. Lan, J. T. Kwok, S. Vucetic, and B. Parvin, "Scaling up graph-based semi-supervised learning via prototype vector machines," IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 3, pp. 444-457, Mar. 2015.
47. D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Schölkopf, "Learning with local and global consistency," in Proc. Conf. Advances Neural Inf. Process. Syst., 2004, pp. 321-328.
48. D. Zhou, J. Huang, and B. Schölkopf, "Learning with hypergraphs: Clustering, classification, and embedding," in Proc. Conf. Advances Neural Inf. Process. Syst., 2006, pp. 1601-1608.
49. X. Zhu, "Semi-supervised learning literature survey," Comput. Sci., Univ. Wisconsin-Madison, Madison, WI, USA, Tech. Rep. 1530, 2005.
50. X. Zhu, Z. Ghahramani, and J. Lafferty, "Semi-supervised learning using gaussian fields and harmonic functions," in Proc. Int. Conf. Mach. Learn., 2003, pp. 912-919.
51. X. Zhu and A. B. Goldberg, "Introduction to semi-supervised learning," Synthesis Lectures Artif. Intell. Mach. Learn., vol. 3, no. 1, pp. 1-130, 2009.
52. X. Zhu, J. Kandola, Z. Ghahramani, and J. D. Lafferty, "Nonparametric transforms of graph kernels for semi-supervised learning," in Proc. Conf. Advances Neural Inf. Process. Syst., 2004, pp. 1641-1648.