Robust and scalable graph-based semisupervised learning

Proceedings of the IEEE

Volumn 100, Issue 9, 2012, Pages 2624-2638

  Liu, Wei ^a   Wang, Jun ^b   Chang, Shih Fu ^a  

^a Columbia University ^*  (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Anchor graphs; graph based semisupervised learning (GSSL); image annotation; image classification; image search; label diagnosis; large scale; noisy labels

Indexed keywords

COMPUTER AIDED DIAGNOSIS; IMAGE ANALYSIS; IMAGE ANNOTATION; IMAGE CLASSIFICATION; SUPERVISED LEARNING;

EMERGING APPLICATIONS; HIGH DIMENSIONAL SPACES; IMAGE SEARCH; LARGE SCALE; MANIFOLD STRUCTURES; NOISY LABELS; SEMI- SUPERVISED LEARNING; UNLABELED DATA;

GRAPHIC METHODS;

EID: 84865425579     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2012.2197809     Document Type: Article

References (66)

1. A. Azran, "The rendezvous algorithm: Multiclass semi-supervised learning with Markov random walks," in Proc. Int. Conf. Mach. Learn., 2007, pp. 49-56.
2. M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples," J. Mach. Learn. Res., vol. 7, pp. 2399-2434, Nov. 2006. (Pubitemid 44708005)
3. A. Blum and S. Chawla, "Learning from labeled and unlabeled data using graph mincuts," in Proc. Int. Conf. Mach. Learn., 2001, pp. 19-26.
4. A. Blum, J. D. Lafferty, M. R. Rwebangira, and R. Reddy, "Semi-supervised learning using randomized mincuts," in Proc. Int. Conf. Mach. Learn., 2004, pp. 13-20.
5. J. Bu, S. Tan, C. Chen, C. Wang, H. Wu, L. Zhang, and X. He, "Music recommendation by unified hypergraph: Combining social media information and music content," in Proc. ACM Int. Conf. Multimedia, 2010, pp. 391-400.
6. O. Chapelle, B. Schölkopf, and A. Zien, Eds., Semi-Supervised Learning. Cambridge, MA: MIT Press, 2006.
7. O. Chapelle, V. Sindhwani, and S. S. Keerthi, 'Optimization techniques for semi-supervised support vector machines," J. Mach. Learn. Res., vol. 9, pp. 203-233, Feb. 2008. (Pubitemid 351469022)
8. J. Chen, H.-R. Fang, and Y. Saad, "Fast approximate kNN graph construction for high dimensional data via recursive lanczos bisection," J. Mach. Learn. Res., vol. 10, pp. 1989-2012, Sep. 2009.
9. T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng, "NUS-wide: A real-world web image database from National University of Singapore," in Proc. ACM Int. Conf. Image Video Retrieval, 2009, DOI: 10.1145/1646396. 1646452.
10. F. Chung, "Spectral graph theory," CBMS Regional Conference Series in Mathematics. Providence, RI: AMS, 1997, No. 92.
11. O. Delalleu, Y. Bengio, and N. L. Roux, "Efficient non-parametric function induction in semi-supervised learning," in Proc. Int. WorkshopArtif. Intell. Stat., 2005, pp. 96-103.
12. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 248-255.
13. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, The PASCAL Visual Object Classes Challenge 2008 (VOC2008) Results. [Online]. Available: http://www. pascal-network.org/challenges/VOC/voc2008/workshop/index. html
14. L. Fei-Fei, R. Fergus, and P. Perona, "Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories," Comput. Vis. Image Understand., vol. 106, no. 1, pp. 59-70, Apr. 2007. (Pubitemid 46528761)
15. R. Fergus, L. Fei-Fei, P. Perona, and A. Zisserman, "Learning object categories from internet image searches," Proc. IEEE, vol. 98, no. 8, pp. 1453-1466, Aug. 2010.
16. R. Fergus, Y. Weiss, and A. Torralba, "Semi-supervised learning in gigantic image collections," Advances in Neural Information Processing Systems, vol. 22. Cambridge, MA: MIT Press, 2010, pp. 522-530.
17. A. Gionis, P. Indyk, and R. Motwani, "Similarity search in high dimensions via hashing," in Proc. Int. Conf. Very Large Data Bases, 1999, pp. 518-529.
18. A. Guillory and J. Bilmes, "Label selection on graphs," Advances in Neural Information Processing Systems, vol. 22. Cambridge, MA: MIT Press, 2010, pp. 691-699.
19. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. New York: Springer-Verlag, 2009.
20. W. H. Hsu, L. S. Kennedy, and S.-F. Chang, "Video search reranking via information bottleneck principle," in Proc. ACM Int. Conf. Multimedia, 2006, pp. 35-44. (Pubitemid 47113499)
21. W. H. Hsu, L. S. Kennedy, and S.-F. Chang, "Reranking methods for visual search," IEEE MultiMedia, vol. 14, no. 3, pp. 14-22, Jul.-Sep. 2007. (Pubitemid 47322511)
22. T. Jebara, J. Wang, and S.-F. Chang, "Graph construction and b-matching for semi-supervised learning," in Proc. Int. Conf. Mach. Learn., 2009, pp. 441-448.
23. Y. Jing and S. Baluja, "Visualrank: Applying pagerank to large-scale image search," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 1877-1890, Nov. 2008.
24. T. Joachims, "Transductive inference for text classification using support vector machines," in Proc. Int. Conf. Mach. Learn., 1999, pp. 200-209.
25. T. Joachims, "Transductive learning via spectral graph partitioning," in Proc. Int. Conf. Mach. Learn., 2003, pp. 290-297.
26. T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Silverman, and A. Y. Wu, "An efficient k-means clustering algorithm: Analysis and implementation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 881-892, Jul. 2002. (Pubitemid 34835464)
27. L. S. Kennedy, S.-F. Chang, and I. Kozintsev, "To search or to label?: Predicting the performance of search-based automatic image classifiers," in Proc. ACM SIGMM Int. Workshop Multimedia Inf. Retrieval, 2006, pp. 249-258. (Pubitemid 47168222)
28. J. M. Kleinberg, A. Slivkins, and T. Wexler, "Triangulation and embedding using small sets of beacons," in Proc. 45th Symp. Found. Comput. Sci., 2004, pp. 444-453. (Pubitemid 40575305)
29. J. Krapac, M. Allan, J. Verbeek, and F. Juried, "Improving web image search results using query-relative classifiers," inProc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 1094-1101.
30. A. Krizhevsky, "Learning multiple layers of features from tiny images," Univ. Toronto, Toronto, ON, Canada, Tech. Rep., Apr. 2009.
31. B. Kveton, M. Valko, A. Rahimi, and L. Huang, "Semi-supervised learning with max-margin graphcuts," inProc Int. Conf.Artif. Intell. Stat., 2010, pp. 421-428.
32. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
33. D. Liu, S. Yan, Y. Rui, and H.-J. Zhang, "Unified tag analysis with multi-edge graph," in Proc. ACM Int. Conf. Multimedia, 2010, pp. 25-34.
34. W. Liu. [Online]. Available: http://www.ee. columbia.edu/dvmm/downloads/ WeiGraphConstructCode2011/dlform.htm
35. W. Liu and S.-F. Chang, "Robust multi-class transductive learning with graphs," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 381-388.
36. W. Liu, J. He, and S.-F. Chang, "Large graph construction for scalable semi-supervised learning," in Proc. Int. Conf. Mach. Learn., 2010, pp. 679-686.
37. W. Liu, J. Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang, "Supervised hashing with kernels," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012.
38. W. Liu, J. Wang, S. Kumar, and S.-F. Chang, "Hashing with graphs," in Proc. Int. Conf. Mach. Learn., 2011, pp. 1-8.
39. H. Q. Min and V. Sindhwani, "Vector-valued manifold regularization," in Proc. Int. Conf. Mach. Learn., 2011, pp. 57-64.
40. A. Oliva and A. Torralba, "Modeling the shape of the scene: A holistic representation of the spatial envelope," Int. J. Comput. Vis., vol. 42, no. 3, pp. 145-175, May 2001. (Pubitemid 32680801)
41. H. A. Sanchez, J. M. Sotoca, and A. M. Uso, "Semi-supervised learning from a translation model between data distributions," in Proc. Int. Joint Conf. Artif. Intell., 2011, pp. 1165-1170.
42. V. Sindhwani, P. Niyogi, and M. Belkin, "Beyond the point cloud: From transductive to semi-supervised learning," in Proc. Int. Conf. Mach. Learn., 2005, pp. 824-831.
43. A. F. Smeaton, P. Over, and W. Kraaij, "Evaluation campaigns and TRECVid," in Proc. ACM SIGMM Int. Workshop Multimedia Inf. Retrieval, 2006, pp. 321-330. (Pubitemid 47168230)
44. D. E. Sturim, D. A. Reynolds, E. Singer, and J. P. Campbel, "Speaker indexing in large audio databases using anchor models," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2001, pp. 429-432. (Pubitemid 32839279)
45. M. Szummer and T. Jaakkola, "Partially labeled classification with Markov random walks," Advances in Neural Information Processing Systems, vol. 14. Cambridge, MA: MIT Press, 2002, pp. 945-952.
46. X. Tian, G. Gasso, and S. Canu, "A multi-kernel framework for inductive semi-supervised learning," in Proc. 19th Eur. Symp. Artif Neural Netw., 2011, pp. 65-70.
47. S. Tong and D. Koller, "Support vector machine active learning with applications to text classification," J. Mach. Learn. Res., vol. 2, pp. 45-66, Nov. 2001.
48. A. Torralba, R. Fergus, and W. T. Freeman, "80 million tiny images: A large dataset for non-parametric object and scene recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 1958-1970, Nov. 2008.
49. I. W. Tsang and J. T. Kwok, "Large-scale sparsified manifold regularization," Advances in Neural Information Processing Systems, vol. 19. Cambridge, MA: MIT Press, 2007, pp. 1401-1408.
50. F. Wang and C. Zhang, "Label propagation through linear neighborhoods," IEEE Trans. Knowl. Data Eng., vol. 20, no. 1, pp. 55-67, Jan. 2008.
51. J. Wang, "Semi-supervised learning for scalable and robust visual search," Ph.D. dissertation, Grad. Schl. Arts Sci., Columbia Univ., New York, NY, 2011.
52. J. Wang, T. Jebara, and S.-F. Chang, "Graph transduction via alternating minimization," in Proc. Int. Conf. Mach. Learn., 2008, pp. 1144-1151.
53. J. Wang, Y.-G. Jiang, and S.-F. Chang, "Label diagnosis through self tuning for web image search," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1390-1397.
54. J. Wang, S. Kumar, and S.-F. Chang, "Semi-supervised hashing for large scale search," IEEE Trans. Pattern Anal. Mach. Intell., 2012, DOI: 10.1109/TPAMI.2012.48.
55. Y. Weiss, A. Torralba, and R. Fergus, "Spectral hashing," Advances in Neural Information Processing Systems, vol. 21. Cambridge, MA: MIT Press, 2009, pp. 1753-1760.
56. M. Wu and B. Scḧolkopf, "Transductive classification via local learning regularization," in Proc. Int. Conf. Artif. Intell. Stat., 2007, pp. 628-635.
57. Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan, "A multimedia retrieval framework based on semi-supervised ranking and relevance feedback," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 723-742, Apr. 2012.
58. Y. Yang, F. Wu, F. Nie, H. T. Shen, Y. Zhuang, and A. G. Hauptman, "Web and personal image annotation by mining label correlation with relaxed visual graph embedding," IEEE Trans. Image Process., vol. 21, no. 3, pp. 1339-1351, Mar. 2012.
59. K. Yu, S. Yu, and V. Tresp, "Blockwise supervised inference on large graphs," in Proc. ICML Workshop Learning With Partially Classified Training Data, 2005, pp. 40-46.
60. K. Zhang, J. T. Kwok, and B. Parvin, "Prototype vector machine for large scale semi-supervised learning," in Proc. Int. Conf. Mach. Learn., 2009, pp. 1233-1240.
61. D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Scḧolkopf, "Learning with local and global consistency," Advances in Neural Information Processing Systems, vol. 16. Cambridge, MA: MIT Press, 2004, pp. 321-328.
62. X. Zhu. [Online]. Available: http://pages.cs. wisc.edu/~jerryzhu/pub/ harmonic-function.m
63. X. Zhu, "Semi-supervised learning literature survey," Comput. Sci., Univ. Wisconsin-Madison, Madison, WI, Tech. Rep. 1530, 2008.
64. X. Zhu, Z. Ghahramani, and J. D. Lafferty, "Semi-supervised learning using Gaussian fields and harmonic functions," in Proc. Int. Conf. Mach. Learn., 2003, pp. 912-919.
65. X. Zhu, J. Lafferty, and Z. Ghahramani, "Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions," in Proc. ICML Workshop Continuum From Labeled to Unlabeled Data in Machine Learning and Data Mining, 2003, pp. 58-65.
66. X. Zhu and J. D. Lafferty, "Harmonic mixtures: Combining mixture models and graph-based methods for inductive and scalable semi-supervised learning," in Proc. Int. Conf. Mach. Learn., 2005, pp. 1052-1059.