Web and personal image annotation by mining label correlation with relaxed visual graph embedding

IEEE Transactions on Image Processing

Volumn 21, Issue 3, 2012, Pages 1339-1351

  Yang, Yi ^a   Wu, Fei ^b   Nie, Feiping ^c   Shen, Heng Tao ^d   Zhuang, Yueting ^b   Hauptmann, Alexander G ^a  

^a Carnegie Mellon University   (United States)

^b ZHEJIANG UNIVERSITY   (China)

^c The University of Texas at Arlington   (United States)

^d UNIVERSITY OF QUEENSLAND   (Australia)

Author keywords

Label correlation mining; multilabel learning; personal album labeling; semisupervised learning; web image annotation

Indexed keywords

AUTOMATIC IMAGE ANNOTATION; CORRELATION MINING; DIGITAL IMAGE; EFFECTIVE MECHANISMS; EIGEN DECOMPOSITION; EVALUATION METRICS; GRAPH MODEL; IMAGE ANNOTATION; IMAGE CATEGORIZATION; IMAGE DATABASE; IMAGE DATASETS; INDUCTIVE ALGORITHMS; LABELED AND UNLABELED DATA; MATRIX; MULTI-LABEL; MULTIPLE LABELS; OPTIMAL SOLUTIONS; PERSONAL IMAGE; SEMI-SUPERVISED LEARNING; SINGLE IMAGES; STRAIGHT-FORWARD METHOD; VISUAL FEATURE; VISUAL GRAPH; VISUAL SIMILARITY; WEB IMAGES;

ALGORITHMS; IMAGE ANALYSIS;

IMAGE RETRIEVAL;

EID: 84863175756     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2011.2169269     Document Type: Article

References (47)

1. R. Datta, D. Joshi, J. Li, and J. Z. Wang, "Image retrieval: Ideas, influences, and trends of the new age," ACM Comput. Surv., vol. 40, no. 2, pp. 1-60, 2008.
2. M. S. Lew, N. Sebe, C. Djeraba, and R. Jain, "Content-based multimedia information retrieval: State of the art and challenges," ACM Trans. Multimedia Comput., Commun. Appl., vol. 2, no. 1, pp. 1-19, Feb. 2006. (Pubitemid 43890693)
3. R. Bergman and H. Nachlieli, "Perceptual segmentation: Combining image segmentation with object tagging," IEEE Trans. Image Process., vol. 20, no. 6, pp. 1668-1681, Jun. 2011.
4. L. Cao, J. Luo, H. S. Kautz, and T. S. Huang, "Image annotation within the context of personal photo collections using hierarchical event and scene models," IEEE Trans. Multimedia, vol. 11, no. 2, pp. 208-219, Feb. 2009.
5. G. Carneiro, A. B. Chan, P. J. Moreno, and N. Vasconcelos, "Supervised learning of semantic classes for image annotation and retrieval," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 3, pp. 394-410, Mar. 2007. (Pubitemid 46336416)
6. J. Liu, B. Wang, M. Li, Z. Li, W. Ma, H. Lu, and S. Ma, "Dual crossmedia relevance model for image annotation," in Proc. ACM Multimedia, 2007, pp. 605-614.
7. X.-J. Wang, L. Zhang, X. Li, and W.-Y. Ma, "Annotating images by mining image search results," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 1919-1932, Nov. 2008.
8. Y. Yang, Y. Zhuang, D. Xu, Y. Pan, D. Tao, and S. J. Maybank, "Retrieval based interactive cartoon synthesis via unsupervised bi-distance metric learning," in Proc. ACM Multimedia, 2009, pp. 311-320.
9. L.Wu, S. Hoi, R. Jin, J. Zhu, and N. Yu, "Distance metric learning from uncertain side information with application to automated photo tagging," ACM Trans. Intell. Syst. Technol., vol. 2, no. 2, pp. 13:1-13:28, Feb. 2011.
10. L. Cao, J. Yu, J. Luo, and T. S. Huang, "Enhancing semantic and geographic annotation of web images via logistic canonical correlation regression," in Proc. ACM Multimedia, 2009, pp. 125-134.
11. H. Hotelling, "Relations between two sets of variates," Biometrika, vol. 28, no. 3/4, pp. 321-377, Dec. 1936.
12. Z. Ma, Y. Yang, F. Nie, J. Uijlings, and N. Sebe, "Exploiting the entire feature space with sparsity for automatic image annotation," in Proc. ACM Multimedia, 2011.
13. Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan, "A multimedia retrieval framework based on semi-supervised ranking and relevance feedback," IEEE Trans. Pattern Anal. Mach. Intell., 2011, to be published.
14. X. Zhu and A. Goldberg, Introduction to Semi-Supervised Learning. San Rafael, CA: Morgan & Claypool, 2009.
15. D. Cai, X. He, and J. Han, "Semi-supervised discriminant analysis," in Proc. ICCV, 2007, pp. 1-7.
16. G. Chen, Y. Song, F. Wang, and C. Zhang, "Semi-supervised multilabel learning by solving a sylvester equation," in Proc. SDM, 2008, pp. 410-419.
17. Z. Zha, T. Mei, J.Wang, Z.Wang, and X.-S. Hua, "Graph-based semisupervised learning with multiple labels," J. Vis. Commun. Image Represent., vol. 20, no. 2, pp. 97-103, 2009.
18. Y. Zhuang, Y. Yang, and F. Wu, "Mining semantic correlation of heterogeneous multimedia data for cross-media retrieval," IEEE Trans. Multimedia, vol. 10, no. 2, pp. 221-229, Feb. 2008.
19. M. Wang, X.-S. Hua, J. Tang, and R. Hong, "Beyond distance measurement: Constructing neighborhood similarity for video annotation," IEEE Trans. Multimedia, vol. 11, no. 3, pp. 465-476, Apr. 2009.
20. M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples," J. Mach. Learn. Res., vol. 7, pp. 2399-2434, Dec. 2006. (Pubitemid 44708005)
21. S. Ji and J. Ye, "Linear dimensionality reduction for multi-label classification," in Proc. IJCAI, 2009, pp. 1077-1082.
22. R. Jin, S. Wang, and Z.-H. Zhou, "Learning a distance metric from multi-instance multi-label data," in Proc. CVPR, 2009, pp. 896-902.
23. T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y.-T. Zheng, "NUSWIDE: A real-world web image database from national university of singapore," in Proc. CIVR, 2009, pp. 1-9.
24. H. Li, M. Wang, and X.-S. Hua, "MSRA-MM 2.0: A large-scale web multimedia dataset," in Proc. ICDMW, 2009, pp. 164-169.
25. A. Loui, J. Luo, S.-F. Chang, D. Ellis, W. Jiang, L. Kennedy, K. Lee, and A. Yanagawa, "Kodak's consumer video benchmark data set: Concept definition and annotation," in Proc. MIR, 2007, pp. 245-254.
26. R. Ando and T. Zhang, "A framework for learning predictive structures from multiple tasks and unlabeled data," J. Mach. Learn. Res., vol. 6, pp. 1817-1853, Dec. 2005.
27. Y. Amit, M. Fink, N. Srebro, and S. Ullman, "Uncovering shared structures in multiclass classification," in Proc. ICML, 2007, pp. 17-24.
28. J. Chen, L. Tang, J. Liu, and J. Ye, "A convex formulation for learning shared structures from multiple tasks," in Proc. ICML, 2009, pp. 137-144.
29. S. Ji, L. Tang, S. Yu, and J. Ye, "A shared-subspace learning framework for multi-label classification," ACMTrans. Knowl. Discovery Data, vol. 4, no. 2, pp. 1-29, May 2010.
30. D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Schölkopf, "Learning with local and global consistency," in Proc. NIPS, 2003, pp. 1-8.
31. Y. Yang, Y. Zhuang, D. Tao, D. Xu, J. Yu, and J. Luo, "Recognizing cartoon image gestures for retrieval and interactive cartoon clip synthesis," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 12, pp. 1745-1756, Dec. 2010.
32. H. Tong, J. He, M. Li, W.-Y. Ma, H.-J. Zhang, and C. Zhang, "Manifold-ranking-based keyword propagation for image retrieval," EURASIP J. Appl. Signal Process., vol. 2006, p. 190, Jan. 2006.
33. Y. Yang, Y. Zhuang, F. Wu, and Y. Pan, "Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval," IEEE Trans. Multimedia, vol. 10, no. 3, pp. 437-446, Apr. 2008. (Pubitemid 351459506)
34. Y. Yang, D. Xu, F. Nie, J. Luo, and Y. Zhuang, "Ranking with local regression and global alignment for cross media retrieval," in Proc. ACM Multimedia, 2009, pp. 175-184.
35. X. Zhu, Z. Ghahramani, and J. Lafferty, "Semi-supervised learning using Gaussian fields and harmonic functions," in Proc. ICML, 2003, pp. 912-919.
36. D. Spielman, "Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems," in Proc. ACM Symp. Theory Comput., 2004, pp. 81-90.
37. R. Fergus, Y. Weiss, and A. Torralba, "Semi-supervised learning in gigantic image collections," in Proc. NIPS, 2009, pp. 1-9.
38. F. Nie, D. Xu, I. W.-H. Tsang, and C. Zhang, "Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction," IEEE Trans. Image Process., vol. 19, no. 7, pp. 1921-1932, Jul. 2010.
39. Y. Yang, F. Nie, S. Xiang, Y. Zhuang, and W.Wang, "Local and global regressive mapping for manifold learning with out-of-sample extrapolation," in Proc. AAAI, 2010, pp. 649-654.
40. G. Fung and O. L. Mangasarian, "Multicategory proximal support vector machine classifiers," Mach. Learn., vol. 59, no. 1/2, pp. 77-97, May 2005. (Pubitemid 40890413)
41. G. H. Golub and C. F. Loan, Matrix Computations, 3rd ed. Baltimore, MD: The Johns Hopkins Univ. Press, 1996.
42. C. Zhang, F. Nie, and S. Xiang, "A general kernelization framework for learning algorithms based on kernel PCA," Neurocomputing, vol. 73, no. 4-6, pp. 959-967, Jan. 2010.
43. K. Fukunaga, Introduction to Statistical Pattern Recognition. Baltimore, MD: Academic, 1990.
44. Y. Yang, D. Xu, F. Nie, S.Yan, and Y. Zhuang, "Image clustering using local discriminant models and global integration," IEEE Trans. Image Process., vol. 19, no. 10, pp. 2761-2773, Oct. 2010.
45. T. Fawcett, "An introduction to ROC analysis," Pattern Recognit. Lett., vol. 27, no. 8, pp. 861-874, Jun. 2006.
46. D. Hand and R. Till, "A simple general ization of the area under the ROC curve for multiple class classification problems," Mach. Learn., vol. 45, no. 2, pp. 171-186, Nov. 2001. (Pubitemid 33635984)
47. D. D. Lewis, Y. Yang, T. G. Rose, and F. Li, "RCV1:A newbenchmark collection for text categorization research," J. Mach. Learn. Res., vol. 5, pp. 361-397, Dec. 2004.