Visual-textual joint relevance learning for tag-based social image search

IEEE Transactions on Image Processing

Volumn 22, Issue 1, 2013, Pages 363-376

  Gao, Yue ^a   Wang, Meng ^b   Zha, Zheng Jun ^c   Shen, Jialie ^d   Li, Xuelong ^e   Wu, Xindong ^b,f  

^a TSINGHUA UNIVERSITY   (China)

^b HEFEI UNIVERSITY OF TECHNOLOGY   (China)

^c NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^d SINGAPORE MANAGEMENT UNIVERSITY   (Singapore)

^e XI AN INSTITUTE OF OPTICS AND PRECISION MECHANICS   (China)

^f University of Vermont   (United States)

Author keywords

Hypergraph learning; social image search; tag; visual textual

Indexed keywords

DATA SETS; HYPEREDGES; HYPERGRAPH; IMAGE SEARCH; LEARNING APPROACH; LEARNING PROCESS; RELEVANCE LEARNING; RESEARCH EFFORTS; RESEARCH FIELDS; SOCIAL MEDIA; TAG; TAG-BASED; TEXTUAL INFORMATION; VISUAL INFORMATION; VISUAL WORD; VISUAL-TEXTUAL;

IMAGE PROCESSING; MATHEMATICAL MODELS;

ESTIMATION;

ALGORITHM; ANIMAL; ARTICLE; FACTUAL DATABASE; HUMAN; IMAGE PROCESSING; METHODOLOGY; PHOTOGRAPHY; SOCIAL MEDIA;

ALGORITHMS; ANIMALS; DATABASES, FACTUAL; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; PHOTOGRAPHY; SOCIAL MEDIA;

EID: 84871666061     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2012.2202676     Document Type: Article

References (63)

1. Z.-J. Zha, L. Yang, T. Mei, M. Wang, and Z. Wang, "Visual query suggestion," in Proc. ACM Conf. Multimedia, 2009, pp. 15-24.
2. J. Shen, D. Tao, and X. Li, "QUC-tree: Integrating query context information for efficient music retrieval," IEEE Trans. Multimedia, vol. 11, no. 2, pp. 313-323, Feb. 2009.
3. J. Shen, D. Tao, and X. Li, "Modality mixture projections for semantic video event detection," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1587-1596, Nov. 2008.
4. M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, and S.-M. Hu, "Global contrast based salient region detection," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 409-416.
5. Y. Yang, Y. Zhuang, F. Wu, and Y. Pan, "Harmonizing hierarchical manifolds for multimedia document semantics understanding and crossmedia retrieval," IEEE Trans. Multimedia, vol. 10, no. 3, pp. 437-446, Apr. 2008. (Pubitemid 351459506)
6. Y. Gao, J. Tang, R. Hong, S. Yan, Q. Dai, N. Zhang, and T. Chua, "Camera constraint-free view-based 3-D object retrieval," IEEE Trans. Image Process., vol. 21, no. 4, pp. 2269-2281, Apr. 2012.
7. R. Hong, M. Wang, M. Xu, S. Yan, and T.-S. Chua, "Dynamic captioning: Video accessibility enhancement for hearing impairment," in Proc. ACM Conf. Multimedia, 2010, pp. 421-430.
8. J. Shen and Z. Cheng, "Personalized video similarity measure," ACM Multimedia Syst. J., vol. 15, no. 7, pp. 421-433, 2011.
9. D. Liu, M. Wang, X.-S. Hua, and H.-J. Zhang, "Semi-automatic tagging of photo albums via exemplar selection and tag inference," IEEE Trans. Multimedia, vol. 13, no. 1, pp. 82-91, Feb. 2011.
10. Y. Gao, J. Tang, R. Hong, Q. Dai, T. Chua, and R. Jain, "W2Go: A travel guidance system by automatic landmark ranking," in Proc. ACM Conf. Multimedia, 2010, pp. 123-132.
11. D. Liu, S. Yan, X.-S. Hua, and H.-J. Zhang, "Image retagging using collaborative tag propagation," IEEE Trans. Multimedia, vol. 13, no. 4, pp. 702-712, Aug. 2011.
12. B. Sigurbjornsson and R. V. Zwol, "Flickr tag recommendation based on collective knowledge," in Proc. ACM Int. World Wide Web Conf., 2008, pp. 327-336.
13. M. Kato, H. Ohshima, S. Oyama, and K. Tanaka, "Can social tagging improve web image search?" in Web Information Systems Engineering (Lecture Notes in Computer Science). vol. 5175. Berlin, Germany: Springer-Verlag, 2008, pp. 235-249.
14. L. Chen, D. Xu, W. Tsang, and J. Luo, "Tag-based web photo retrieval improved by batch mode re-tagging," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3440-3446.
15. K. Yang, M. Wang, X. S. Hua, and H. J. Zhang, "Social image search with diverse relevance ranking," in Proc. ACM Conf. Multimedia Model., 2009, pp. 174-184.
16. X. Li, C. G. M. Snoek, and M. Worring, "Learning tag relevance by neighbor voting for social image retrieval," in Proc. ACM Conf. Multimedia Inf. Retr., 2008, pp. 180-187.
17. D. Liu, M. Wang, L. Yang, X. S. Hua, and H. Zhang, "Tag quality improvement for social images," in Proc. IEEE Int. Conf. Multimedia, Jul. 2009, pp. 350-353.
18. Y. Gao, M. Wang, H. Luan, J. Shen, S. Yan, and D. Tao, "Tag-based social image search with visual-text joint hypergraph learning," in Proc. ACM Conf. Multimedia, 2011, pp. 1517-1520.
19. Z.-J. Zha, M. Wang, Y.-T. Zheng, Y. Yang, R. Hong, and T.-S. Chua, "Interactive video indexing with statistical active learning," IEEE Trans. Multimedia, vol. 14, no. 1, pp. 17-27, Feb. 2012.
20. A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-based image retrieval at the end of early years," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp. 1349-1380, Dec. 2000. (Pubitemid 32088465)
21. Y. Gao, M. Wang, Z. Zha, Q. Tian, Q. Dai, and N. Zhang, "Less is more: Efficient 3-D object retrieval with query view selection," IEEE Trans. Multimedia, vol. 11, no. 5, pp. 1007-1018, Oct. 2011.
22. Z.-J. Zha, L. Yang, T. Mei, M. Wang, Z. Wang, T.-S. Chua, and X.-S. Hua, "Visual query suggestion: Toward capturing user intent in internet image search," ACM Trans. Multimedia Comput., Commun. Appl., vol. 6, no. 3, pp. 1-13, 2010.
23. F. Li, Q. Dai, W. Xu, and G. Er, "Multilabel neighborhood propagation for region-based image retrieval," IEEE Trans. Multimedia, vol. 10, no. 8, pp. 1692-1604, Dec. 2008.
24. X. Li, C. G. Snoek, and M. Worring, "Unsupervised multi-feature tag relevance learning for social image retrieval," in Proc. ACM Int. Conf. Image Video Retrieval, 2010, pp. 10-17.
25. G. Zhu, S. Yan, and Y. Ma, "Image tag refinement toward low-rank, content-tag prior and error sparsity," in Proc. ACM Int. Conf. Multimedia, 2010, pp. 461-470.
26. X. Li, C. G. Snoek, and M. Worring, "Learning social tag relevance by neighbor voting," IEEE Trans. Multimedia, vol. 11, no. 7, pp. 1310-1322, Nov. 2009.
27. J. Bu, S. Tan, C. Chen, C. Wang, H. Wu, L. Zhang, and X. He, "Music recommendation by unified hypergraph: Combining social media information and music content," in Proc. ACM Int. Conf. Multimedia, 2010, pp. 391-400.
28. D. Liu, X.-S. Hua, L. Yang, M. Wang, and H.-J. Zhang, "Tag ranking," in Proc. ACM Conf. World Wide Web, 2019, pp. 351-360.
29. H. Xu, J. Wang, X. S. Hua, and S. Li, "Tag refinement by regularized LDA," in Proc. ACM Int. Conf. Multimedia, 2009, pp. 573-576.
30. A. Sun and S. S. Bhowmick, "Image tag clarity: In search of visualrep-resentative tags for social images," in Proc. SIGMM Workshop Social Media, 2009, pp. 19-26.
31. J. Fan, Y. Shen, N. Zhou, and Y. Gao, "Harvesting large-scale weakly-tagged image databases from the web," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 802-809.
32. D. Liu, X. S. Hua, M. Wang, and H. Zhang, "Boost search relevance for tag-based social image retrieval," in Proc. IEEE Int. Conf. Multimedia, Jul. 2009, pp. 1636-1639.
33. M. Wang, K. Yang, X. S. Hua, and H.-J. Zhang, "Toward a relevant and diverse search of social images," IEEE Trans. Multimedia, vol. 12, no. 8, pp. 829-842, Dec. 2010.
34. D. Zhou, J. Huang, and B. Schokopf, "Learning with hypergraphs: Clustering, classification, and embedding," in Proc. Adv. Neural Inf. Process. Syst., 2007, pp. 1-8.
35. Y. Huang, Q. Liu, S. Zhang, and D. Metaxas, "Video object segmentation by hypergraph cut," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1738-1745.
36. S. Xia and E. Hancock, "Clustering using class specific hyper graphs," in Proc. Int. Joint IAPR Workshop, Struct., Syntactic, Stat. Pattern Recognit., 2008, pp. 318-328.
37. R. Zass and A. Shashua, "Probabilistic graph and hypergraph matching," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
38. Y. Huang, Q. Liu, S. Zhang, and D. Metaxas, "Image retrieval via probabilistic hypergraph ranking," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3376-3383.
39. A. K. C. Wong and S. W. Lu, "Recognition and shape synthesis of 3-D objects based on attributed hypergraphs," IEEE Trans. Pattern Anal. Mach. Intell., vol. 11, no. 3, pp. 279-290, Mar. 1989.
40. S. Xia and E. Hancock, "3D object recognition using hyper-graphs and ranked local invariant features," in Proc. Int. Joint IAPR Struct., Syntactic, Stat. Pattern Recognit., 2008, pp. 117-126.
41. S. Xia and E. Hancock, "Learning large scale class specific hyper graphs for object recognition," in Proc. Int. Conf. Image Graph., 2008, pp. 366-371.
42. Q. Liu, Y. Huang, and D. Metaxas, "Hypergraph with sampling for image retrieval," Pattern Recognit., vol. 44, nos. 10-11, pp. 2255-2262, 2011.
43. X. Zhu, "Semi-supervised learning with graphs," Ph.D. dissertation, School Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, 2005.
44. Y.-G. Jiang and C.-W. Ngo, "Bag-of-visual-words expansion using visual relatedness for video indexing," in Proc. ACM SIGIR Conf. Res. Develop. Inf. Retr., 2008, pp. 769-770.
45. R. Ji, H. Yao, X. Sun, B. Zhong, P. Xu, and W. Gao, "Toward semantic embedding in visual vocabulary," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 918-925.
46. J. Yang, Y.-G. Jiang, A. G. Hauptmann, and C.-W. Ngo, "Evaluating bag-of-visual-words representations in scene classification," in Proc. ACM SIGMM Workshop Multimedia Inf. Retr., 2007, pp. 197-206.
47. R. Ji, L.-Y. Duan, J. Chen, H. Yao, J. Yuan, Y. Rui, and W. Gao, "Location discriminative vocabulary coding for mobile landmark search," Int. J. Comput. Vis., vol. 96, no. 3, pp. 290-314, 2012.
48. Y.-G. Jiang, C.-W. Ngo, and J. Yang, "Toward optimal bag-of-features for object categorization and semantic video retrieval," in Proc. ACM Int. Conf. Image Video Retr., 2007, pp. 494-501. (Pubitemid 350229664)
49. R. Ji, H. Yao, W. Liu, X. Sun, and Q. Tian, "Task-dependent visualcodebook compression," IEEE Trans. Image Process., vol. 21, no. 4, pp. 2282-2293, Apr. 2012.
50. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
51. R. Yan, A. Hauptmann, and R. Jin, "Multimedia search with pseudo relevance feedback," in Proc. Int. Conf. Image Video Retr., 2003, pp. 238-247.
52. J. Wang, Y. G. Jiang, and S. F. Chang, "Label diagnosis through self tuning for web image search," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1390-1397.
53. W. H. Hsu, L. S. Kennedy, and S. F. Chang, "Video search reranking through random walk over document-level context graph," in Proc. ACM Conf. Multimedia, 2007, pp. 971-980.
54. L. Wu, X. Hua, N. Yu, W. Ma, and S. Li, "Flickr distance," in Proc. ACM Conf. Multimedia, 2008, pp. 31-40.
55. T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y.-T. Zheng, "NUS-WIDE: A real-world web image database from National University of Singapore," in Proc. ACM Conf. Image Video Retr., Santorini, Greece, Jul. 2009, pp. 1-9.
56. D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Schokopf, "Learning with local and global consistency," in Proc. Adv. Neural Inf. Process. Syst., 2004, pp. 1-8.
57. M. Wang, X.-S. Hua, R. Hong, J. Tang, G.-J. Qi, and Y. Song, "Unified video annotation via multigraph learning," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 5, pp. 733-746, May 2009.
58. Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan, "A multimedia retrieval framework based on semi-supervised ranking and relevance feedback," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 723-742, Apr. 2012.
59. Y. Gao, M. Wang, D. Tao, R. Ji, and Q. Dai, "3D object retrieval and recognition with hypergraph analysis," IEEE Trans. Image Process., DOI: 10.1109/TIP.2012.2199502.
60. M. Wang, X.-S. Hua, J. Tang, and R. Hong, "Beyond distance measurement: Constructing neighborhood similarity for video annotation," IEEE Trans. Multimedia, vol. 11, no. 3, pp. 465-476, Apr. 2009.
61. M. Wang, X.-S. Hua, T. Mei, R. Hong, G.-J. Qi, Y. Song, and L.-R. Dai, "Semi-supervised kernel density estimation for video annotation," Comput. Vis. Image Understand., vol. 113, no. 3, pp. 348-396, 2009.
62. K. Jarvelin and J. Kekalainen, "Cumulated gain-based evaluation of IR techniques," ACM Trans. Inf. Syst., vol. 20, no. 4, pp. 422-466, 2002.
63. R. H. V. Leuken, L. Garcia, X. Olivares, and R. Zwol, "Visual diversification of image search results," in Proc. ACM Int. World Wide Web Conf., 2009, pp. 341-350.