Social image tagging with diverse semantics

IEEE Transactions on Cybernetics

Volumn 44, Issue 12, 2014, Pages 2493-2508

  Qian, Xueming ^a   Hua, Xian Sheng ^b   Tang, Yuan Yan ^c   Mei, Tao ^d  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b MICROSOFT   (United States)

^c UNIVERSITY OF MACAU   (Macao)

^d MICROSOFT RESEARCH ASIA   (China)

Author keywords

Image tagging; Semantic; Social media; Tag diversity; Tag enrichment; Tag relevance

Indexed keywords

SEMANTICS; USER INTERFACES; WEBSITES;

ENRICHMENT APPROACHES; IMAGE SHARING; PERSONAL EXPERIENCE; SOCIAL IMAGE TAGGING; SOCIAL IMAGES; SOCIAL MEDIA; TAG DIVERSITY; TAG RELEVANCE;

IMAGE ANNOTATION;

EID: 84911940036     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2014.2309593     Document Type: Article

References (93)

1. D. Liu, X. Hua, L. Yang, M. Wang, and H. Zhang, "Tag ranking," in Proc. WWW, 2009, pp. 351-360.
2. M. Ames and M. Naaman, "Why we tag: Motivations for annotation in mobile and online media," in Proc. SIGCHI Conf. Human Factors Comput. Syst., 2007, pp. 971-980.
3. X. Li, L. Chen, L. Zhang, W. Ma, and F. Lin, "Image annotation by large-scale content-based image retrieval," in Proc. ACM MM, 2006, pp. 2057-2063.
4. X. Rui, M. Li, Z. Li, W. Ma, and N. Yu, "Bipartite graph reinforcement model for web image annotation," in Proc. ACM MM, 2007, pp. 585-594.
5. C. Wang, F. Jing, L. Zhang, and H. Zhang, "Scalable search-based image annotation," Multimedia Syst., vol. 14, no. 4, pp. 205-220, 2008.
6. L. Kennedy, S. Chang, and I. Kozintsev, "To search or to label? Predicting the performance of search-based automatic image classifiers," in Proc. ACM MIR, 2006, pp. 249-258.
7. R. Datta, D. Joshi, J. Li, and J. Wang, "Image retrieval: Ideas, influences, and trends of the new age," ACM Comput. Surveys, vol. 40, no. 2, pp. 1-60, 2008.
8. X. Li, C. Snoek, and M. Worring, "Learning social tag relevance by neighbor voting," IEEE Trans. Multimedia, vol. 11, no. 7, pp. 1310-1322, Nov. 2009.
9. K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. M. Blei, and M. I. Jordan, "Matching words and pictures," J. Mach. Learn. Res., vol. 3, no. 6, pp. 1107-1135, 2003.
10. E. Chang, G. Kingshy, G. Sychay, and G. Wu, "CBSA: Content-based soft annotation for multimodal image retrieval using Bayes point machines," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 2, pp. 26-38, Jan. 2003.
11. J. Li and J. Wang, "Real-time computerized annotation of pictures," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 6, pp. 985-1002, Jun. 2008.
12. R. Datta, D. Joshi, J. Li, and J. Z. Wang, "Tagging over time: Realworld image annotation by lightweight meta-learning," in Proc. ACM MM, 2007, pp. 393-402.
13. C. Cusano, G. Ciocca, and R. Schettini, "Image annotation using SVM," in Proc. SPIE, 2004, pp. 330-338.
14. P. Quelhas, F. Monay, J.-M. Odobez, D. Gatica-Perez, and T. Tuytelaars, "A thousand words in a scene," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 9, pp. 1575-1589, Sep. 2007.
15. A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-based image retrieval at the end of the early years," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
16. A. Torralba, R. Fergus, and W. T. Freeman, "80 million tiny images: A large data set for nonparametric object and scene recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 1958-1970, Nov. 2008.
17. X. Wang, L. Zhang, X. Li, and W. Ma, "Annotating images by mining image search results," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 1919-1932, Nov. 2008.
18. X. Li, C. G. M. Snoek, and M. Worring, "Annotating images by harnessing worldwide user-tagged photos," in Proc. ICASSP, 2009, pp. 3717-3720.
19. T. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng, "NUS-WIDE: A real-world web image database from National University of Singapore," in Proc. CIVR, 2009, pp. 1-9.
20. B. Sigurbjörnsson and R. Zwol, "Flickr tag recommendation based on collective knowledge," in Proc. WWW, 2008, pp. 327-336.
21. K. Weinberger, M. Slaney, and R. Zwol, "Resolving tag ambiguity," in Proc. ACM MM, 2008, pp. 111-119.
22. D. Liu, X. Hua, M. Wang, and H. Zhang, "Retagging social images based on visual and semantic consistency," in Proc. ACM WWW, 2010, pp. 1149-1150.
23. Y. Jin, L. Khan, L. Wang, and M. Awad, "Image annotations by combining multiple evidence and wordnet," in Proc. ACM MM, 2005, pp. 706-715.
24. X. Wang, L. Zhang, M. Liu, Y. Li, and W. Ma, "ARISTA: Image search to annotation on billions of web photos," in Proc. CVPR, 2010, pp. 2987-2994.
25. H. Chen, M. Chang, P. Chang, M. Tien, W. Hsu, and J. Wu, "SheepDog: Group and tag recommendation for Flickr photos by automatic search-based learning," in Proc. ACM MM, 2008, pp. 1155-1156.
26. C. Wang, F. Jing, L. Zhang, and H. Zhang, "Scalable search-based image annotation," Multimedia Syst., vol. 14, no. 4, pp. 205-220, 2008.
27. C. Wang, F. Jing, L. Zhang, and H. Zhang, "Content-based image annotation refinement," in Proc. CVPR, Jun. 2007, pp. 1-8.
28. J. Jia, N. Yu, X. Rui, and M. Li, "Multigraph similarity reinforcement for image annotation refinement," in Proc. ICIP, 2008, pp. 993-996.
29. G. Zhu, S. Yan, and Y. Ma, "Image tag refinement towards low-rank, content-tag prior and error sparsity," in Proc. ACM MM, 2010, pp. 461-470.
30. H. Xu, J. Wang, X. Hua, and S. Li, "Tag refinement by regularized LDA," in Proc. ACM MM, 2009, pp. 573-576.
31. L. Wu, L. Yang, N. H. Yu, and X. Hua, "Learning to tag," in Proc. ACM WWW, 2009, pp. 361-370.
32. D. Liu, X. Hua, M. Wang, and H. Zhang, "Image retagging," in Proc. ACM MM, 2010.
33. M. Huiskes and M. Lew, "The MIR Flickr retrieval evaluation," in Proc. ACM MIR, 2008.
34. L. Wu, X.-S. Hua, N. Yu, W.-Y. Ma, and S. Li, "Flickr distance," in Proc. ACM MM, 2008, pp. 31-40.
35. X. Li, C. G. M. Snoek, and M. Worring, "Learning tag relevance by neighbor voting for social image retrieval," in Proc. ACM MIR, 2008, pp. 180-187.
36. M. Wang, K. Yang, X. Hua, and H. Zhang, "Visual tag dictionary: Interpreting tags with visual words," in Proc. WSMC, 2009.
37. N. Zhou, W. Cheung, G. Qiu, and X. Xue, "A hybrid probabilistic model for unified collaborative and content-based image tagging," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1281-1294, Jul. 2011.
38. X. Qian and X. Hua, "Graph-cut based tag enrichment," in Proc. SIGIR, 2011.
39. X. Qian, X. Liu, C. Zheng, Y. Du, and X. Hou, "Tagging photos using users' vocabularies," Neurocomputing, vol. 111, pp. 144-153, 2013.
40. X. Qian, X. Hua, and X. Hou, "Tag filtering based on similar compatible principle," in Proc. ICIP, 2012, pp. 2349-2352.
41. D. Cai, X. He, Z. Li, W. C. Ma, and J. C. Wen, "Hierarchical clustering of WWW image search results using visual, textual and link information," in Proc. ACM MM Conf., 2004, pp. 952-959.
42. J. Carbonell and J. Goldstein, "The use of MMR, diversity-based reranking for reordering documents and producing summaries," in Proc. SIGIR Conf., 1998, pp. 335-336.
43. W. Goffman, "A searching procedure for information retrieval," Inform. Storage Retrieval, vol. 2, pp. 73-78, 1964.
44. A. Jaimes, S. C. Chang, and A. C. Loui, "Detection of non-identical duplicate consumer photographs," in Proc. ACM MM Conf., 2003, pp. 16-20.
45. F. Jing, C. Wang, Y. Yao, K. Deng, L. Zhang, and W. Ma, "IGroup: Web image search results clustering," in Proc. ACM MM, 2006, pp. 587-596.
46. R. H. V. Leuken, L. Garcia, X. Olivares, and R. Zwol, "Visual diversification of image search results," in Proc. WWW, 2009, pp. 341-350.
47. K. Song, Y. Tian, T. Huang, and W. Gao, "Diversifying the image retrieval results," in Proc. ACM MM, 2006, pp. 707-710.
48. S. Srinivasan and N. Sawant, "Finding near-duplicate images on the web using fingerprints," in Proc. ACM MM, 2008, pp. 881-884.
49. A. Sun and S. S. Bhowmick, "Image tag clarity: In search of visual representative tags for social images," in Proc. 1st SIGMM WSM, 2009, pp. 19-26.
50. B. Wang, Z. Li, M. Li, and W. Ma, "Large-scale duplicate detection for web image search," in Proc. ICME, 2006, pp. 353-356.
51. C. Zhai, W. Cohen, and J. Lafferty, "Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval," Inform. Process. Manag., pp. 10-17, 2006.
52. C. Zhai and J. Lafferty, "A risk minimization framework for information retrieval," Inform. Process. Manag., pp. 31-55, 2006.
53. W. Zhao and C. Ngo, "Scale-rotation invariant pattern entropy for keypoint-based near-duplicate detection," IEEE Trans. Image Process., vol. 18, no. 2, pp. 412-423, Feb. 2009.
54. J. Zhu, S. Hoi, M. Lyu, and S. Yan, "Near-duplicate keyframe retrieval by nonrigid image matching," in Proc. ACM MM, 2008, pp. 41-50.
55. M. Wang, K. Yang, X. Hua, and H. Zhang, "Towards a relevant and diverse search of social images," IEEE Trans. Multimedia, vol. 12, no. 8, pp. 829-842, Dec. 2010.
56. X. Qian, G. Liu, D. Guo, Z. Li, Z. Wang, and H. Wang, "Object categorization using hierarchical wavelet packet texture descriptors," in Proc. ISM, 2009, pp. 44-51.
57. R. Cilibrasi and P. M. B. Vitanyi, "The Google similarity distance," IEEE Trans. Knowl. Data Eng., vol. 19, no. 3, pp. 370-383, Mar. 2007.
58. S. Zhu, G. Wang, C. Ngo, and Y. Jiang, "On the sampling of web images for learning visual concept classifiers," in Proc. CIVR, 2010.
59. S. Feng, C. Lang, and D. Xu, "Beyond tag relevance: Integrating visual attention model and multiinstance learning for tag saliency ranking," in Proc. CIVR, 2010, pp. 288-295.
60. S. Zhang, J. Huang, H. Li, and D. Metaxas, "Automatic image annotation and retrieval using group sparsity," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 3, pp. 838-849, Jun. 2012.
61. Z. Ma, F. Nie, Y. Yang, J. Uijlings, and N. Sebe, "Web image annotation via subspace-sparsity collaborated feature selection," IEEE Trans. Multimedia, vol. 14, no. 4, pp. 1021-1030, Aug. 2012.
62. Y. Han, F. Wu, Q. Tian, and Y. Zhuang, "Image annotation by input-output structural grouping sparsity," IEEE Trans. Image Process., vol. 21, no. 6, pp. 3066-3079, Jun. 2012.
63. J. Deng, W. Dong, R. Socher, L.-J. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in Proc. CVPR, 2009.
64. D. Liu, S. Yan, X.-S. Hua, and H.-J. Zhang, "Image retagging using collaborative tag propagation," IEEE Trans. Multimedia, vol. 13, no. 4, pp. 702-712, Aug. 2011.
65. J. Shen, W. Meng, S. Yan, H. Pang, and X. Hua, "Effective music tagging through advanced statistical modeling," in Proc. SIGIR, 2010.
66. M. Wang, H. Li, D. Tao, K. Lu, and X. Wu, "Multimodal graph-based reranking for web image search," IEEE Trans. Image Process., vol. 21, no. 11, pp. 4649-4661, Nov. 2012.
67. Y. Gao, M. Wang, Z. Zha, J. Shen, X. Li, and X. Wu, "Visual-textual joint relevance learning for tag-based social image search," IEEE Trans. Image Process., vol. 22, no. 1, pp. 363-376, Jan. 2013.
68. Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan, "A multimedia retrieval framework based on semi-supervised ranking and relevance feedback," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 723-742, Apr. 2012.
69. Y. Yang, F. Wu, F. Nie, H. Shen, Y. Zhuang, and A. Hauptmann, "Web and personal image annotation by mining label correlation with relaxed visual graph embedding," IEEE Trans. Image Process, vol. 21, no. 3, pp. 1339-1351, Mar. 2012.
70. M. Wang, B. Ni, X. Hua, and T. Chua, "Assistive tagging: A survey of multimedia tagging with human-computer joint exploration," ACM Comput. Surveys, vol. 44, no. 4, Aug. 2012.
71. G. Li, M. Wang, Z. Lu, R. Hong, and T. Chua, "In-video product annotation with web information mining," ACM Trans. Multimedia Comput., Commun. Applic., vol. 8, no. 4, pp. 55:1-55:19, 2012.
72. E. Moxley, J. Kleban, and B. S. Manjunath, "SpiritTagger: A geo-aware tag suggestion tool mined from Flickr," in Proc. ACM MIR., 2008.
73. K. Yang, X. Hua, M. Wang, and H. Zhang, "Tag tagging: Towards more descriptive keywords of image content," IEEE Trans. Multimedia, vol. 13, no. 4, pp. 662-673, Aug. 2011.
74. Y. Yang, F. Wu, D. Xu, Y. Zhuang, and L. Chia, "Cross-media retrieval using query dependent search methods," Pattern Recognit., vol. 43, no. 8, pp. 2927-2936, 2010.
75. M. Wang, R. Hong, G. Li, Z. Zha, S. Yan, and T. Chua, "Event driven web video summarization by tag localization and key-shot identification," IEEE Trans. Multimedia, vol. 14, no. 4, pp. 975-985, Aug. 2012.
76. P. Moëllic, J. Haugeard, and G. Pitel, "Image clustering based on a shared nearest neighbors approach for tagged collections," in Proc. CIVR, 2008, pp. 269-278.
77. D. Cai, X. He, Z. Li, W.-Y. Ma, and J.-R. Wen, "Hierarchical clustering of www image search results using visual, textual and link information," in Proc. ACM MM, 2004.
78. R. Leuken, L. Garcia, X. Olivares, and R. Zwol, "Visual diversification of image search results," in Proc. WWW, 2009.
79. A. Radu, J. Stöttinger, B. Ionescu, M. Menéndez, and F. Giunchiglia, "Representativeness and diversity in photos via crowd-sourced media analysis," in Proc. 10th Int. Workshop-AMR, 2012.
80. K. Weinberger, M. Slaney, and R. van Zwol, "Resolving tag ambiguity," in Proc. ACM MM, 2008.
81. R. Zwol, V. Murdock, L. Garcia, and G. Ramirez, "Diversifying image search with user generated content," in Proc. ACM MIR, 2008.
82. K. Song, Y. Tian, W. Gao, and T. Huang, "Diversifying the image retrieval results," in Proc. ACM MM, 2006.
83. B. Zhang et al., "Improving web search results using affinity graph," in Proc. SIGIR, 2005.
84. C. Ziegler, S. McNee, J. Konstan, and G. Lausen, "Improving recommendation lists through topic diversification," in Proc. WWW, 2005.
85. M. Paramita, J. Tang, and M. Sanderson, "Generic and spatial approaches to image search results diversification," in Proc. ECIR, 2009.
86. T. Deselaers, T. Gass, P. Dreuw, and H. Ney, "Jointly optimising relevance and diversity in image retrieval," in Proc. ACM CIVR, 2009, pp. 1-8.
87. S. Rudinac, A. Hanjalic, and M. Larson, "Finding representative and diverse community contributed images to create visual summaries of geographic areas," in Proc. ACM MM, 2011, pp. 1109-1112.
88. A. Popescu and I. Kanellos, "Creating visual summaries for geographic regions," in Proc. 10th Int. Workshop AMR, 2012.
89. Y. Xue and X. Qian, "Visual summarization of landmarks via viewpoint modeling," in Proc. ICIP, 2012, pp. 2873-2876.
90. S. Jiang, X. Qian, Y. Xue, F. Li, and X. Hou, "Generating representative images for landmark by discovering high frequency shooting locations from community-contributed photos," in Proc. ICME, 2013.
91. Q. Li, Y. Gu, and X. Qian, "LCMKL: Latent-community and multikernel learning based image annotation," in Proc. ACM CIKM, 2013.
92. J. Li, X. Qian, Y. Tang, L. Yang, and T. Mei, "GPS estimation for places of interest from social users' uploaded photos," IEEE Trans. Multimedia, vol. 15, no. 8, pp. 2058-2071, Dec. 2013.
93. J. Li, X. Qian, Y. Tang, L. Yang, and C. Liu, "GPS estimation from users' photos," in Proc. MMM, 2013, pp. 118-129.