Facilitating Image Search with a Scalable and Compact Semantic Mapping

IEEE Transactions on Cybernetics

Volumn 45, Issue 8, 2015, Pages 1561-1574

  Wang, Meng ^a   Li, Weisheng ^b   Liu, Dong ^c   Ni, Bingbing ^d   Shen, Jialie ^e   Yan, Shuicheng ^f  

^a HEFEI UNIVERSITY OF TECHNOLOGY   (China)

^b CHONGQING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^c Fu Foundation School of Engineering and Applied Science   (United States)

^d ADVANCED DIGITAL SCIENCES CENTER   (Singapore)

^e SINGAPORE MANAGEMENT UNIVERSITY   (Singapore)

^f NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Compact semantic mapping (CSM); image search; semantic gap

Indexed keywords

MAPPING; SCALABILITY;

COMPUTATIONALLY EFFICIENT; CONCEPT PROTOTYPE; EMBEDDING MATRICES; IMAGE SEARCH; LATENT SEMANTICS; SEMANTIC EMBEDDING; SEMANTIC GAP; SEMANTIC MAPPING;

SEMANTICS;

EID: 85028209417     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2014.2356136     Document Type: Article

References (55)

1. K. Barnard et al., "Matching words and pictures," J. Mach. Learn. Res., vol. 3, pp. 1107-1135, Jan. 2003.
2. J. C. Caicedo, J. G. Moreno, E. A. Niño, and F. A. González, "Combining visual features and text data for medical image retrieval using latent semantic kernels," in Proc. Int. Conf. Multimedia Inf. Retrieval, 2010, pp. 359-366.
3. G. Carneiro, A. B. C. Chan, P. J. Moreno, and N. Vasconcelos, "Supervised learning of semantic classes for image annotation and retrieval," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 3, pp. 394-410, Mar. 2007.
4. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, pp. 27.1-27.27, Apr. 2011.
5. E. Y. Chang, K. Goh, G. Sychay, and G. Wu, "CBSA: Content-based soft annotation for multimodal image retrieval using Bayes point machines," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 1, pp. 26-38, Jan. 2003.
6. L. Chen, D. Xu, I. W. Tsang, and X. Li, "Spectral embedded hashing for scalable image retrieval," IEEE Trans. Cybern., vol. 44, no. 7, pp. 2168-2216, Jul. 2013.
7. T.-S. Chua et al., "NUS-WIDE: A real-world web image database from National University of Singapore," in Proc. ACM Int. Conf. Image Video Retrieval, 2009, pp. 8-10.
8. R. Cilibrasi and P. M. B. Vitányi, "The Google similarity distance," IEEE Trans. Knowl. Data Eng., vol. 19, no. 3, pp. 370-383, Mar. 2007.
9. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, 1st ed. New York, NY, USA: Cambridge Univ. Press, 2000
10. R. Datta, D. Joshi, J. Li, and J. Z. Wang, "Image retrieval: Ideas, influences, and trends of the new age," J. ACM Comput. Surv., vol. 40, no. 2, p. 2007, Apr. 2008.
11. F. Dornaika and A. Bosaghzadeh, "Exponential local discriminant embedding and its application to face recognition," IEEE Trans. Cybern., vol. 43, no. 3, pp. 921-934, Jun. 2013.
12. L. Fei-Fei, R. Fergus, and P. Perona, "Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories," in Proc. Conf. Comput. Vis. Pattern Recognit. Workshop (CVPR), 2004, p. 178.
13. S. L. Feng, R. Manmatha, and V. Lavrenko, "Multiple Bernoulli relevance models for image and video annotation," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), 2004, pp. 1002-1009.
14. D. R. Hardoon, S. R. Szedmak, and J. Shawe-Taylor, "Canonical correlation analysis: An overview with application to learning methods," Neural Comput., vol. 16, no. 12, pp. 2639-2664, 2004.
15. R. Hong et al., "Image annotation by multiple-instance learning with discriminative feature mapping and selection," IEEE Trans. Cybern., vol. 44, no. 5, pp. 669-680, May 2014.
16. C. Hou, F. Nie, X. Li, D. Yi, and Y. Wu, "Joint embedding learning and sparse regression: A framework for unsupervised feature selection," IEEE Trans. Cybern., vol. 44, no. 6, pp. 793-804, Jun. 2014.
17. W. H. Hsu, L. S. Kennedy, and S.-F. Chang, "Video search reranking via information bottleneck principle," in Proc. 14th Annu. ACM Int. Conf. Multimedia, 2006, pp. 35-44.
18. Y. Huang, Q. Liu, S. Zhang, and D. N. Metaxas, "Image retrieval via probabilistic hypergraph ranking," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), San Francisco, CA, USA, 2010, pp. 3376-3383.
19. M. J. Huiskes and M. S. Lew, "The MIR flickr retrieval evaluation," in Proc. ACM Int. Conf. Multimedia Inf. Retrieval, 2008, pp. 39-43.
20. J. Jeon, V. Lavrenko, and R. Manmatha, "Automatic image annotation and retrieval using cross-media relevance models," in Proc. 26th Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2003, pp. 119-126.
21. Y. Jing and S. Baluja, "Pagerank for product image search," in Proc. 17th Int. Conf. World Wide Web, 2008, pp. 307-316.
22. L. Kennedy, S. F. Chang, and A. Natsev, "Query-adaptive fusion for multimodal search," Proc. IEEE, vol. 96, no. 4, pp. 567-588, Apr. 2008.
23. A. Ksibi, A. B. Ammar, and C. B. Amar, "Enhanced context-based query-to-concept mapping in social image retrieval," in Proc. Int. Workshop Content-Based Multimedia Indexing, Veszprem, Hungary, 2003, pp. 85-89.
24. F.-F. Li, R. Fergus, and P. Perona, "Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories," Comput. Vis. Image Und., vol. 106, no. 1, pp. 59-70, 2007.
25. H. Li, Y. Wei, L. Li, and C. L. P. Chen, "Hierarchical feature extraction with local neural response for image recognition," IEEE Trans. Cybern., vol. 43, no. 2, pp. 412-424, Apr. 2013.
26. P. Li, M. Wang, J. Cheng, C. Xu, and H. Lu, "Spectral hashing with semantically consistent graph for image indexing," IEEE Trans. Multimedia, vol. 15, no. 1, pp. 141-152, Jan. 2013.
27. X. Li, C. G. Snoek, M. Worring, and A. W. Smeulders, "Harvesting social images for bi-concept search," IEEE Trans. Multimedia, vol. 14, no. 4, pp. 1091-1104, Aug. 2012.
28. D. Liu, X. S. Hua, L. Yang, M. Wang, and H. J. Zhang, "Tag ranking," in Proc. 18th Int. Conf. World Wide Web, 2009, pp. 351-360.
29. D. Liu, S. Yan, X. S. Hua, and H. J. Zhang, "Image retagging using collaborative tag propagation," IEEE Trans. Multimedia, vol. 13, no. 4, pp. 702-712, Aug. 2011.
30. L. Liu, L. Shao, X. Zhen, and X. Li, "Learning discriminative key poses for action recognition," IEEE Trans. Cybern., vol. 43, no. 6, pp. 1860-1870, Dec. 2013.
31. G. A. Miller, "WordNet: A lexical database for English," ACM Commun., vol. 38, no. 11, pp. 39-41, Nov. 1995.
32. S. Y. Neo, J. Zhao, M.-Y. Kan, and T.-S. Chua, "Video retrieval using high level features: Exploiting query matching and confidence-based weighting," in Proc. Int. Conf. Content-Based Image Video Retrieval, Tempe, AZ, USA, 2006, pp. 143-152.
33. L. Nie, M. Wang, Y. Gao, Z. J. Zha, and T. S. Chua, "Beyond text QA: Multimedia answer generation by harvesting web information," IEEE Trans. Multimedia, vol. 15, no. 2, pp. 426-441, Feb. 2013.
34. L. Nie, S. Yan, M. Wang, R. Hong, and T. S. Chua, "Harvesting visual concepts for image search with complex queries," in Proc. 20th ACM Int. Conf. Multimedia, 2012, pp. 59-68.
35. N. Rasiwasia et al., "A new approach to cross-modal multimedia retrieval," in Proc. Int. Conf. Multimedia, 2010, pp. 251-260.
36. J. R. Smith and S.-F. Chang, "Visually searching the web for content," IEEE MultiMedia, vol. 4, no. 3, pp. 12-20, Jul./Sep. 1997.
37. C. G. Snoek et al., "Adding semantics to detectors for video retrieval," IEEE Trans. Multimedia, vol. 9, no. 5, pp. 975-986, Aug. 2007.
38. C. G. M. Snoek, M. Worring, and A. W. M. Smeulders, "Early versus late fusion in semantic video analysis," in Proc. 13th Annu. ACM Int. Conf. Multimedia, 2005, pp. 399-402. [Online]. Available: http://doi.acm.org/10.1145/1101149.1101236
39. J. Song, Y. Yang, X. Li, Z. Huang, and Y. Yang, "Robust hashing with local models for approximate similarity search," IEEE Trans. Cybern., vol. 44, no. 7, pp. 1225-1236, Jul. 2014.
40. R. K. Srihari, "Automatic indexing and content-based retrieval of captioned images," IEEE Comput., vol. 28, no. 9, pp. 49-56, Sep. 1995.
41. D. Tao, L. Jin, Z. Yang, and X. Li, "Rank preserving sparse learning for Kinect based scene classification," IEEE Trans. Cybern., vol. 43, no. 5, pp. 1406-1417, Oct. 2013.
42. N. Vasconcelos, "Minimum probability of error image retrieval," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2322-2336, Aug. 2004.
43. D. Wang, X. Li, J. Li, and B. Zhang, "The importance of queryconcept- mapping for automatic video retrieval," in Proc. ACM Int. Conf. Multimedia, 2007, pp. 285-288.
44. J. Wang et al., "Brain state decoding for rapid image retrieval," in Proc. 17th ACM Int. Conf. Multimedia, 2009, pp. 945-954.
45. M. Wang and X. S. Hua, "Active learning in multimedia annotation and retrieval: A survey," ACM Trans. Intell. Syst. Technol., vol. 2, no. 2, pp. 10-31, 2011.
46. M. Wang, G. Li, Z. Lu, Y. Gao, and T. S. Chua, "When Amazon meets Google: Product visualization by exploring multiple information sources," ACM Trans. Internet Technol., vol. 12, no. 4, Jul. 2013, Art. ID 12.
47. M. Wang, H. Li, D. Tao, K. Lu, and X. Wu, "Multimodal graph-based reranking for web image search," IEEE Trans. Image Process., vol. 21, no. 11, pp. 4649-4661, Nov. 2012.
48. M. Wang, B. Ni, X. S. Hua, and T. S. Chua, "Assistive tagging: A survey of multimedia tagging with human-computer joint exploration," ACM Comput. Surv., vol. 4, no. 4, Aug. 2012, Art. ID 25.
49. M. Wang, K. Yang, X. S. Hua, and H. J. Zhang, "Towards a relevant and diverse search of social images," IEEE Trans. Multimedia, vol. 12, no. 8, pp. 829-842, Dec. 2010.
50. Q. Wang, Y. Yuan, P. Yan, and X. Li, "Saliency detection by multipleinstance learning," IEEE Trans. Cybern., vol. 43, no. 2, pp. 660-672, Apr. 2013.
51. S. Wei, D. Xu, X. Li, and Y. Zhao, "Joint optimization toward effective and efficient image search," IEEE Trans. Cybern., vol. 43, no. 6, pp. 2216-2227, Dec. 2013.
52. K. Q. Weinberger and O. Chapelle, "Large margin taxonomy embedding with an application to document categorization," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2009, pp. 1737-1744.
53. H. C. Wu, R. W. Luk, K. F. Wong, and K. L. Kwok, "Interpreting TFIDF weights as making relevance decisions," ACM Trans. Inf. Syst., vol. 26, no. 3, pp. 1-37, 2008.
54. B. Xie, Y. Mu, D. Tao, and K. Huang, "m-SNE: Multiview stochastic neighbor embedding," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 4, pp. 1088-1096, Aug. 2011.
55. J. Yu, D. Liu, D. Tao, and H. S. Seah, "On combining multiple features for cartoon character retrieval and clip synthesis," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 5, pp. 1413-1427, Oct. 2012.