Image location inference by multisaliency enhancement

IEEE Transactions on Multimedia

Volumn 19, Issue 4, 2017, Pages 813-821

  Qian, Xueming ^a   Wang, Huan ^a   Zhao, Yisi ^a   Hou, Xingsong ^a   Hong, Richang ^b   Wang, Meng ^b   Tang, Yuan Yan ^c  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b HEFEI UNIVERSITY OF TECHNOLOGY   (China)

^c UNIVERSITY OF MACAU   (Macao)

Author keywords

Location estimation; region of interest (ROI); salient map; salient region; spatial constraint; visual phrase

Indexed keywords

CONTENT BASED RETRIEVAL; IMAGE SEGMENTATION; LOCATION;

LOCATION ESTIMATION; REGION OF INTEREST; SALIENT MAPS; SALIENT REGIONS; SPATIAL CONSTRAINTS; VISUAL PHRASE;

IMAGE RETRIEVAL;

EID: 85017608681     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2016.2638207     Document Type: Article

References (79)

1. S. Zhang, Q. Tian, G. Hua, Q. Huang, and S. Li, "Descriptive visual words and visual phrases for image," in Proc. 17th ACM Int. Conf. Multimedia, 2009, pp. 75-84.
2. Y. Li, D. Crandall, andD. Huttenlocher, "Landmark classification in largescale image collections," in Proc. 12th Int. Conf. Comput. Vis., 2009, pp. 1957-1964.
3. M.Donoser andD. Schmalstieg, "Discriminative feature to point matching in image based localization," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2014, pp. 516-523.
4. J. Li, X. Qian, Y. Tang, L. Yang, and T. Mei, "GPS estimation for places of interest from social users' uploaded photos," IEEE Trans. Multimedia, vol. 15, no. 8, pp. 2058-2071, Dec. 2013.
5. C.Hauff and G.Houben, "Placing images on theworld map:Amicroblogbased enrichment approach," in Proc. 35th Int. ACMSIGIR Conf. Res.Dev. Inf. Retrieval, 2012, pp. 691-700.
6. H. Liu, T.Mei, J. Luo, H. Li, and S. Li, "Finding perfect rendezvous on the go: Accurate mobile visual localization and its applications to routing," in Proc. 20th ACM Int. Conf. Multimedia, 2012, pp. 9-18.
7. E. Gavves, C. Snoek, and A. Smeulders, "Visual synonyms for landmark image retrieval," Comput. Vis. Image Understand., vol. 116, no. 2, pp. 238-249, 201l.
8. W. Zhou, Y. Lu, H. Li, Y. Song, and Q. Tian, "Spatial coding for large scale partial-duplicate web image search," in Proc. 18th ACM Int. Conf. Multimedia, 2010, pp. 511-520.
9. J. Hays and A. Efros, "IM2GPS: Estimating geographic information from a single image," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2008, pp. 1-8.
10. O. Chum, J. Philbin, J. Sivic, M. Isard, and A. Zisserman, "Total recall: Automatic query expansion with a generative feature model for object retrieval," in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2007, pp. 1-8.
11. J.Wang, J.Yang, and K.Yu, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., Jun. 2006, vol. 2, pp. 2169-2178.
12. X. Qian, H. Wang, G. Liu, and X. Hou, "HWVP: Hierarchical wavelet packet texture descriptors and their applications in scene categorization and semantic concept retrieval," Multimedia Tools Appl., vol. 69, no. 3, pp. 897-920, 2014.
13. G. Baatz, K. K?oser, D. Chen, R. Grzeszczuk, and M. Pollefeys, "Leveraging 3D city models for rotation invariant place-of-interest recognition," Int. J. Comput. Vis., vol. 96, no. 3, pp. 315-334, 2012.
14. Y. Xue and X. Qian, "Visual summarization of landmarks via viewpoint modeling," in Proc. IEEE Int. Conf. Image Process., Sep.-Oct. 2012, pp. 2873-2876.
15. J. Li, X. Qian, Y. Tang, L. Yang, and C. Liu, "GPS estimation from users' photos," in Proc. 19th Int. Conf. Multimedia Model., 2013, pp. 118-129.
16. J. Philbin, O. Chum,M. Isard, J. Sivic, and A. Zisserman, "Object retrieval with large vocabularies and fast spatial matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2007, pp. 1-8.
17. C.Wu, F. Fraundorfer, J. Frahm, and M. Pollefeys, "3D model search and pose estimation from single images using VIP features," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog. Workshops, Jun. 2008, pp. 1-8.
18. Y. Zheng et al., "Tour the world: Building a web-scale landmark recognition engine," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 1085-1092.
19. X. Yang, X. Qian, and Y. Xue, "Scalable mobile image retrieval by exploring contextual saliency," IEEE Trans. Image Process., vol. 24, no. 6, pp. 1709-1721, Jun. 2015.
20. M. Park, J. Luo, R. Collins, and Y. Liu, "Beyond GPS: Determining the camera viewing direction of a geo-tagged image," in Proc. 18th ACM Int. Conf. Multimedia, 2010, pp. 631-634.
21. D. Crandall, L. Backstrom, D. Huttenlocher, and J. Kleinberg, "Mapping the world's photos," in Proc. Int. Conf. World Wide Web, 2009, pp. 761-770.
22. E. Kalogerakis, O.Vesselova, J.Hays, A. Efros, andA.Hertzmann, "Image sequence geolocation with human travel priors," in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep.-Oct. 2009, 253-260.
23. T. Quack, B. Leibe, and L. Gool, "World-scale mining of objects and events from community photo collections," in Proc. Int. Conf. Content-Based Image Video Retrieval, 2008, pp. 47-56.
24. O. Penatti, F. Silva, and E. Valle, "Visual word spatial arrangement for image retrieval and classification," Pattern Recog., vol. 47, no. 2, pp. 705-720, 2014.
25. Y. Zhao, X. Qian, and T. Mu, "Image taken place estimation via geometric constrained spatial layer matching," in Proc. 21st Int. Conf. MultiMedia Model., 2015, pp. 436-446.
26. J. Yuan, Y. Wu, and M. Yang, "Discovery of collocation patterns: From visual words to visual phrases," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2007. pp. 1-8.
27. J. Han, M. Xu, X. Li, L. Guo, and T. Liu, "Interactive object-based image retrieval and annotation on iPad," Multimedia Tools Appl., vol. 72, no. 3, pp. 2275-2297, 2014.
28. S. Jiang, X. Qian, J. Shen, Y. Fu, and T. Mei, "Author topic model-based collaborative filtering for personalized POI recommendations," IEEE Trans. Multimedia, vol. 17, no. 6, pp. 907-918, Jun. 2015.
29. O. Laere, S. Schockaert, and B. Dhoedt, "Ghent university at the 2010 placing task," in Proc. MediaEval Workshop, 2010, pp. 1-2.
30. P. Kelm, S. Schmiedeke, and T. Sikora, "How spatial segmentation improves the multimodal geo-tagging," in Proc. MediaEval, 2012, pp. 1-2.
31. S. Zhang et al., "Building contextual visual vocabulary for large-scale image applications," in Proc. 18th ACM Int. Conf. Multimedia, 2010, pp. 501-510.
32. H. Li, Y. Wang, T. Mei, J. Wang, and S. Li, "Interactive multimodal visual search on mobile device," IEEE Trans. Multimedia, vol. 15, no. 3, pp. 594-607, Apr. 2013.
33. J. Sang et al., "Interaction design for mobile visual search," IEEE Trans. Multimedia, vol. 15, no. 7, pp. 1665-1676, Nov. 2013.
34. D. Nist?er and H. Stew?enius, "Scalable recognition with a vocabulary tree," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2006, vol. 2, pp. 2161-2168.
35. X. Qian, Y. Zhao, and J. Han, "Image location estimation by salient region matching," IEEE Trans. Image Process., vol. 24, no. 6, pp. 4348-4358, Nov. 2015.
36. Z. Wu, Q. Ke, M. Isard, and J. Sun, "Bundling features for large scale partial-duplicate web image search," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 25-32
37. J. Chen, B. Feng, L. Zhu, P. Ding, and B. Xu, "Effective nearduplicate image retrieval with image-specific visual phrase selection," in Proc. IEEE Int. Conf. Image Process., Sep.-Oct. 2010, pp. 1909-1912.
38. J. Li et al., "Improved image GPS location estimation by mining salient features," Signal Process., Image Commun., vol. 38, pp. 141-150, 2015.
39. R. Ji, L.Duan, J. Chen, andW. Gao, "Towards compact topical descriptor," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 2925-2932.
40. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Lost in quantization: Improving particular object retrieval in large scale image databases," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2008, pp. 1-8.
41. W. Tang, R. Cai, Z. Li, and L. Zhang, "Contextual synonym dictionary for visual object retrieval," in Proc. 19th ACM Int. Conf. Multimedia, 2011, pp. 503-512.
42. S. Jiang, X. Qian, T. Mei, and Y. Fu, "Personalized travel sequence recommendation on multi-source big social media," IEEE Trans. Big Data, vol. 1, no. 2, pp. 43-56, Mar. 2016.
43. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
44. Y. Yang et al., "A multimedia retrieval framework based on semisupervised ranking and relevance feedback," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 723-742, Apr. 2012.
45. L. Shao, L. Liu, and X. Li, "Feature learning for image classification via multi-objective genetic programming," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 7, pp. 1359-1371, Jul. 2014.
46. Y. Yang, Z. Ma, A. Hauptmann, and N. Sebe, "Feautre selection for multimedia analysis by sharing information among multiple tasks," IEEE Trans. Multimedia, vol. 15, no. 3, pp. 661-669, Apr. 2013.
47. C. Yang, J. Shen, J. Peng, and J. Fan, "Image collection summarization via dictionary learning for sparse representation," Pattern Recog., vol. 46, no. 3, pp. 948-961, 2013.
48. W. Min, B. Bao, and C. Xu, "Multimodal spatio-temporal theme modeling landmark analysis," IEEE Multimedia Mag., vol. 21, no. 3, pp. 20-29, Jul./Sep. 2014.
49. J. Huang, H. Liu, J. Shen, and S. Yan, "Towards efficient sparse coding for scalable image annotation," in Proc. 21st ACM Int. Conf. Multimedia, 2013, pp. 947-956.
50. H. Bay, T. Tuytelaars, and L.Gool, "Surf: Speeded up robust features," in Proc. 9th Eur. Conf. Comput. Vis., 2006, pp. 404-417.
51. L. Itti, C.Koch, and E. Niebur, "Amodel of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
52. M. Cheng, N. Mitra, X. Huang, P. Torr, and S. Hu, "Global contrast based salient region detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 569-582, Mar. 2015.
53. H. Li and K. N. Ngan, "A co-saliency model of image pairs," IEEE Trans. Image Process., vol. 20, no. 12, pp. 3365-3375, Dec. 2011.
54. X. Cao, Z. Tao, B. Zhang, H. Fu, andW. Feng, "Self-adaptively weighted co-saliency detection via rank constraint," IEEE Trans. Image Process., vol. 23, no. 9, pp. 4175-4186, Sep. 2014.
55. D. Zhang, J. Han, C. Li, and J. Wang, "Co-saliency detection via looking deep and wide," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2015, pp. 2994-3002.
56. H. Fu, X. Cao, and Z. Tu, "Cluster-based co-saliency detection," IEEE Trans. Image Process., vol. 22, no. 10, pp. 3766-3778, Oct. 2013.
57. X. Cao, Y. Cheng, Z. Tao, and H. Fu, "Co-saliency detection via base reconstruction," in Proc. 22nd ACM Int. Conf. Multimedia, 2014, pp. 997-1000.
58. X. Yang, X. Qian, and T. Mei, "Learning salient visual word for scalable mobile image retrieval," Pattern Recog., vol. 48, no. 10, pp. 3093-3101, 2015.
59. X. Qian, Y. Xue, Y. Tang, X. Hou, and T. Mei, "Landmark summarization with diverse viewpoints," IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 11, pp. 1857-1869, Nov. 2015.
60. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1106-1114.
61. X. Qian, X. Liu, X. Ma, D. Lu, and C. Xu, "What Is happening in the video? Annotate video by sentence," IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 9, pp. 1746-1757, Sep. 2016.
62. R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, "Supervised hashing via image representation learning," in Proc. 28th AAAI Conf. Artif. Intell., 2014, pp. 2156-2162.
63. R. Zhang, L. Lin, R. Zhang, W. Zuo, and L. Zhang, "Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification," IEEE Trans. Image Process., vol. 24, no. 12, pp. 4766-4779, Dec. 2015
64. K. Lin, H. Yang, J. Hsiao, and C. Chen, "Deep learning of binary hash codes for fast image retrieval," in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops, Jun. 2015, pp. 27-35.
65. F. Zhao, Y. Huang, L. Wang, and T. Tan, "Deep semantic ranking based hashing for multi-label image retrieval," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2015, pp. 1556-1564.
66. D. Lu,X. Liu, and X. Qian, "Tag based image search by social re-ranking," IEEE Trans. Multimedia, vol. 18, no. 8, pp. 1628-1639, Aug. 2016.
67. G. Zhao, X. Qian, and T. Mei, "Service rating prediction by exploring social mobile users' geographic locations," IEEE Trans. Big Data, to be published, doi: 10.1109/TBDATA.2016.2552541.
68. X. Qian, X. Tan, Y. Zhang, R. Hong, and M. Wang, "Enhancing sketchbased image retrieval by re-ranking and relevance feedback," IEEE Trans. Image Process., vol. 25, no. 1, pp. 195-208, Jan. 2016.
69. X. Lu,Y.Yuan, X. Zheng, "Jointly dictionary learning for change detection in multispectral imagery," IEEE Trans. Cybern., to be published, doi: 10.1109/TCYB.2016.2531179.
70. X. Lu, Y. Yuan, and P. Yan, "Image super-resolution via double sparsity regularized manifold learning," IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 12, pp. 2022-2033, Dec. 2013.
71. X. Lu, Y. Yuan, and P. Yan, "Alternatively constrained dictionary learning for image super-resolution," IEEE Trans. Cybern., vol. 44, no. 3, pp. 366-377, Mar. 2014.
72. X. Lu andX. Li, "Multi-resolution imaging," IEEE Trans.Cybern., vol. 44, no. 1, pp. 149-160, Jan. 2014.
73. X. Lu X. Li, and L. Mou, "Semi-supervised multi-task learning for scene recognition," IEEE Trans. Cybern., vol. 45, no. 9, pp. 1967-1976, Sep. 2015.
74. X. Lu, Z. Wang, and X. Lu , "Surveillance video synopsis via scaling down objects," IEEE Trans. Image Process., vol. 25, no. 2, pp. 740-755, Feb. 2016.
75. X. Lu, X. Zheng, and X. Li, "Latent semantic minimal hashing for image retrieval," IEEE Trans. Image Process., vol. 26, no. 1, pp. 355-368, Jan. 2017.
76. X. Lu, L. Mou, and X. Lu, "Scene recognition by manifold regularized deep learning architecture," IEEE Trans.NeuralNetw. Learn. Syst., vol. 26, no. 10, pp. 2222-2233, Oct. 2015.
77. D. Zhang, J. Han, C. Li, J. Wang, and X. Li, "Detection of co-salient objects by looking deep and wide," Int. J. Comput. Vis., vol. 120, no. 2, pp. 215-232, 2016.
78. Y. Zhang, X. Qian, X. Tan, J. Han, and Y. Tang, "Sketch-based image retrieval by salient contour reinforcement," IEEE Trans.Multimedia, vol. 18, no. 8, pp. 1604-1615, Aug. 2016.
79. D. Zhang, J. Han, J. Han, and L. Shao, "Cosaliency detection based on intrasaliency prior transfer and deep intersaliency mining," IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 6, pp. 1163-1176, Jun. 2016.