A hybrid mobile visual search system with compact global signatures

IEEE Transactions on Multimedia

Volumn 17, Issue 7, 2015, Pages 1019-1030

  Chen, David M ^a   Girod, Bernd ^a  

^a Stanford University   (United States)

Author keywords

Compact signatures; image retrieval; interframe compression; mobile visual search

Indexed keywords

DATABASE SYSTEMS; HYBRID SYSTEMS; IMAGE RETRIEVAL; MOBILE DEVICES; OBJECT RECOGNITION; QUERY LANGUAGES; QUERY PROCESSING; SAFETY DEVICES;

COMPACT SIGNATURES; DATABASE MATCHING; INTER-FRAME CODING; INTERFRAME COMPRESSION; MOBILE VISUAL SEARCHES; NETWORK TRANSMISSION; RETRIEVAL ACCURACY; RETRIEVAL PERFORMANCE;

SEARCH ENGINES;

EID: 84933500630     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2015.2427744     Document Type: Article

References (52)

1. G. Takacs et al., "Outdoors augmented reality on mobile phone using loxel-based visual feature organization," in Proc. ACMMultimedia Inf. Retrieval, Oct. 2008, pp. 427-434.
2. S. Tsai et al., "Mobile product recognition," in Proc. ACM Int. Conf. Multimedia, Oct. 2010, pp. 1587-1590.
3. P. Fockler, T. Zeidler, B. Brombach, E. Bruns, and O. Bimber, "PhoneGuide: Museum guidance supported by on-device object recognition on mobile phones," in Proc. Int. Conf. Mobile Ubiquitous Multimedia, Dec. 2005, pp. 3-10.
4. W. Liu, T. Mei, Y. Zhang, J. Li, and S. Li, "Listen, look, and gotcha: Instant video search with mobile phones by layered audio-video indexing," in Proc. ACM Int. Conf. Multimedia, Oct. 2013, pp. 887-896.
5. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
6. H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool, "Speeded-up robust features (SURF)," Comput. Vis. Image Understand., vol. 110, no. 3, pp. 346-359, Jun. 2008.
7. V. Chandrasekhar et al., "CHoG: Compressed histogram of gradients - a low bitrate feature descriptor," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 2504-2511.
8. G. Takacs et al., "Unified real-time tracking and recognition with rotation-invariant fast features," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 934-941.
9. E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, "ORB: An efficient alternative to SIFT or SURF," in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 2564-2571.
10. A. Alahi, R. Ortiz, and P. Vandergheynst, "FREAK: Fast retina keypoint," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 510-517.
11. J. Sivic and A. Zisserman, "Video Google: A text retrieval approach to object matching in videos," in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2003, pp. 1470-1477.
12. D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2006, pp. 2161-2168.
13. F. Perronnin, Y. Liu, J. Sanchez, and H. Poirier, "Large-scale image retrieval with compressed Fisher vectors," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 3384-3391.
14. H. Jegou, M. Douze, C. Schmid, and P. Perez, "Aggregating local descriptors into a compact image representation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 3304-3311.
15. D. Chen et al., "Residual enhanced visual vectors for on-device image matching," in Proc. IEEE Asilomar Conf. Signals, Syst., Comput., Nov. 2011, pp. 850-854.
16. C. Yeo, P. Ahammad, and K. Ramchandran, "Rate-efficient visual correspondences using random projections," in Proc. IEEE Int. Conf. Image Process., Oct. 2008, pp. 217-220.
17. V. Chandrasekhar et al., "Transform coding of image feature descriptors," in Proc. SPIE Conf. Vis. Commun. Image Process., Jan. 2009, pp. 1-10.
18. V. Chandrasekhar et al., "Compressed histogram of gradients: A low bitrate descriptor," Int. J. Comput. Vis., vol. 96, no. 3, pp. 384-399, 2012.
19. S. Tsai et al., "Location coding for mobile image retrieval," in Proc. Int. Conf. Mobile Multimedia Commun., Sep. 2009, pp. 1-7.
20. J. Chao and E. Steinbach, "Preserving SIFT features in JPEG-encoded images," in Proc. IEEE Int. Conf. Image Process., Sep. 2011, pp. 301-304.
21. J. He et al., "Mobile product search with bag of hash bits and boundary reranking," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 3005-3012.
22. M. Makar, S. Tsai, V. Chandrasekhar, D. Chen, and B. Girod, "Interframe coding of canonical patches for low bit-rate mobile augmented reality," Int. J. Semantic Comput., vol. 7, no. 1, pp. 5-24, Mar. 2013.
23. A. Oliva and A. Torralba, "Modeling the shape of the scene: A holistic representation of the spatial envelope," Int. J. Comput. Vis., vol. 42, no. 3, pp. 145-175, May 2001.
24. D. Chen et al., "Tree histogram coding for mobile image matching," in Proc. IEEE Data Compression Conf., Mar. 2009, pp. 143-152.
25. H. Jegou, M. Douze, and C. Schmid, "Packing bag-of-features," in Proc. IEEE Int. Conf. Comput. Vis., Sep. 2009, pp. 2357-2364.
26. D. Chen et al., "Inverted index compression for scalable image matching," in Proc. IEEE Data Compression Conf., Mar. 2010, p. 13.
27. R. Ji et al., "Location discriminative vocabulary coding for mobile landmark search," Int. J. Comput. Vis., vol. 96, pp. 290-314, Feb. 2012.
28. Evaluation Framework for Compact Descriptors for Visual Search, ISO/IEC JTCI/SC29/WG11 N12202, Jul. 2011.
29. Requirements for Compact Descriptor for Video Analysis, ISO/IEC JTCI/SC29/WG11 N14095, Oct. 2013.
30. D. Chen et al., "Residual enhanced visual vector as a compact signature for mobile visual search," Signal Process., vol. 93, no. 8, pp. 2316-2327, Aug. 2013.
31. D. Chen and B. Girod, "Memory-efficient image databases for mobile visual search," IEEE Multimedia Mag., vol. 21, no. 1, pp. 14-23, Jan.-Mar. 2014.
32. R. Arandjelovic, "Three things everyone should know to improve object retrieval," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 2911-2918.
33. H. Jegou, M. Douze, and C. Schmid, "On the burstiness of visual elements," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 1169-1176.
34. J. Delhumeau, P.-H. Gosselin, H. Jegou, and P. Perez, "Revisiting the VLAD image representation," in Proc. ACM Int. Conf. Multimedia, Oct. 2013, pp. 653-656.
35. Y. Weiss, A. Torralba, and R. Fergus, "Spectral hashing," in Neural Inf. Process. Syst., Dec. 2008, pp. 1753-1760.
36. A. Joly and O. Buisson, "Random maximum margin hashing," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 873-880.
37. Y. Gong and S. Lazebnik, "Iterative quantization: A procrustean approach to learning binary codes," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 817-824.
38. W. Zhou, Y. Lu, H. Li, and Q. Tian, "Scalar quantization for large scale image search," in Proc. ACM Int. Conf. Multimedia, Oct. 2012, pp. 169-178.
39. Z. Drezner and N. Farnum, "A generalized binomial distribution," Commun. Statist., vol. 22, no. 11, pp. 3051-3063, 1993.
40. G. Schindler, M. Brown, and R. Szeliski, "City-scale location recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2007, pp. 1-7.
41. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Lost in quantization: Improving particular object retrieval in large scale image databases," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2008, pp. 1-8.
42. Compact Descriptors for Visual Search: Performance Improvements of the Scalable Compressed Fisher Vector, ISO/IEC JTCI/SC29/WG11 MPEG2013/M28061, Jan. 2013.
43. J. Lin, L.-Y. Duan, T. Huang, and W. Gao, "Robust Fisher codes for large scale image retrieval," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., May 2013, pp. 1513-1517.
44. J. Lin et al., "Rate-adaptive compact Fisher codes for mobile visual search," IEEE Signal Process. Lett., vol. 21, no. 2, pp. 195-198, Feb. 2014.
45. L.-Y. Duan, J. Lin, Z. Wang, T. Huang, and W. Gao, "Weighted component hashing of binary aggregated descriptors for fast visual search," IEEE Trans. Multimedia, to be published.
46. K. van de Sande, T. Gevers, and C. Snoek, "Evaluating color descriptors for object and scene recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1582-1596, Sep. 2010.
47. O. Chum, J. Philbin, J. Sivic, M. Isard, and A. Zisserman, "Total recall: Automatic query expansion with a generative feature model for object retrieval," in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2007, pp. 1-8.
48. O. Chum, A. Mikulik, M. Perdoch, and J. Matas, "Total recall II: Query expansion revisited," inProc. IEEEConf.Comput. Vis. PatternRecog., Jun. 2011, pp. 889-896.
49. R. Arandjelovic and A. Zisserman, "Multiple queries for large scale specific object retrieval," in Proc. Brit. Mach. Vis. Conf., Sep. 2012, pp. 1-8.
50. "Visual query expansion with or without geometry: Refining local descriptors by feature aggregation," Pattern Recog., vol. 47, no. 10, pp. 3466-3476, Oct. 2014.
51. D. Chen, M. Makar, A. Araujo, and B. Girod, "Interframe coding of global image signatures for mobile augmented reality," in Proc. IEEE Data Compression Conf., Mar. 2014, pp. 33-42.
52. M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, no. 6, pp. 381-395, 1981.