Instant mobile video search with layered audio-video indexing and progressive transmission

IEEE Transactions on Multimedia

Volumn 16, Issue 8, 2014, Pages 2242-2255

  Liu, Wu ^a,b   Mei, Tao ^c   Zhang, Yongdong ^a  

^a INSTITUTE OF COMPUTING TECHNOLOGY   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^c MICROSOFT RESEARCH   (United States)

Author keywords

Audio video signature; Layered audio video indexing; Mobile video search; Progressive transmission

Indexed keywords

AUDIO EQUIPMENT; CELLULAR TELEPHONE SYSTEMS; INDEXING (OF INFORMATION); MOBILE DEVICES; SMARTPHONES; TELEPHONE SETS; VIDEO RECORDING;

COMPUTING CAPACITY; MOBILE VIDEO; NOVEL APPLICATIONS; PROGRESSIVE TRANSMISSION; SEARCH ACCURACY; VIDEO CONTENTS; VIDEO INDEXING; VIDEO SIGNATURES;

SEARCH ENGINES;

EID: 84912114379     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2014.2359332     Document Type: Article

References (39)

1. H. Liu, T. Mei, J. Luo, H. Li, and S. Li, "Finding perfect rendezvous on the go: Accurate mobile visual localization and its applications to routing," in Proc. ACM Multimedia, 2012, pp. 9-18.
2. B. Girod, V. Chandrasekhar, D. M. Chen, N.-M. Cheung, R. Grzeszczuk, Y. A. Reznik, G. Takacs, S. S. Tsai, and R. Vedantham, "Mobile visual search," IEEE Signal Process. Mag., vol. 28, no. 4, pp. 61-76, 2011.
3. H. Li, Y. Wang, T. Mei, J. Wang, and S. Li, "Interactive multimodal visual search on mobile device," IEEE Trans. Multimedia, vol. 15, no. 3, pp. 594-607, May 2013.
4. E. Miluzzo, N. D. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. B. Eisenman, X. Zheng, and A. T. Campbell, "Sensing meets mobile social networks: The design, implementation and evaluation of the cenceme application," ACM SenSys, pp. 337-350, 2008.
5. J. Liu, Z. Huang, H. Cai, H. T. Shen, C.-W. Ngo, and W. Wang, "Near-duplicate video retrieval: Current research and future trends," in Proc. ACM Comput. Surv., 2013, vol. 45, no. 4, p. 44.
6. Y.-C. Su, T.-H. Chiu, Y.-Y. Chen, C.-Y. Yeh, and W. H. Hsu, "Enabling low bitrate mobile visual recognition: A performance versus bandwidth evaluation," in Proc. ACM Multimedia, 2013, pp. 73-82.
7. W. Liu, T. Mei, Y. Zhang, J. Li, and S. Li, "Listen, look, and gotcha: Instant video search with mobile phones by layered audio-video indexing," in Proc. ACM Multimedia, 2013, pp. 887-896.
8. J. Sang, T. Mei, Y.-Q. Xu, C. Zhao, C. Xu, and S. Li, "Interaction design for mobile visual search," IEEE Trans. Multimedia, vol. 15, no. 7, pp. 1665-1676, Jul. 2013.
9. "Google Goggles," [Online]. Available: http://www.google.com/mobile/goggles/
10. X. Yang and K.-T. T. Cheng, "Accelerating surf detector on mobile devices," in Proc. ACM Multimedia, 2012, pp. 569-578.
11. J. Xia, K. Gao, D. Zhang, and Z. Mao, "Geometric context-preserving progressive transmission in mobile visual search," in Proc. ACM Multimedia, 2012, pp. 953-956.
12. V. R. Chandrasekhar, S. S. Tsai, G. Takacs, D. M. Chen, N.-M. Cheung, Y. Reznik, R. Vedantham, R. Grzeszczuk, and B. Girod, "Low latency image retrieval with progressive transmission of chog descriptors," in Proc. ACM Multimedia on Mobile Cloud Media Computing, 2010, pp. 41-46.
13. V. Chandrasekhar, G. Takacs, D. M. Chen, S. S. Tsai, Y. Reznik, R. Grzeszczuk, and B. Girod, "Compressed histogram of gradients: A low-bitrate descriptor," Int. J. Comput. Vision, vol. 96, no. 3, pp. 384-399, 2012.
14. R. Ji, L.-Y. Duan, J. Chen, H. Yao, Y. Rui, S.-F. Chang, and W. Gao, "Towards low bit rate mobile visual search with multiple-channel coding," in Proc. ACM Multimedia, 2011, pp. 573-582.
15. J. He, J. Feng, X. Liu, T. Cheng, T.-H. Lin, H. Chung, and S.-F. Chang, "Mobile product search with bag of hash bits and boundary reranking," in Proc. CVPR, 2012, pp. 3005-3012.
16. G.-L. Wu, Y.-H. Kuo, T.-H. Chiu, W. H. Hsu, and L. Xie, "Scalable mobile video retrieval with sparse projection learning and pseudo label mining," IEEE MultiMedia, vol. 20, no. 3, pp. 47-57, 2013.
17. H. Bay, A. Ess, T. Tuytelaars, and L. J. V. Gool, "Speeded-Up Robust Features (SURF)," Comput. Vis. Image Understanding, vol. 110, no. 3, pp. 346-359, 2008.
18. J. Sivic and A. Zisserman, "Video google: A text retrieval approach to object matching in videos," in Proc. ICCV, 2003, pp. 1470-1477.
19. K.-Y. Tseng, Y.-L. Lin, Y.-H. Chen, and W. H. Hsu, "Sketch-based image retrieval on mobile devices using compact hash bits," in Proc. ACM Multimedia, 2012, pp. 913-916.
20. D. M. Chen, N.-M. Cheung, S. S. Tsai, V. Chandrasekhar, G. Takacs, R. Vedantham, R. Grzeszczuk, and B. Girod, "Dynamic selection of a feature-rich query frame for mobile video retrieval," in Proc. ICIP, 2010, pp. 1017-1020.
21. A. Wang, "An industrial strength audio search algorithm," in Proc. ISMIR, 2003, pp. 7-13.
22. Y. Wu, S. Lu, T. Mei, J. Zhang, and S. Li, "Local visual words coding for low bit rate mobile visual search," in Proc. ACM Multimedia, 2012, pp. 989-992.
23. Y. Weiss, A. Torralba, and R. Fergus, "Spectral hashing," in Proc. NIPS, 2008, pp. 1753-1760.
24. M. Norouzi and D. J. Fleet, "Minimal loss hashing for compact binary codes," in Proc. ICML, 2011, pp. 353-360.
25. L. Zhang, Y. Zhang, X. Gu, J. Tang, and Q. Tian, "Scalable similarity search with topology preserving hashing," IEEE Trans. Image Process., vol. 23, no. 7, pp. 3025-3039, Jul. 2014.
26. L. Pang, W. Zhang, H.-K. Tan, and C.-W. Ngo, "Video hyperlinking: Libraries and tools for threading and visualizing large video collection," in Proc. ACM Multimedia, 2012, pp. 1461-1464.
27. J. Wan, S. Tang, Y. Zhang, L. Huang, and J. Li, "Data driven multi-index hashing," in Proc. ICIP, 2013, pp. 2670-956.
28. R. Salakhutdinov and G. Hinton, "Semantic hashing," Int. J. Approx. Reasoning, vol. 50, no. 7, pp. 969-978, Jul. 2009.
29. A. Andoni and P. Indyk, "Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions," Commun. ACM, vol. 51, no. 1, pp. 117-122, Jan. 2008.
30. Q. Lv, W. Josephson, Z. Wang, M. Charikar, and K. Li, "Multi-probe lsh: Efficient indexing for high-dimensional similarity search," in Proc. VLDB, 2007, pp. 950-961.
31. M. Norouzi, A. Punjani, and D. J. Fleet, "Fast search in hamming space with multi-index hashing," in Proc. CVPR, 2012, pp. 3108-3115.
32. Y. Liu, C. Xu, and H. Lu, "Audio-visual large-scale video copy detection," Int. J. Comput. Math., vol. 88, no. 18, pp. 3803-3816, 2011.
33. C. Snoek, M. Worring, and A. W. M. Smeulders, "Early versus late fusion in semantic video analysis," in Proc. ACM Multimedia, 2005, pp. 399-402.
34. T. Mei, Y. Rui, S. Li, and Q. Tian, "Multimedia search reranking: A literature survey," ACM Computing Surveys, vol. 46, no. 3, pp. 1-38, Jan 2014.
35. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object retrieval with large vocabularies and fast spatial matching," in Proc. CVPR, 2007, pp. 1-8., 2007, pp. 1-8.
36. H. Xie, K. Gao, Y. Zhang, S. Tang, J. Li, and Y. Liu, "Efficient feature detection and effective post-verification for large scale near-duplicate image search," IEEE Trans. Multimedia, vol. 13, no. 6, pp. 1319-1332, Jun. 2011.
37. H. T. Shen, J. Shao, Z. Huang, and X. Zhou, "Effective and efficient query processing for video subsequence identification," IEEE Trans. Knowl. Data Eng., vol. 21, no. 3, pp. 321-334, 2009.
38. M. Muja and D. G. Lowe, "Fast matching of binary features," in Proc. ICRV, 2012, pp. 404-410.
39. Y. Zhang, L. Zhang, and Q. Tian, "A prior-free weighting scheme for binary code ranking," IEEE Trans. Multimedia, vol. 16, no. 4, pp. 1127-1139, Apr. 2014.