Learning hash codes for efficient content reuse detection

SIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2012, Pages 405-414

  Zhang, Qi ^a   Wu, Yan ^a   Ding, Zhuoye ^a   Huang, Xuanjing ^a  

^a FUDAN UNIVERSITY   (China)

Author keywords

content reuse detection; GPUs; learning to hash

Indexed keywords

CONTENT RE-USE; DETECTION APPROACH; DETECTION RATES; DOCUMENT COLLECTION; EXECUTION TIME; EXPERIMENTAL COMPARISON; GPUS; GRAPHICAL PROCESSING UNIT (GPUS); LEARNING TO HASH; SCALABLE APPROACH; SIGNATURE GENERATION; STATE-OF-THE-ART SYSTEM;

HASH FUNCTIONS; PROGRAM PROCESSORS;

INFORMATION RETRIEVAL;

EID: 84866605487     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2348283.2348339     Document Type: Conference Paper

References (33)

1. J. Aberdeen, J. Burger, D. Day, L. Hirschman, P. Robinson, and M. Vilain. Mitre: description of the alembic system used for muc-6. In Proceedings of MUC6, pages 141-155, Morristown, NJ, USA, 1995.
2. R. J. Bayardo, Y. Ma, and R. Srikant. Scaling up all pairs similarity search. In Proceedings of the 16th international conference on World Wide Web, WWW '07, pages 131-140, 2007.
3. J. Bolz, I. Farmer, E. Grinspun, and P. Schröoder. Sparse matrix solvers on the gpu: conjugate gradients and multigrid. ACM Trans. Graph., 22:917-924, July 2003.
4. A. Z. Broder. On the resemblance and containment of documents. In Proceedings of SEQUENCES 1997, page 21, Washington, DC, USA, 1997. IEEE Computer Society.
5. A. Z. Broder. Identifying and filtering near-duplicate documents. In Proceedings of COM 2000, pages 1-10, London, UK, 2000. Springer-Verlag.
6. B. Bustos, O. Deussen, S. Hiller, and D. Keim. A graphics hardware accelerated algorithm for nearest neighbor search. In V. Alexandrov, G. van Albada, P. Sloot, and J. Dongarra, editors, Computational Science ĺC ICCS 2006, volume 3994 of Lecture Notes in Computer Science, pages 196-199. Springer Berlin / Heidelberg, 2006.
7. A. Chowdhury, O. Frieder, D. Grossman, and M. C. McCabe. Collection statistics for fast duplicate document detection. ACM Trans. Inf. Syst., 20(2):171-191, 2002.
8. L. Dugan. 230 million tweets per day, 50 million daily users and other twitter stats. WWW.MEDIABISTRO.COM, 2011.
9. S. Edelkamp, D. Sulewski, and C. Yĺźcel. Perfect hashing for state space exploration on the gpu. In ICAPS, pages 57-64. AAAI, 2010.
10. K. Fatahalian, J. Sugerman, and P. Hanrahan. Understanding the efficiency of gpu algorithms for matrix-matrix multiplication. In Proceedings of the ACM SIGGRAPH/EUROGRAPHICS conference on Graphics hardware, HWWS '04, pages 133-137, 2004.
11. A. Gionis, P. Indyk, and R. Motwani. Similarity search in high dimensions via hashing. In VLDB '99, pages 518-529, San Francisco, CA, USA, 1999. Morgan Kaufmann Publishers Inc.
12. N. K. Govindaraju, M. Henson, M. C. Lin, and D. Manocha. Interactive visibility ordering and transparency computations among geometric primitives in complex environments. In Proceedings of the 2005 symposium on Interactive 3D graphics and games, I3D '05, pages 49-56, 2005.
13. G. Hinton and R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, 2006.
14. G. E. Hinton, S. Osindero, and Y.-W. Teh. A fast learning algorithm for deep belief nets. Neural Comput., 18:1527-1554, July 2006.
15. J. W. Kim, K. S. Candan, and J. Tatemura. Efficient overlap and content reuse detection in blogs and online news articles. In Proceedings of the 18th international conference on World wide web, WWW '09, pages 81-90, 2009.
16. A. Kołcz, A. Chowdhury, and J. Alspector. Improved robustness of signature-based near-replica detection via lexicon randomization. In Proceedings of SIGKDD 2004, pages 605-610, 2004.
17. J. Krüger and R. Westermann. Linear algebra operators for gpu implementation of numerical algorithms. In ACM SIGGRAPH 2005 Courses, SIGGRAPH '05, 2005.
18. D. Metzler, Y. Bernstein, W. B. Croft, A. Moffat, and J. Zobel. Similarity measures for tracking information flow. In Proceedings of the 14th ACM international conference on Information and knowledge management, pages 517-524, 2005.
19. K. Muthmann, W. M. Barczyński, F. Brauer, and A. Löser. Near-duplicate detection for web-forums. In IDEAS '09, pages 142-151, New York, NY, USA, 2009. ACM.
20. M. Norouzi and D. Fleet. Minimal loss hashing for compact binary codes. In L. Getoor and T. Scheffer, editors, Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 353-360, June 2011.
21. J. D. Owens, M. Houston, D. Luebke, S. Green, J. E. Stone, and J. C. Phillips. GPU Computing. Proceedings of the IEEE, 96(5):879-899, 2008.
22. R. Rivest. The md5 message-digest algorithm, 1992.
23. J. Seo and W. B. Croft. Local text reuse detection. In SIGIR '08, pages 571-578, New York, NY, USA, 2008.
24. N. Shivakumar and H. Garcia-Molina. Scam: A copy detection mechanism for digital documents. In Digitial Library, 1995.
25. N. Shivakumar and H. Garcia-Molina. Finding near-replicas of documents and servers on the web. In Proceedings of WebDB 1998, pages 204-212, London, UK, 1999. Springer-Verlag.
26. T. Skopal, J. Pokorný, and V. Snásel. Pm-tree: Pivoting metric tree for similarity search in multimedia databases. In ADBIS (Local Proceedings), 2004.
27. B. Stein. Principles of hash-based text retrieval. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '07, pages 527-534, 2007.
28. M. Theobald, J. Siddharth, and A. Paepcke. Spotsigs: robust and efficient near duplicate detection in large web collections. In SIGIR '08, pages 563-570, New York, NY, USA, 2008. ACM.
29. A. Wächter and L. T. Biegler. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming, 106(1):25-57, Mar. 2006.
30. Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems, pages 1753-1760. MIT Press, 2008.
31. C. Xiao, W. Wang, X. Lin, and J. X. Yu. Efficient similarity joins for near duplicate detection. In Proceeding of the 17th international conference on World Wide Web, WWW '08, pages 131-140, 2008.
32. D. Zhang, J. Wang, D. Cai, and J. Lu. Self-taught hashing for fast similarity search. In Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval, SIGIR '10, pages 18-25, 2010.
33. Q. Zhang, Y. Zhang, H. Yu, and X. Huang. Efficient partial-duplicate detection based on sequence matching. In Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval, SIGIR '10, pages 675-682, 2010.