Finding near-duplicate web pages: A large-scale evaluation of algorithms

Proceedings of the Twenty-Ninth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn 2006, Issue , 2006, Pages 284-291

  Henzinger, Monika ^a  

^a EPFL   (Switzerland)

Author keywords

Content duplication; Near duplicate documents; Web pages

Indexed keywords

CONTENT DUPLICATION; NEAR-DUPLICATE DOCUMENTS; CONTENT DUPLICATIONS; NEAR DUPLICATE DOCUMENTS; PRECISIONS; SHINGLING ALGORITHMS;

ALGORITHMS; ELECTRONIC DOCUMENT EXCHANGE; SEARCH ENGINES; DATA STRUCTURES; LARGE SCALE SYSTEMS; RANDOM PROCESSES;

WEBSITES;

EID: 33750296887     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. S. Brin, J. Davis, and H. Garcia-Molina. Copy Detection Mechanisms for Digital Documents. In 1995 ACM SIGMOD International Conference on Management of Data (May 1995), 398-409.
2. A. Broder. Some applications of Rabin's fingerprinting method. In Renato Capocelli, Alfredo De Santis, and Ugo Vaccaro, editors, Sequences II: Methods in Communications, Security, and Computer Science, 1993:143-152.
3. A. Broder, S. Glassman, M. Manasse, and G. Zweig. Syntactic Clustering of the Web. In 6th International World Wide Web Conference (Apr. 1997), 393-404.
4. M. S. Charikar. Similarity Estimation Techniques from Rounding Algorithms. In 34th Annual ACM Symposium on Theory of Computing (May 2002).
5. M. S. Charikar. Private communication.
6. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In 6th Symposium on Operating System Design and Implementation (Dec. 2004), 137-150.
7. D. Fetterly, M. Manasse, and M. Najork. On the Evolution of Clusters of Near-Duplicate Web Pages. In 1st Latin American Web Congress (Nov. 2003), 37-45.
8. D. Fetterly, M. Manasse, and M. Najork. Detecting Phrase-Level Duplication on the World Wide Web. To appear in 28th Annual International ACM SIGIR Conference (Aug. 2005).
9. N. Heintze. Scalable Document Fingerprinting. In Proc. of the 2nd USENIX Workshop on Electronic Commerce (Nov 1996).
10. T. C. Hoad and J. Zobel. Methods for identifying versioned and plagiarised documents. Journal of the American Society for Information Science and Technology 54(3):203-215, 2003.
11. U. Manber. Finding similar files in a large file system. In Proc. of the USENIX Winter 1994 Technical Conference (Jan. 1994).
12. M. Rabin. Fingerprinting by random polynomials. Report TR-15-81, Center for Research in Computing Technology, Harvard University, 1981.
13. N. Shivakumar and H. Garcia-Molina. SCAM: a copy detection mechanism for digital documents. In Proc. International Conference on Theory and Practice of Digital Libraries (June 1995).
14. N. Shivakumar and H. Garcia-Molina. Building a scalable and accurate copy detection mechanism. In Proc. ACM Conference on Digital Libraries (March 1996), 160-168.
15. N. Shivakumar and H. Garcia-Molina. Finding near-replicas of documents on the web. In Proc. Workshop on Web Databases (March 1998), 204-212.