Locality sensitive hashing for scalable structural classification and clustering of web documents

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2013, Pages 359-368

  Hachenberg, Christian ^a   Gottron, Thomas ^a  

^a University of Koblenz Landau   (Germany)

Author keywords

Locality sensitive hashing; Template detection; Template fingerprints

Indexed keywords

INFORMATION EXTRACTION TECHNIQUES; LOCALITY SENSITIVE HASH; LOCALITY SENSITIVE HASHING; OPEN DIRECTORY PROJECTS; STRUCTURAL CLASSIFICATION; TEMPLATE DETECTION; TEMPLATE FINGERPRINTS; WEB CONTENT MANAGEMENT SYSTEMS;

CLASSIFICATION (OF INFORMATION); KNOWLEDGE MANAGEMENT; PATTERN RECOGNITION; WORLD WIDE WEB;

INFORMATION RETRIEVAL SYSTEMS;

EID: 84889591088     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2505515.2505673     Document Type: Conference Paper

References (20)

1. Ziv Bar-Yossef and Sridhar Rajagopalan. Template detection via data mining and its applications. In WWW '02: Proceedings of the 11th International Conference on World Wide Web, pages 580-591. ACM Press, 2002.
2. Lorenzo Blanco, Nilesh Dalvi, and Ashwin Machanavajjhala. Highly efficient algorithms for structural clustering of large websites. In Proceedings of the 20th international conference on Worldwide web, WWW'11, pages 437-446. ACM, 2011.
3. David Buttler. A short survey of document structure similarity algorithms. In IC '04: Proceedings of the International Conference on Internet Computing, pages 3-9. CSREA Press, 2004.
4. Deepayan Chakrabarti, Ravi Kumar, and Kunal Punera. Page-level template detection via isotonic smoothing. In WWW '07: Proceedings of the 16th International Conference on World Wide Web, pages 61-70. ACM Press, 2007.
5. Isabel F. Cruz, Slava Borisov, Michael A. Marks, and Timothy R. Webbs. Measuring structural similarity among web documents: preliminary results. In EP '98: Proceedings of the 7th Int. Conference on Electronic Publishing, Artistic Imaging, and Digital Typography, pages 513-524, 1998.
6. Thomas Gottron. Bridging the Gap: From Multi Document Template Detection to Single Document Content Extraction. In EuroIMSA '08: Proceedings of the IASTED Conference on Internet and Multimedia Systems and Applications 2008, pages 66-71, 2008.
7. Thomas Gottron. Clustering template based web documents. In ECIR '08: Proceedings of the 30th European Conference on Information Retrieval, pages 40-51. Springer, 2008.
8. Thomas Gottron. Detecting website redesigns via template similarity on streams of documents. In ITA'09: Proceedings of the 3rd International Conference on Internet Technologies and Applications, pages 35-43, 2009.
9. Thomas Gottron and Roman Schneider. A hybrid approach to statistical and semantical analysis of web documents. In EuroIMSA'09: Proceedings of 5th European Conference on Internet and Multimedia Systems and Applications, pages 115-120, 2009.
10. Sachindra Joshi, Neeraj Agrawal, Raghu Krishnapuram, and Sumit Negi. A bag of paths model for measuring structural similarity in web documents. In KDD '03: Proceedings of the ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 577-582. ACM Press, 2003.
11. VI Levenshtein. Binary Codes Capable of Correcting Deletions, Insertions and Reversals. Soviet Physics Doklady, 10:707-710, 1966.
12. Gurmeet Singh Manku, Arvind Jain, and Anish Das Sarma. Detecting near-duplicates for web crawling. In Proceedings of the 16th international conference on World Wide Web, WWW'07, pages 141-150. ACM, 2007.
13. William M. Rand. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336):846-850, 1971.
14. D. C. Reis, P. B. Golgher, A. S. da Silva, and A. F. Laender. Automatic web news extraction using tree edit distance. In WWW '04: Proceedings of the 13th International Conference on World Wide Web, pages 502-511. ACM Press, 2004.
15. R. Rivest. The md5 message-digest algorithm, 1992.
16. Lei Shi, Cheng Niu, Ming Zhou, and Jianfeng Gao. A DOM tree alignment model for mining parallel data from the web. In ACL '06: Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL, pages 489-496. Association for Computational Linguistics, 2006.
17. Benno Stein. Principles of Hash-based Text Retrieval. In SIGIR'07: Proceedings of the 30th International ACM Conference on Research and Development in Information Retrieval, pages 527-534. ACM, July 2007.
18. Martin Theobald, Jonathan Siddharth, and Andreas Paepcke. Spotsigs: robust and efficient near duplicate detection in large web collections. In SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pages 563-570, New York, NY, USA, 2008. ACM.
19. Karane Vieira, André da Costa Carvalho, Klessius Berlt, Edleno de Moura, Altigran da Silva, and Juliana Freire. On finding templates on web collections. World Wide Web, 12:171-211, 2009. 10.1007/s11280-009-0059-3.
20. T. A. Welch. A technique for high-performance data compression. Computer, 17(6):8-19, 1984.