A cost-effective method for detecting web site replicas on search engine databases

Data and Knowledge Engineering

Volumn 62, Issue 3, 2007, Pages 421-437

  da Costa Carvalho, André Luiz ^a   de Moura, Edleno Silva ^a   da Silva, Altigran Soares ^a   Berlt, Klessius ^a   Bezerra, Allan ^a  

^a FEDERAL UNIVERSITY OF AMAZONAS   (Brazil)

Author keywords

Mirror; Search engines; Site replication

Indexed keywords

COST EFFECTIVENESS; DATA STORAGE EQUIPMENT; DATABASE SYSTEMS; QUALITY CONTROL; SEARCH ENGINES; USER INTERFACES;

QUALITY IMPROVEMENT; QUERY PROCESSING TIME; SITE REPLICATION;

WEBSITES;

EID: 34248158280     PISSN: 0169023X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.datak.2006.08.010     Document Type: Article

References (14)

1. Bharat K., and Broder A.Z. Mirror, mirror on the web: a study of host pairs with replicated content. Computer Networks 31 11-16 (1999) 1579-1590
2. Calado P.P., Moura E., Ribeiro-Neto B., Silva I., and Ziviani N. Local versus global link information in the web. ACM Transactions on Information Systems (TOIS) 21 1 (2003) 42-63
3. N. Shivakumar, H. Garcia-Molina, Finding near-replicas of documents on the web, in: WEBDB: International Workshop on the World Wide Web and Databases, WebDB, LNCS, 1998, pp. 204-212.
4. Chowdhury A., Frieder O., and Grossman D. Collection statistics for fast duplicate document detection. ACM Transaction on Information Systems 20 2 (2002) 171-191
5. Bharat K., Broder A.Z., Dean J., and Henzinger M.R. A comparison of techniques to find mirrored hosts on the www. IEEE Data Engineering Bulletin 23 4 (2000) 21-26
6. J. Cho, N. Shivakumar, H. Garcia-Molina, Finding replicated web collections, in: Proceedings of ACM SIGMOD Conference, 2000, pp. 355-366.
7. Brin S., and Page L. The anatomy of a large-scale hypertextual web search engine. Proceedings of the 1999 Conference on World Wide Web 30 1-7 (1998) 107-117
8. T. Westerveld, W. Kraaij, D. Hiemstra, Retrieving Web pages using content, links, URLs and anchors, in: The Tenth Text REtrieval Conference (TREC-2001), 2001, pp. 663-672.
9. A. Z. Broder, On the resemblance and containment of documents, in: I.C. Society (Ed.), Proceedings of Compression and Complexity of Sequences, 1997, pp. 21-29.
10. A.S. da Silva, E.A. Veloso, P.B. Golgher, B.A. RibeYiro-Neto, A.H.F. Laender, N. Ziviani, Cobweb - a crawler for the brazilian web., in: Proceedings of the 1999 String Processing and Information Retrieval Symposium, 1999, pp. 184-191.
11. Heydon A., and Najork M. Mercator: a scalable, extensible web crawler. Proceedings of the 1999 Conference on World Wide Web 2 4 (1999) 219-229
12. Salton G., and McGill M.J. Introduction to Modern Information Retrieval. 1st ed. (1983), McGraw-Hill
13. Baeza-Yates R., and Ribeiro-Neto B. Modern Information Retrieval (1999), Addison-Wesley
14. D. Hawking, E. Voorhees, N. Craswell, P. Bailey, Overview of the trec8 web track, in: 8th Text REtrieval Conference, 1999.