Geographical partition for distributed web crawling

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2005, Pages 55-60

  Exposto, José ^a   Macedo, Joaquim ^b   Pina, António ^b   Alves, Albano ^a   Rufino, José ^a  

^a INSTITUTO POLITÉCNICO DE BRAGANÇA   (Portugal)

^b UNIVERSITY OF MINHO   (Portugal)

Author keywords

parallel crawling; web mining; web partitioning

Indexed keywords

DISTRIBUTED CRAWLER; DISTRIBUTED CRAWLING; DISTRIBUTED WEB CRAWLING; EVALUATION RESULTS; GEOGRAPHICAL LOCATIONS; GEOGRAPHICAL ZONES; PARTITIONING STRATEGIES; PORTUGUESE WEB; WEB MINING; WEB SERVERS;

INFORMATION RETRIEVAL; SERVERS; WORLD WIDE WEB;

KNOWLEDGE MANAGEMENT;

EID: 34250682644     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1096985.1096999     Document Type: Conference Paper

References (17)

1. C. C. Aggarwal, F. Al-Garawi, and P. S. Yu. Intelligent crawling on the world wide web with arbitrary predicates. In World Wide Web, pages 96-105, 2001.
2. A.-L. Barabási and R. Albert. Emergence of scaling in random networks. SIAM Journal on Scientific Computing, 286(5439):509-512, 1999.
3. CAIDA. NetGeo - The Internet Geographic Database. http://www.caida.org/ tools/utilities/netgeo/, 2002.
4. S. Chakrabarti, M. van den Berg, and B. Dom. Focused crawling: A new approach to topic-specific web resource discovery. In Proceedings of the 8th International WWW Conference, 1999.
5. J. Cho and H. Garcia-Molina. Parallel crawlers. In Proc. of the 11th International World-Wide Web Conference, 2002.
6. C. Chung and C. Clarke. Topic-oriented collaborative crawling. In 11th International Conference on Information and Knowledge Management (CIKM'02), pages 34-42, 2002.
7. J. Ding, L. Gravano, and N. Shivakumar. Computing geographical scopes of web resources. In 26th International Conference on Very Large Databases, VLDB 2000, Cairo, Egypt, September 10-14 2000.
8. J. Exposto, J. Macedo, A. Pina, A. Alves, and J. Rufino". Multi-objective web graph partitioning for efficient distributed web crawling. Technical report, University of Minho, Department of Computer Science (Work in progress), 2005.
9. Jung the java universal network/graph framework. http://jung.sourceforge. net/, 2005.
10. G. Karypis and V. Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on Scientific Computing, 20(1):359-392, 1998.
11. G. Karypis and V. Kumar. A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices - version 4.0, 1998.
12. J. Macedo, A. Pina, P. Azevedo, O. Belo, M. Santos, J. J. Almeida, and L. Silva. NetCensus Project. http://marco.uminho.pt/macedo/netcensus/, 2001.
13. R. Periakaruppan and E. Nemeth. GTrace: A graphical traceroute tool. In 13th Conference on Systems Administration (LISA-99), pages 69-78, 1999.
14. K. Schloegel, G. Karypis, and V. Kumar. A new algorithm for multi-objective graph partitioning. Technical report, University of Minnesota, Department of Computer Science, 1999.
15. M. J. Silva, B. Martins, M. Chaves, N. Cardoso, and A. P. Afonso. Adding geographic scopes to web resources. In ACM SIGIR 2004 Workshop on Geographic Information Retrieval, 2004.
16. US National Geospatial Intelligence Agency. Geographic Names Database. http://earth-info.nima.mil/gns/html/index.html.
17. XLDB Group. WPT03. Linguateca, http://www.linguateca.pt, 2003.