Evaluating methods to rediscover missing web pages from the web infrastructure

Proceedings of the ACM International Conference on Digital Libraries

Volumn , Issue , 2010, Pages 59-68

  Klein, Martin ^a   Nelson, Michael L ^a  

^a ECS Building   (United States)

Author keywords

Digital preservation; Search engines; Web page discovery

Indexed keywords

AUTOMATED METHODS; BOOKMARKING; DIGITAL PRESERVATION; EVALUATING METHOD; INTERNET SEARCH ENGINE; RETRIEVAL PERFORMANCE; WEB INFRASTRUCTURE; WEB PAGE;

DIGITAL IMAGE STORAGE; INFORMATION RETRIEVAL; SEARCH ENGINES; WEBSITES;

DIGITAL LIBRARIES;

EID: 77955099622     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1816123.1816133     Document Type: Conference Paper

References (40)

1. How does Google calculate the number of results? http://www.google.com/ support/webmasters/bin/answer.py?hl=en&answer=70920.
2. The size of the World Wide Web. http://www.worldwidewebsize.com/.
3. H. Ashman. Electronic document addressing: Dealing with change. ACM Computing Surveys, 32(3):201-212, 2000.
4. H. Ashman, H. Davis, J. Whitehead, and S. Caughey. Missing the 404: Link integrity on the world wide web. In Proceedings of WWW '98, pages 761-762, 1998.
5. R. Baeza-Yates, Álvaro Pereira, and N. Ziviani. Genealogical trees on the web: a search engine user perspective. In Proceeding of WWW '08, pages 367-376, 2008.
6. T. Berners-Lee. Cool URIs don't change, 1998. http://www.w3.org/Provider/ Style/URI.html.
7. K. Bischoff, C. Firan, W. Nejdl, and R. Paiu. Can All Tags Be Used for Search? In Proceedings of CIKM '08, pages 193-202, 2008.
8. H. C. Davis. Hypertext link integrity. ACM Computing Surveys, page 28.
9. H. C. Davis. Referential integrity of links in open hypermedia systems. In Proceedings of HYPERTEXT '98, pages 207-216, 1998.
10. B. D. Davison. Topical locality in the web. In Proceedings of SIGIR '00, pages 272-279, 2000.
11. J. Dean and M. R. Henzinger. Finding Related Pages in the World Wide Web. Computer Networks, 31(11-16):1467-1479, 1999.
12. R. P. Dellavalle, E. J. Hester, L. F. Heilig, A. L. Drake, J. W. Kuntzman, M. Graber, and L. M. Schilling. INFORMATION SCIENCE: Going, Going, Gone: Lost Internet References. Science, 302(5646):787-788, 2003.
13. T. L. Harrison and M. L. Nelson. Just-in-Time Recovery of Missing Web Pages. In Proceedings of HYPERTEXT '06, pages 145-156, 2006.
14. B. He and I. Ounis. Inferring Query Performance Using Pre-retrieval Predictors. In Proceedings of SPIRE '04, pages 43-54, 2004.
15. M. Henzinger, B.-W. Chang, B. Milch, and S. Brin. Query-free News Search. In Proceedings of WWW '03, pages 1-10, 2003.
16. M. R. Henzinger, A. Heydon, M. Mitzenmacher, and M. Najork. On Near-Uniform URL Sampling. Computer Networks, 33(1-6):295-308, 2000.
17. A. Jatowt, Y. Kawai, S. Nakamura, Y. Kidawara, and K. Tanaka. A Browser for Browsing the Past Web. In Proceedings of WWW '06, pages 877-878, 2006.
18. B. Kahle. Preserving the Internet. Scientific American, 276:82-83, March 1997.
19. F. Keller and M. Lapata. Using the Web to Obtain Frequencies for Unseen Bigrams. Computational Linguistics, 29(3):459-484, 2003.
20. M. Klein and M. L. Nelson. A Comparison of Techniques for Estimating IDF Values to Generate Lexical Signatures for the Web. In Proceedings of WIDM '08, pages 39-46, 2008.
21. M. Klein and M. L. Nelson. Revisiting Lexical Signatures to (Re-)Discover Web Pages. In Proceedings of ECDL '08, pages 371-382, 2008.
22. M. Klein and M. L. Nelson. Inter-Search Engine Lexical Signature Performance. In Proceedings of JCDL '09, pages 413-414, 2009.
23. W. C. Koehler. Web Page Change and Persistence - A Four-Year Longitudinal Study. Journal of the American Society for Information Science and Technology, 53(2):162-171, 2002.
24. S. Lawrence, D. M. Pennock, G. W. Flake, R. Krovetz, F. M. Coetzee, E. Glover, F. A. Nielsen, A. Kruger, and C. L. Giles. Persistence of Web References in Scientific Research. Computer, 34(2):26-31, 2001.
25. C. C. Marshall, F. McCown, and M. L. Nelson. Evaluating Personal Archiving Strategies for Internet-based Information. In Proceedings of IS&T Archiving '07, pages 48-52, 2007.
26. F. McCown. Lazy Preservation: Reconstructing Websites from the Web Infrastructure. PhD thesis, Old Dominion University, 2007.
27. F. McCown and M. L. Nelson. Characterization of Search Engine Caches. In Proceedings of IS&T Archiving '07, pages 48-52, 2007. (Also available as arXiv:cs/0703083v2).
28. F. McCown, J. A. Smith, and M. L. Nelson. Lazy Preservation: Reconstructing Websites by Crawling the Crawlers. In Proceedings of WIDM '06, pages 67-74, 2006.
29. M. L. Nelson, F. McCown, J. A. Smith, and M. Klein. Using the Web Infrastructure to Preserve Web Pages. IJDL, 6(4):327-349, 2007.
30. S.-T. Park, D. M. Pennock, C. L. Giles, and R. Krovetz. Analysis of Lexical Signatures for Improving Information Persistence on the World Wide Web. ACM Transactions on Information Systems, 22(4):540-572, 2004.
31. T. A. Phelps and R. Wilensky. Robust Hyperlinks Cost Just Five Words Each. Technical Report UCB//CSD-00-1091, University of California at Berkeley, Berkeley, CA, USA, 2000.
32. P. Rusmevichientong, D. M. Pennock, S. Lawrence, and C. L. Giles. Methods for Sampling Pages Uniformly from the World Wide Web. In AAAI Fall Symposium on Using Uncertainty Within Computation, pages 121-128, 2001.
33. G. Salton, A. Wong, and C. S. Yang. A Vector Space Model for Automatic Indexing. Communications of the ACM, 18(11):613-620, 1975.
34. A. J. Silva, M. A. Goncalves, A. H. Laender, M. A. Modesto, M. Cristo, and N. Ziviani. Finding What is Missing from a Digital Library: A Case Study in the Computer Science Field. Information Processing and Management, 45(3):380 - 391, 2009.
35. D. Spinellis. The decay and failures of web references. Communications of the ACM, 46(1):71-77, 2003.
36. K. Sugiyama, K. Hatano, M. Yoshikawa, and S. Uemura. Refinement of TF-IDF Schemes for Web Pages using their Hyperlinked Neighboring Pages. In Proceedings of HYPERTEXT '03, pages 198-207, 2003.
37. M. Theall. Methodologies for Crawler Based Web Surveys. Internet Research: Electronic Networking and Applications, 12:124-138, 2002.
38. X. Wan and J. Yang. Wordrank-based Lexical Signatures for Finding Lost or Related Web Pages. In A P Web, pages 843-849, 2006.
39. X. Zhu and R. Rosenfeld. Improving Trigram Language Modeling with the World Wide Web. In Proceedings of ICASSP '01, pages 533-536, 2001.
40. Z. Zhuang, R. Wagle, and C. L. Giles. What's There and What's Not?: Focused Crawling for Missing Documents in Digital Libraries. In Proceedings of JCDL '05, pages 301-310, 2005.