Efficient update of indexes for dynamically changing Web documents

World Wide Web

Volumn 10, Issue 1, 2007, Pages 37-69

  Lim, Lipyeow ^a   Wang, Min ^a   Padmanabhan, Sriram ^b   Vitter, Jeffrey Scott ^c   Agarwal, Ramesh ^b  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b IBM ALMADEN RESEARCH CENTER   (United States)

^c PURDUE UNIVERSITY   (United States)

Author keywords

Document management; Indexing methods; Web search

Indexed keywords

DOCUMENT MANAGEMENT; INDEXING METHODS; WEB SEARCH;

ALGORITHMS; INFORMATION MANAGEMENT; SEARCH ENGINES; SYNCHRONIZATION; WORLD WIDE WEB;

INDEXING (OF INFORMATION);

EID: 33947313305     PISSN: 1386145X     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11280-006-0009-2     Document Type: Article

References (33)

1. Apparao, V., Byrne, S., Champion, M., Isaacs, S., Jacobs, I., Le Hors, A., Nicol, O., Robie, J., Sutor, R., Wilson, C., Wood, L.: Document Object Model Level 1. (W3C Recommendation) http://www.w3.org/TR/REC-DOM-Level-1/ (1998)
2. Arge, L., Procopiuc, O., Ramaswamy, S., Suel, T., Vahrenhold, J., Vitter, J.S.: A unified approach for indexed and non-indexed spatial joins. In: Proceedings of the 7th Intl. Conf. on Extending Database Technology (EDBT '00), vol. 1777, pp. 413-429 (2000)
3. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley, Reading, MA (1999)
4. Baeza-Yates, R.A., Navarro, G.: Block addressing indices for approximate text retrieval. Journal of the American Society on Information Systems 51(1), 69-82 (2000)
5. Boyer, R.S., Moore, J.S.: A fast string searching algorithm. Commun. ACM 20, 762-772 (1976)
6. Brewington, B., Cybenko, G.: Keeping up with the changing web. IEEE Comput. 33(5), 52-58 (May 2000)
7. Broder, A.Z.: On the resemblence and containment of documents. In: Proceedings of Compression and Complexity of Sequences 1997, pp. 21-29 (1997)
8. Brown, E.W., Callan, J.P., Croft, W.B.: Fast incremental indexing for full-text information retrieval. In: 20th Intl. Conf. on Very Large Data Bases, pp. 192-202 (1994)
9. Cho, J., Garcia-Molina, H.: The evolution of the web and implications for an incremental crawler. In: 26th Intl. Conf. on Very Large Data Bases (2000)
10. Cho, J., Garcia-Molina, H.: Estimating frequency of change. ACM Transactions on Internet Technology 3(3), 256-290 (2003)
11. Clarke, C., Cormack, G.: Dynamic inverted indexes for a distributed full-text retrieval system. Tech. report CS-95-01, University of Waterloo CS Dept. (1995)
12. Clarke, C., Cormack, G., Burkowski, F.: Fast inverted indexes with on-line update. Tech. report CS-94-40, University of Waterloo CS Dept. (1994)
13. Cutting, D., Perdersen, J.: Optimizations for dynamic inverted index maintenance. In: Proceedings of SIGIR, pp. 405-411 (1990)
14. Easton, M.C.: Key-sequence data sets on indelible storage. IBM J. Res. Develop. 230-241 (May 1986)
15. Fetterly, D., Manasse, M., Najork, M., Wiener, J.: A large-scale study of the evolution of web pages. In: Proceedings of the 12th Intl. conf. on World Wide Web, pp. 669-678. ACM Press (2003)
16. Fischer, M.J., Ladner, R.E.: Data structures for efficient implementation of sticky pointers in text editors. Dept. of Computer Science. Univ. of Washington, Tech. Report 79-06-08 (June 1979)
17. Frakes, W., Baeza-Yates, R. (eds.): Information Retrieval: Data Structures and Algorithms. Prentice-Hall, Englewood Cliffs, NJ (1992)
18. Kaszkiel, M., Zobel, J.: Passage retrieval revisited. In: Proceedings of the 20th Annual Intl. ACM SIGIR Conf., pp. 178-185. ACM (1997)
19. Knuth, D.E., Morris, J.H., Pratt, V.B.: Fast pattern matching in strings. SIAM J. Comput. 6, 323-350 (1977)
20. Lawrence, S., Lee Giles, C.: Accessibility of information on the web. Nature 400,107-109 (1999)
21. Li, Q., Moon, B.: Indexing and querying xml data for regular path expressions. In: 27th Intl. Conf. on Very Large Data Bases, pp. 361-370 (2001)
22. Lim, L., Wang, M., Padmanabhan, S., Vitter, J.S., Agarwal, R.C.: Characterizing web document change. In: Advances in Web-age Information Management, 2nd Intl. Conf., WA1M 2001, pp. 133-144 (2001)
23. Manber, U., Wu, S.: GLIMPSE: a tool to search through entire file systems. In: Proceedings of the Winter 1994 USENIX Conf., pp. 23-32. USENIX (1994)
24. Melnik, S., Raghavan, S., Yang, B., Garcia-Molina, H.: Building a distributed full-text index for the web. In: Proceedings of the 10th Intl. WWW Conf. (2001)
25. Page, L., Brin, S.: The anatomy of a large-scale hypertextual web search engine. In: Proceedings of the 7th Intl. WWW Conf., pp. 107-1.1.7 (1998)
26. Salton, G., Allan, J., Buckley, C.: Approaches to passage retrieval in full text information systems. In: Korfhage, R., Rasmussen, E.M., Willett, P. (eds.) Proceedings of the 16th Annual Intl. ACM-SIGIR Conf, pp. 49-58. ACM (1993)
27. Tomasic, A., Garcia-Molina, H., Shoens, K.: Incremental updates of inverted lists for text document retrieval. In: Proceedings of 1994 ACM SIGMOD Intl. Conf. of Management of Data, pp. 289-300 (May 1994)
28. Ukkonen, E.: Algorithms for approximate string matching. Inf. Control 64, 100-118 (1985)
29. Vitter, J.S.: Faster methods for random sampling. Commun. ACM, 27 (July 1984)
30. Vitter, J.S.: An efficient I/O interface for optical disks. ACM Trans, on Database Systems, pp. .129-1.62 (June 1985)
31. Witten, I.H., Moffat, A., Bell, T.C.: Managing gigabytes: compressing and indexing documents and images. Morgan Kaufmann, Los Altos, CA 94022, USA, second edition (1999)
32. Zhang, C., Naughton, J.F., DeWitt, D.J., Luo, Q., Lohman, G.: On supporting containment queries in relational database management systems. In: Proceedings of 2001. ACM SIGMOD Intl. Conf. of Management of Data, pp. 361-370 (2001 )
33. Zobel, J., Moffat, A., Wilkinson, R., Sacks-Davis, R.: Efficient retrieval of partial documents. Inf. Process. Manag. 31(3), 361-377 (1995)