I/O-efficient techniques for computing pagerank

International Conference on Information and Knowledge Management, Proceedings

Volumn , Issue , 2002, Pages 549-557

  Chen, Yen Yu ^a   Gan, Qingqing ^a   Suel, Torsten ^a  

^a POLYTECHNIC UNIVERSITY   (United States)

Author keywords

External memory algorithms; Link based ranking; Out of core; Pagerank; Search engines

Indexed keywords

ALGORITHMS; GRAPH THEORY; ITERATIVE METHODS; OPTIMIZATION; SEARCH ENGINES;

EXTERNAL MEMORY ALGORITHMS; PAGERANK TECHNIQUE;

WORLD WIDE WEB;

EID: 0037818472     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/584792.584882     Document Type: Conference Paper

References (43)

1. J. Abello, A. Buchsbaum, and J. Westbrook. A functional approach to external graph algorithms. In European Symposium on Algorithms, pages 332-343, 1998.
2. M. Adler and M. Mitzenmacher. Towards compressing web graphs. In Proc. of the IEEE Data Compression Conference (DCC), March 2001.
3. A. Arasu, J. Cho, H. Garcia-Molina, and S. Raghavan. Searching the web. ACM Transactions on Internet Technologies, 1(1), June 2001.
4. A. Arasu, J. Novak, Tomkins A, and J. Tomlin. Pagerank computation and the structure of the web: Experiments and algorithms. In Poster presentation at the 11th Int. World Wide Web Conference, May 2002.
5. Ask Jeeves, Inc. Teoma search engine. http://www.teoma.com.
6. B. Awerbuch and T. Leighton. A simple local-control approximation algorithm for multicommodity flow. In IEEE Symp. on Foundations of Computer Science, pages 459-468, 1993.
7. O. Axelsson. Iterative Solution Methods. Cambridge University Press, 1994.
8. R. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addision Wesley, 1999.
9. K. Bharat, A. Broder, M. Henzinger, P. Kumar, and S. Venkatasubramanian. The connectivity server: Fast access to linkage information on the web. In 7th Int. World Wide Web Conference, May 1998.
10. K. Bharat and M. Henzinger. Improved algorithms for topic distillation in a hyperlinked environment. In Proc. 21st Int. Conf. on Research and Development in Inf. Retrieval (SIGIR), August 1998.
11. S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. In Proc. of the Seventh World Wide Web Conference, 1998.
12. A. Broder, R. Kumar, F. Maghoul, P. Raghavan, S. Rajagopalan, R. Stata, A. Tomkins, and J. Wiener. Graph structure in the web: experiments and models. In 9th Int. World Wide Web Conference, 2000.
13. S. Chakrabarti, B. Dom, D. Gibson, J. Kleinberg, P. Raghavan, and S. Rajagopalan. Automatic resource list compilation by analyzing hyperlink structure and associated text. In Proc. of the 7th Int. World Wide Web Conference, May 1998.
14. S. Chakrabarti, B. Dom, and P. Indyk. Enhanced hypertext categorization using hyperlinks. In Proc. of the ACM SIGMOD Int. Conf. on Management of Data, pages 307-318, June 1998.
15. S. Chakrabarti, B. Dom, R. Kumar, P. Raghavan, S. Rajagopalan, A. Tomkins, David Gibson, and J. Kleinberg. Mining the web's link structure. IEEE Computer, 32(8):60-67, 1999.
16. S. Chakrabarti, M. van den Berg, and B. Dom. Focused crawling: A new approach to topic-specific web resource discovery. In Proc. of the 8th Int. World Wide Web Conference, May 1999.
17. Y. Chen, Q. Gan, and T. Suel. I/o-efficient techniques for computing pagerank. Technical Report TR-CIS-2002-03, Polytechnic University, CIS Department, September 2002.
18. Y. Chiang, M. Goodrich, E. Grove, R. Tamassia, D. Vengroff, and J. Vitter. External-memory graph algorithms. In Proc. of the 6th Annual ACM-SIAM Symposium on Discrete Algorithms, January 1995.
19. S. Chien, C. Dwork, S. Kumar, and D. Sivakumar. Towards exploiting link evolution. Unpublished manuscript, 2001.
20. J. Cho, H. Garcia-Molina, and L. Page. Efficient crawling through URL ordering. In 7th Int. World Wide Web Conference, May 1998.
21. J. Feigenbaum. Massive graphs: Algorithms, applications, and open problems. Invited Lecture at the 1999 SIAM Annual Meeting.
22. G. Flake, S. Lawrence, L. Giles, and F. Coetzee. Self-organization and identification of web communities. IEEE Computer, pages 66-71, 2002.
23. J. Guillaume, M. Latapy, and L. Viennot. Efficient and simple encodings for the web graph. 2001. Unpublished manuscript.
24. T.H. Haveliwala. Efficient computation of pagerank. Technical report, Stanford University, October 1999. Available at http://dbpubs.stanford.edu:8090/pub/1999-31.
25. T.H. Haveliwala. Topic-sensitive pagerank. In Proc. of the 11th Int. World Wide Web Conference, May 2002.
26. International WWW Conference Committee. Proceedings of the World Wide Web Conferences, 1994-2002. http://www.iw3c2.org/Conferences/.
27. A. Kamath, O. Palmon, and S. Plotkin. Fast approximation algorithm for minimum cost multicommodity flow. In ACM-SIAM Symp. on Discrete Algorithms, 1995.
28. J. Kleinberg. Authoritative sources in a hyperlinked environment. Proc. of the 9th ACM-SIAM Symposium on Discrete Algorithms, pages 668-677, January 1998.
29. R. Kumar, P. Raghavan, S. Rajagopalan, and A. Tomkins. Extracting large-scale knowledge bases from the web. In Proc. of the 25th Int. Conf. on Very Large Data Bases, September 1999.
30. R. Kumar, P. Raghavan, S. Rajagopalan, and A. Tomkins. Trawling the web for emerging cyber-communities. In Proc. of the 8th Int. World Wide Web Conference, May 1999.
31. R. Lempel and S. Moran. The Stochastic Approach for Link-Structure Analysis (SALSA) and the TKC Effect. In Proc. of the 9th Int. World Wide Web Conference, May 2000.
32. M. Marchiori. The quest for correct information on the web: Hyper search engines. In Proc. of the Sixth Int. World Wide Web Conference, April 1997.
33. S. Muthukrishnan and T. Suel. Second-order methods for distributed approximate single- and multicommodity flow. In 2nd Int. Workshop on Randomization and Approximation Techniques in Computer Science, pages 369-383. Springer LNCS, 1998.
34. M. Najork and J. Wiener. Breadth-first search crawling yields high-quality pages. In 10th Int. World Wide Web Conference, 2001.
35. L. Page, S. Brin, R. Motwani, and T. Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Computer Science Department, Stanford University, 1999. Available at http://dbpubs.stanford.edu:8090/pub/1999-66.
36. P. Pirolli, J. Pitkow, and R. Rao. Silk from a sow's ear: Extracting usable structures from the web. In Proc. of ACM Conf. on Human Factors in Computing Systems, April 1996.
37. R. Randall, R. Stata, R. Wickremesinghe, and J. Wiener. The link database: Fast access to graphs of the web. In Proc. of the IEEE Data Compression Conference (DCC), March 2002.
38. M. Richardson and P. Domingos. The intelligent surfer: Probabilistic combination of link and content information in pagerank. In Advances in Neural Information Processing Systems, 2002.
39. V. Shkapenyuk and T. Suel. Design and implementation of a high-performance distributed web crawler. In Proc. of the Int. Conf. on Data Engineering, February 2002.
40. T. Suel and J. Yuan. Compressing the graph structure of the web. In Proc. of the IEEE Data Compression Conference (DCC), March 2001.
41. R. Varga. Matrix Iterative Analysis. Prentice-Hall, 1962.
42. I. H. Witten, A. Moffat, and T. C. Bell. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann, second edition, 1999.
43. D. Zhang and Y. Dong. An efficient algorithm to rank web resources. In Proc. of the 9th Int. World Wide Web Conference, May 2000.