Fast and compact web graph representations

ACM Transactions on the Web

Volumn 4, Issue 4, 2010, Pages

  Claude, Francisco ^a   Navarro, Gonzalo ^b  

^a UNIVERSITY OF WATERLOO   (Canada)

^b UNIVERSITY OF CHILE   (Chile)

Author keywords

Compression; Data structures; Web graph

Indexed keywords

ACCESS TIME; COMPRESSION; COMPRESSION RATIOS; GRAMMAR-BASED COMPRESSION; GRAPH REPRESENTATION; MAIN MEMORY; RESEARCH TOPICS; SECONDARY MEMORIES; STATE OF THE ART; WEB GRAPHS;

DATA STRUCTURES;

DATA COMPRESSION;

EID: 77958035555     PISSN: 15591131     EISSN: 1559114X     Source Type: Journal    
DOI: 10.1145/1841909.1841913     Document Type: Article

References (52)

1. ADLER, M. AND MITZENMACHER, M. 2001. Towards compressing Web graphs. In Proceedings of the 11th Data Compression Conference (DCC). 203-212.
2. AIELLO, W., CHUNG, F., AND LU, L. 2000. A random graph model for massive graphs. In Proceedings of the 32th ACM Symposium on Theory of Computing (STOC). 171-180.
3. ASANO, Y.,MIYAWAKI, Y., AND NISHIZEKI, T. 2008. Efficient compression of Web graphs. In Proceedings of the 14th Conference on Computing and Combinatorics (COCOON). LNCS 5092. 1-11.
4. BADUE, C., BAEZA-YATES, R., RIBEIRO-NETO, B., AND ZIVIANI, N. 2001. Distributed query processing using partitioned inverted files. In Proceedings of the 8th International Symposium on String Processing and Information Retrieval (SPIRE). 10-20.
5. BHARAT, K., BRODER, A., HENZINGER, M., KUMAR, P., AND VENKATASUBRAMANIAN, S. 1998. The Connectivity Server: Fast access to linkage information on the Web. In Proceedings of the 7th World Wide Web Conference (WWW). 469-477.
6. BLANDFORD, D. 2006. Compact data structures with fast queries. Ph.D. thesis, Tech. rep. School of Computer Science, Carnegie Mellon University. TR CMU-CS-05-196.
7. BLANDFORD, D., BLELLOCH, G., AND KASH, I. 2003. Compact representations of separable graphs. In Proceedings of the 14th Symposium on Discrete Algorithms (SODA). 579-588.
8. BOLDI, P., SANTINI, M., AND VIGNA, S. 2008. A large time-aware Web graph. SIGIR Forum 42, 2, 33-38.
9. BOLDI, P., SANTINI, M., AND VIGNA, S. 2009. Permuting Web graphs. In Proceedings of the 6th Workshop on Algorithms and Models for the Web Graph (WAW). 116-126.
10. BOLDI, P. AND VIGNA, S. 2004a. The WebGraph framework I: Compression techniques. In Proceedings of the 13th World Wide Web Conference (WWW). 595-602.
11. BOLDI, P. AND VIGNA, S. 2004b. The WebGraph framework II: Codes for the world-wide Web. In Proceedings of the 14th Data Compression Conference (DCC). 528.
12. BRISABOA, N., LADRA, S., AND NAVARRO, G. 2009. K2-trees for compact Web graph representation. In Proceedings of the 16th International Symposium on String Processing and Information Retrieval (SPIRE). Lecture Notes in Computer Science, vol.5721. Springer, 18-30.
13. BRODER, A., KUMAR, R.,MAGHOUL, F., RAGHAVAN, P., RAJAGOPALAN, S., STATA, R., TOMKINS, A., AND WIENER, J. 2000. Graph structure in the Web. J. Comput. Netw. 33, 1-6, 309-320.
14. BUEHRER, G. AND CHELLAPILLA, K. 2008. A scalable pattern mining approach to Web graph compression with communities. In Proceedings of the International Conference on Web Search and Web Data (WSDM). 95-106.
15. CHAKRABARTI, D., PAPADIMITRIOU, S., MODHA, D., AND FALOUTSOS, C. 2004. Fully automatic cross-associations. In Proceedings of the ACM Special Interest Group on Knowledge Discovery and Data Mining (SIGKDD).
16. CHARIKAR, M., LEHMAN, E., LIU, D., PANIGRAHY, R., PRABHAKARAN, M., SAHAI, A., AND SHELAT, A. 2005. The smallest grammar problem. IEEE Trans. Inform. Theory 51, 7, 2554-2576.
17. CHUANG, R., GARG, A., HE, X., KAO, M.-Y., AND LU, H.-I. 1998. Compact encodings of planar graphs with canonical orderings and multiple parentheses. In Proceedings of the International Symposium on Automata, Languages, and Programming, Lecture Notes in Computer Science, vol.1443, 118-129.
18. CLARK, D. 1996. Compact pat trees. Ph.D. thesis, University of Waterloo.
19. CLAUDE, F. AND NAVARRO, G. 2007. A fast and compactWeb graph representation. In Proceedings of the 14th International Symposium on String Processing and Information Retrieval (SPIRE). Lecture Notes in Computer Science, vol.4726. 105-116.
20. CLAUDE, F. AND NAVARRO, G. 2008. Practical rank/select queries over arbitrary sequences. In Proceedings of the 15th International Symposium on String Processing and Information Retrieval (SPIRE). Lecture Notes in Computer Science, vol.5280. 176-187.
21. CLAUDE, F. AND NAVARRO, G. 2010. Extended compact Web graph representations. In Algorithms and Applications, T. Elomaa, H. Mannila, and P. Orponen, Eds. Lecture Notes in Computer Science, vol.6060. Springer, 77-91.
22. CORMEN, T. H., LEISERSON, C. E., RIVEST, R. L., AND STEIN, C. 2001. Introduction to Algorithms, 2nd Ed. MIT Press and McGraw-Hill.
23. DEO, N. AND LITOW, B. 1998. A structural approach to graph compression. In Proceedings of the 23th MFCS Workshop on Communications. 91-101.
24. DONATO, D., LAURA, L., LEONARDI, S., MEYER, U., MILLOZZI, S., AND SIBEYN, J. 2006. Algorithms and experiments for the WebGraph. J. Graph Algor. Appl. 10, 2, 219-236.
25. FERRAGINA, P.,MANZINI, G.,M̈AKINEN, V., AND NAVARRO, G. 2007. Compressed representations of sequences and full-text indexes. ACM Trans. Algor. 3, 2, Article 20.
26. FINK, A. AND VOSS, S. 1999. Aications of modern heuristic search methods to pattern sequencing problems. Comput. Operat. Resear. 26, 17-34.
27. GOLYNSKI, A.,MUNRO, I., AND RAO, S. 2006. Rank/select operations on large alphabets: A tool for text indexing. In Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 368-373.
28. GONZÁLEZ, R. AND NAVARRO, G. 2007. Compressed text indexes with fast locate. In Proceedings of the 18th Symposium on Combinatorial Pattern Matching (CPM). Lecture Notes in Computer Science, vol.4580. 216-227.
29. GROSSI, R., GUPTA, A., AND VITTER, J. 2003. High-order entropy-compressed text indexes. In Proceedings of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 841-850.
30. HE, X., KAO, M.-Y., AND LU, H.-I. 2000. A fast general methodology for information-theoretically optimal encodings of graphs. SIAM J. Comput. 30, 838-846.
31. HU, T. AND TUCKER, A. 1971. Optimal computer-search trees and variable-length alphabetic codes. SIAM J. Appl. Math. 21, 514-532.
32. JACOBSON, G. 1989. Succinct static data structures. Ph.D. thesis, Carnegie Mellon University.
33. KLEINBERG, J., KUMAR, R., RAGHAVAN, P., RAJAGOPALAN, S., AND TOMKINS, A. 1999. The Web as a graph: Measurements, models, and methods. In Proceedings of the 5th Annual InternationalConference on Computing and Combinatorics (COCOON). Lecture Notes in Computer Science, vol.1627. 1-17.
34. KLEINBERG, J. M. 1999. Authoritative sources in a hyperlinked environment. J. ACM 46, 5, 604-632.
35. KUMAR, R., RAGHAVAN, P., RAJAGOPALAN, S., AND TOMKINS, A. 1999. Extracting large scale knowledge bases from the Web. In Proceedings of the 25th Conference on Very Large Data Bases (VLDB). 639-650.
36. LARSSON, J. AND MOFFAT, A. 2000. Off-line dictionary-based compression. Proc. IEEE 88, 11, 1722-1732.
37. MUNRO, I. 1996. Tables. In Proceedings of the 16th Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS). Lecture Notes in Computer Science, vol.1180. 37-42.
38. MUNRO, I. AND RAMAN, V. 1997. Succinct representation of balanced parentheses, static trees, and planar graphs. In Proceedings of the 38th Symposium on Foundations of Computer Science (FOCS). 118-126.
39. NAVARRO, G. AND M̈AKINEN, V. 2007. Compressed full-text indexes. ACM Comput. Surv. 39, 1, Article 2.
40. RAGHAVAN, S. AND GARCIA-MOLINA, H. 2003. Representing Web graphs. In Proceedings of the 19th International Conference on Data Engineering (ICDE). 405.
41. RAMAN, R., RAMAN, V., AND RAO, S. S. 2002. Succinct indexable dictionaries with applications to encoding k-ary trees and multisets. In Proceedings of the ACM-SIAM 13th Symposium on Discrete Algorithms (SODA). 233-242.
42. RANDALL, K., STATA, R., WICKREMESINGHE, R., AND WIENER, J. 2001. The LINK database: Fast access to graphs of the Web. Tech. rep. 175, Compaq Systems Research Center, Palo Alto, CA.
43. RYTTER, W. 2003. Application of Lempel-Ziv factorization to the approximation of grammar-based compression. Theor. Comput. Sci. 302, 1-3, 211-222.
44. SAITO, H., TOYODA, M., KITSUREGAWA, M., AND AIHARA, K. 2007. A large-scale study of link spam detection by graph algorithms. In Proceedings of the 3rd International Workshop on Adversarial Information Retrieval on the Web (AIRWeb). ACM Press.
45. SAKAMOTO, H. 2005. A fully linear-time approximation algorithm for grammar-based compression. J. Disc. Algor. 3, 2-4, 416-430.
46. SHIEH, W., CHEN, T., SHANN, J., AND CHUNG, C. 2003. Inverted file compression through document identifier reassignment. Inform. Process. Manag. 39, 1, 117-131.
47. SUEL, T. AND YUAN, J. 2001. Compressing the graph structure of the Web. In Proceedings of the 11th Data Compression Conference (DCC). 213-222.
48. TOMASIC, A. AND GARCIA-MOLINA, H. 1993. Performance of inverted indices in shared-nothing distributed text document information retrieval systems. In Proceedings of the 2nd International Conference on Parallel and Distributed Information Systems (PDIS). 8-17.
49. VITTER, J. S. 2006. Algorithms and data structures for external memory. Found. Trends Theoret. Comput. Sci. 2, 4, 305-474.
50. WAN, R. 2003. Browsing and searching compressed documents. Ph.D. thesis, Department of Computer Science and Software Engineering, University of Melbourne.
51. ZIV, J. AND LEMPEL, A. 1977. A universal algorithm for sequential data compression. IEEE Trans. Inform. Theory 23, 337-343.
52. ZIV, J. AND LEMPEL, A. 1978. Compression of individual sequences via variable-rate coding. IEEE Trans. Inform. Theory 24, 5, 530-536.