Theory and applications of b-bit minwise hashing

Communications of the ACM

Volumn 54, Issue 8, 2011, Pages 101-109

  Li, Ping ^a   König, Arnd Christian ^b  

^a Department of Obstetrics Gynecology   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL OVERHEADS; LARGE DATASETS; REAL LIFE DATASETS; SET SIMILARITY; SIMPLE ALGORITHM; STORAGE REQUIREMENTS;

INFORMATION MANAGEMENT; INFORMATION RETRIEVAL;

ALGORITHMS;

EID: 79961058834     PISSN: 00010782     EISSN: 15577317     Source Type: Journal    
DOI: 10.1145/1978542.1978566     Document Type: Article

References (25)

1. Andoni, A., Indyk, P. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Commun. ACM 51 (2008), 117-122.
2. Broder, A.Z. On the resemblance and containment of documents. In The Compression and Complexity of Sequences (Positano, Italy, 1997), 21-29.
3. Broder, A.Z., Charikar, M., Frieze, A.M., Mitzenmacher, M. Min-wise independent permutations. J. Comput. Syst. Sci. 60, 3 (2000), 630-659.
4. Broder, A.Z., Glassman, S.C., Manasse, M.S., Zweig, G. Syntactic clustering of the web. In WWW (Santa Clara, CA , 1997), 1157-1166.
5. Charikar, M.S. Similarity estimation techniques from rounding algorithms. In STOC (Montreal, Quebec, Canada, 2002), 380-388. (Pubitemid 35009398)
6. Cohen, E., Datar, M., Fujiwara, S., Gionis, A., Indyk, P., Motwani, R., Ullman, J.D., Yang, C. Finding interesting associations without support pruning. IEEE Trans. Knowl. Data Eng. 13, 1 (2001), 64-78. (Pubitemid 32301044)
7. Fetterly, D., Manasse, M., Najork, M., Wiener, J.L. A large-scale study of the evolution of web pages. In WWW (Budapest, Hungary, 2003), 669-678.
8. Forman, G., Eshghi, K., Suermondt, J. Efficient detection of large-scale redundancy in enterprise file systems. SIGOPS Oper. Syst. Rev. 43, 1 (2009), 84-91.
9. Gamon, M., Basu, S., Belenko, D., Fisher, D., Hurst, M., König, A.C. Blews: Using blogs to provide context for news articles. In AAAI Conference on Weblogs and Social Media (Redmond, WA, 2008).
10. Gionis, A., Gunopulos, D., Koudas, N. Efficient and tunable similar set retrieval. In SIGMOD (Santa Barbara, CA , 2001), 247-258.
11. Goemans, M.X., Williamson, D.P. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM 42, 6 (1995), 1115-1145.
12. Indyk, P. A small approximately min-wise independent family of hash functions. J. Algorithms 38, 1 (2001), 84-90. (Pubitemid 33667201)
13. Indyk, P., Motwani, R. Approximate nearest neighbors: Towards removing the curse of dimensionality. In STOC (Dallas, TX, 1998), 604-613.
14. Itoh, T., Takei, Y., Tarui, J. On the sample size of k-restricted min-wise independent permutations and other k-wise distributions. In STOC (San Diego, CA , 2003), 710-718.
15. Kushilevitz, E., Ostrovsky, R., Rabani, Y. Efficient search for approximate nearest neighbor in high dimensional spaces. In STOC (Dallas, TX, 1998), 614-623.
16. Li, P., Church, K.W. A sketch algorithm for estimating two-way and multi-way associations. Comput. Linguist. 33, 3 (2007), 305-354 (Preliminary results appeared in HLT /EM NLP 2005).
17. Li, P., Church, K.W., Hastie, T.J. One sketch for all: Theory and applications of conditional random sampling. In NIPS (Vancouver, British Columbia, Canada, 2008) (Preliminary results appeared in NIPS 2006).
18. Li, P., Hastie, T.J., Church, K.W. Improving random projections using marginal information. In COLT (Pittsburgh, PA, 2006), 635-649. (Pubitemid 44072224)
19. Li, P., König, A.C., Gui, W. b-Bit minwise hashing for estimating three-way similarities. In NIPS (Vancouver, British Columbia, Canada, 2010).
20. Li, P., Moore, J., König, A.C. b-Bit minwise hashing for large-scale linear SVM. Technical report, 2011.http://www.stat.cornell.edu/?li/b- bithashing/HashingSVM.pdf
21. Cherkasova, L., Eshghi, K., Morrey III, C.B., Tucek, J., Veitch, A. Applying Syntactic similarity algorithms for enterprise information management. In KDD (Paris, France, 2009), 1087-1096.
22. Manasse, M., McSherry, F., Talwar, K. Consistent weighted sampling. Technical Report MSR -TR -2010-73, Microsoft Research, 2010.
23. Pandey, S., Broder, A., Chierichetti, F., Josifovski, V., Kumar, R., Vassilvitskii, S. Nearest-neighbor caching for contentmatch applications. In WWW (Madrid, Spain, 2009), 441-450.
24. Rajaraman, A., Ullman, J. Mining of Massive Datasets. http://i.stanford.edu/ullman/mmds.html
25. Urvoy, T., Chauveau, E., Filoche, P., Lavergne, T. Tracking web spam with html style similarities. ACM Trans. Web 2, 1 (2008), 1-28.