Similarity search and locality sensitive hashing using ternary content addressable memories

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2010, Pages 375-386

  Shinde, Rajendra ^a   Goel, Ashish ^a   Gupta, Pankaj ^b   Dutta, Debojyoti ^c  

^a STANFORD UNIVERSITY   (United States)

^b TWITTER INC   (United States)

^c CISCO SYSTEMS   (United States)

Author keywords

locality sensitive hashing; nearest neighbor search; similarity search; tcam

Indexed keywords

ACCESS CONTROL LISTS; ADDRESS LOOKUP; APPROXIMATE NEAREST NEIGHBOR; ASSOCIATIVE MEMORIES; BIT VECTOR; BRUTE FORCE SEARCH; CELL PROBES; COMPUTATIONAL BIOLOGY; CURSE OF DIMENSIONALITY; DATA BASE; DATA SETS; DATABASE ENTRY; ETHERNET SWITCHES; EUCLIDEAN SPACES; EXTENSIVE SIMULATIONS; FAST SOLUTIONS; HIGH DIMENSIONAL SPACES; HIGH-SPEED DATA; LARGE-SCALE STREAMING; LINEAR SPACES; LOCALITY SENSITIVE HASHING; LOOKUPS; LOWER BOUNDS; MACHINE LEARNING APPLICATIONS; NEAREST NEIGHBOR SEARCH; NEAREST NEIGHBORS; NETWORKING APPLICATIONS; QUERY TIME; SIMILARITY SEARCH; SUBLINEAR; TERNARY CONTENT ADDRESSABLE MEMORIES; TERNARY VECTORS; WIKIPEDIA; WILD CARDS;

ACCESS CONTROL; ASSOCIATIVE PROCESSING; ASSOCIATIVE STORAGE; BIOINFORMATICS; DATA STRUCTURES; DATABASE SYSTEMS; GEOMETRY; IMAGE RETRIEVAL; LOGIC GATES; TABLE LOOKUP; TELECOMMUNICATION INDUSTRY; VECTOR SPACES; WORLD WIDE WEB;

DATA MINING;

EID: 77954752889     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1807167.1807209     Document Type: Conference Paper

References (43)

1. M. Abramowitz and I. Stegun. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, Chapter 7. Dover, 1972.
2. C. Aggarwal, J. Han, J. Wang, and P. Yu. On demand classification of streams. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004.
3. N. Ailon and B. Chazelle. Approximate nearest neighbors and the fast johnson-lindenstrauss transform. In Proceedings of the 38th Annual ACM Symposium on Theory of Computing, 2006.
4. A. Andoni and P. Indyk. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Communications of the ACM, 51(1):117-122, 2006.
5. A. Andoni and P. Indyk. Near optimal hashing algorithms for approximate nearest neighbor in high dimensions. In Proceedings of 47th Annual IEEE Symposium on Foundations of Computer Science, 2006.
6. A. Andoni, P. Indyk, and M. Patrascu. On the optimality of dimension reduction method. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, 2006.
7. N. Bandi, A. Metwally, D. Agrawal, and A. E. Abbadi. Tcam-conscious algorithms for data streams. In Proceedings of the 23rd International Conference on Data Engineering, 2007.
8. J. Bentley. Multidimensional binary search trees used for associative searching. Communications of the ACM, 18.
9. P. Berkhin. A Survey of Clustering Data Mining Techniques. Springer, 2002.
10. A. Beygelzimer, S. Kakade, and J. Langford. Cover trees for nearest neighbours. In Proceedings of the Twenty-Third International Conference on Machine Learning, 2006.
11. J. Buhler. Efficient large scale sequence comparison by locality-sensitive hashing. Bioinformatics, 17.
12. M. Charikar. Similarity estimation techniques from rounding algorithms. In Proceedings of 34th Annual ACM Symposium on Theory of Computing, 2002.
13. CISCO. Cisco catalyst 4500 series http://www.cisco.com/.
14. M. Covell and S. Baluja. Lsh banding for large-scale retrieval with memory and recall constraints. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, 2009.
15. T. Cover and P. Hart. Nearest neighbour pattern classification. IEEE Transactions on Information Theory, 13.
16. M. Datar, N. Immorlica, P. Indyk, and V. Mirrokni. Locality sensitive hashing scheme based on p-stable distributions. In Proceedings of the 20th Annual ACM Symposium on Computational Geometry, 2004.
17. D. Dutta and T. Chen. Speeding up tandem mass spectrometry database search: metric embeddings and fast near neighbor search. Bioinformatics.
18. A. Gionis, P. Indyk, and R. Motwani. Similarity search in high dimensions via hashing. In Proceedings of the 25th VLDB conference, 1999.
19. S. Har-Peled. A replacement for voronoi diagrams of near linear size. In Proceedings of the 42nd Annual IEEE Symposium on the Foundations of Computer Science, 2001.
20. T. H. Haveliwala, A. Gionis, and P. Indyk. Scalable techniques for clustering the web. In Proceedings of the Third International Workshop on the Web and Databases, 2000.
21. P. Indyk, N. Koudas, and S. Muthukrishnan. Identifying representative trends in massive time series data sets using sketches. In Proceedings of 26th International Conference on Very Large Data Bases, 2000.
22. P. Indyk and R. Motwani. Approximate nearest neighbors: Towards removing the curse of dimensionality. In Proceedings of the 30th Annual ACM Symposium on Theory of Computing, 1998.
23. R. Krauthgamer and J. Lee. Navigating nets:simple algorithms for proximity search. In Proceedings of the fifteenth annual ACM-SIAM Symposium of Discrete Algorithms, 2004.
24. B. Kulis and K. Grauman. Kernelized locality-sensitive hashing for scalable image search. In Proceedings of the Twelth International Conference on Computer Vision, 2009.
25. K. Lakshminarayanan, A. Rangarajan, and S. Venkatachary. Algorithms for advanced packet classification using ternary cams. In Proceedings of the ACM SIGCOMM 2005 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, 2005.
26. Q. Lv, W. Josephson, Z. Wang, M. Charikar, and K. Li. Multi-probe lsh: Efficient indexing for high-dimensional similarity search. In Proceedings of the Thirty-Third International Conference on Very Large Data Bases, 2007.
27. G. Manku, A. Jain, and A. Sharma. Detecting near duplicates for web crawling. In Proceedings of the 16th International World Wide Web Conference, 2007.
28. N. Microsystems. Netlogic microsystems: http://www.netlogicmicro.com/.
29. M. Minsky and S. Papert. Perceptrons. In MIT Press, 1969.
30. R. Motwani, A. Naor, and R. Panigrahi. Lower bounds on locality sensitive hashing. In Proceedings of the 22nd Annual ACM Symposium on Computational Geometry, 2006.
31. K. Pagiamtzis and A. Sheikholeslami. Content-addressable memory (cam) circuits and architectures: A tutorial and survey. IEEE Journal of Solid-State Circuits, 41(3):712-727, 2006.
32. R. Panigrahi. Entropy based nearest neighbor search in high dimensions. In Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms, 2006.
33. R. Panigrahy, K. Talwar, and U. Wieder. A geometric approach to lower bounds for approximate near-neighbor search and partial match. In Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science, 2008.
34. D. Ravichandran, P. Pantel, and E. Hovy. Using locality sensitive hash functions for high speed noun clustering. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, 2005.
35. C. Rijsbergen. Information Retrieval. Butterworth, 1979.
36. R. Salakhutdinov and G. Hinton. Semantic hashing. Int. J. Approx. Reasoning, 50(7):969-978, 2009.
37. D. Shah and P. Gupta. Fast updates on ternary-cams for packet lookups and classification. In Proceedings of Hot Interconnects VIII, 2000.
38. G. Tzanetakis and P. Cook. MARSYAAS A Framework for audio analysis. Cambridge University Press.
39. R. Weber, H. Schek, and S. Blott. A quantititative analysis and performance study for similarity search methods in high dimensional spaces. In Proceedings of the Twenty-Fourth International Conference on Very Large Data Bases, 1998.
40. Y. Weiss, A. Torralba, and R. Fergus. Small codes and large image databases for recognition. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2008.
41. Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems, 2008.
42. A. Yao. Should tables be sorted. J. Assoc. Comput. Mach, 28.
43. Y. Tao, K. Yi, C. Sheng, and P. Kalnis. Quality and Efficiency in High Dimensional Nearest Neighbor Search. In SIGMOD, 2009.