MR-DSJ: Distance-based self-join for large-scale vector data analysis with MapReduce

Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI)

Volumn P-214, Issue , 2013, Pages 37-56

  Seidl, Thomas ^a   Fries, Sergej ^a   Boden, Brigitte ^a  

^a RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; DATA HANDLING; DATA MINING; FILTRATION; INFORMATION ANALYSIS; ITERATIVE METHODS;

COMPUTER CLUSTERS; DATA MINING TASKS; DISTRIBUTED PROGRAMMING MODEL; EXPERIMENTAL EVALUATION; FILTER TECHNIQUES; GRID PARTITIONING; LARGE-SCALE VECTORS; NEAR-DUPLICATE DETECTION;

CLUSTERING ALGORITHMS;

EID: 84901806243     PISSN: 16175468     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. [ABE+10] Alexander Alexandrov, Dominic Battré, Stephan Ewen, Max Heimel, Fabian Hueske, Odej Kao, Volker Markl, Erik Nijkamp, and Daniel Warneke. Massively Parallel Data Analysis with PACTs on Nephele. PVLDB, 3:1625-1628, 2010.
2. [ABPA+09] Azza Abouzeid, Kamil Bajda-Pawlikowski, Daniel J. Abadi, Alexander Rasin, and Avi Silberschatz. HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads. PVLDB, 2(1):922-933, 2009.
3. [ASM+12] F.N. Afrati, A.D. Sarma, D. Menestrina, A. Parameswaran, and J.D. Ullman. Fuzzy Joins Using MapReduce. ICDE, 2012.
4. [AU10] Foto N. Afrati and JeffreyD. Ullman. Optimizing Joins inaMap-Reduce Environment. In EDBT, pages 99-110, 2010.
5. [BBBK00] Christian Böhm, Bernhard Braunmüller, Markus M. Breunig, and Hans-Peter Kriegel. High Performance Clustering Based on the Similarity Join. In CIKM, pages 298-305, 2000.
6. [BBKK01] C. Böhm, B. Braunmüller, F. Krebs, and H.P. Kriegel. Epsilon grid order: An algorithm for the similarity join on massive high-dimensional data. In SIGMOD, pages 379-388, 2001.
7. [BKS93] Thomas Brinkhoff, Hans-Peter Kriegel, and Bernhard Seeger. Efficient Processing of Spatial Joins Using R-Trees. In SIGMOD, pages 237-246, 1993.
8. [BML10] Ranieri Baraglia, Gianmarco De Francisci Morales, and Claudio Lucchese. Document Similarity Self-Join with MapReduce. In ICDM, pages 731-736, 2010.
9. [BPE+10] Spyros Blanas, Jignesh M. Patel, Vuk Ercegovac, Jun Rao, Eugene J. Shekita, and Yuanyuan Tian. A comparison of join algorithms for log processing in MapReduce. In SIGMOD, pages 975-986, 2010.
10. [CGK06] Surajit Chaudhuri, Venkatesh Ganti, and Raghav Kaushik. A Primitive Operator for Similarity Joins in Data Cleaning. In ICDE, page 5, 2006.
11. [Dat06] Chris J. Date. The relational database dictionary -A comprehensive glossary of relational terms and concepts, with illustrative examples. O'Reilly, 2006.
12. [DG04] Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI, pages 137-150, 2004.
13. [DNSS92] David J. DeWitt, Jeffrey F. Naughton, Donovan A. Schneider, and S. Seshadri. Practical Skew Handling in Parallel Joins. In VLDB, pages 27-40, 1992.
14. [DQRJ+10] Jens Dittrich, Jorge-Arnulfo Quiané-Ruiz, Alekh Jindal, Yagiz Kargin, Vinay Setty, and Jörg Schad. Hadoop++: Making a Yellow Elephant Run Like a Cheetah. PVLDB, 3:518-529, 2010.
15. [DS00] Jens-Peter Dittrich and Bernhard Seeger. Data Redundancy and Duplicate Detection in Spatial Join Processing. In ICDE, pages 535-546, 2000.
16. [EKSX96] Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In SIGKDD, pages 226-231, 1996.
17. [HW99] C.J. Hahn and S.G. Warren. Extended edited synoptic cloud reports from ships and land stations over the globe, 1952-1996. NDP026C, Carbon Dioxide Information Analysis Center, 1999.
18. [LSCO12] Wei Lu, Yanyan Shen, Su Chen, and Beng Chin Ooi. Efficient Processing of k Nearest Neighbor Joins using MapReduce. PVLDB, 5(10):1016-1027, 2012.
19. [MF12] Ahmed Metwally and Christos Faloutsos. V-SMART-Join: A Scalable MapReduce Framework for All-Pair Similarity Joins of Multisets and Vectors. PVLDB, 5(8):704-715, 2012.
20. [Mon00] Alvaro E. Monge. Matching Algorithms within a Duplicate Detection System. IEEE Data Eng. Bull., 23(4):14-20, 2000.
21. [OR11] Alper Okcan and Mirek Riedewald. Processing theta-joins using MapReduce. In SIGMOD, pages 949-960, 2011.
22. [PD96] Jignesh M. Patel and David J. DeWitt. Partition Based Spatial-Merge Join. In SIGMOD Conference, pages 259-270, 1996.
23. [PPR+09] Andrew Pavlo, Erik Paulson, Alexander Rasin, Daniel J. Abadi, David J. DeWitt, Samuel Madden, and Michael Stonebraker. A comparison of approaches to large-scale data analysis. In SIGMOD, pages 165-178, 2009.
24. [RKV95] Nick Roussopoulos, Stephen Kelley, and Frédéic Vincent. Nearest Neighbor Queries. In SIGMOD, pages 71-79, 1995.
25. [RRS00] Sridhar Ramaswamy, Rajeev Rastogi, and Kyuseok Shim. Efficient Algorithms for Mining Outliers from Large Data Sets. In SIGMOD, pages 427-438, 2000.
26. [RU11] A. Rajaraman and J.D. Ullman. Mining of massive datasets. Cambridge Univ Pr, 2011.
27. [SRT12] Yasin N. Silva, Jason M. Reed, and Lisa M. Tsosie. MapReduce-based similarity join for metric spaces. In Proceedings of the 1st International Workshop on Cloud Intelligence, Cloud-I '12, pages 3:1-3:8, New York, NY, USA, 2012. ACM.
28. [VCL10] R. Vernica, M.J. Carey, and C. Li. Efficient parallel set-similarity joins using MapReduce. In SIGMOD, pages 495-506, 2010.
29. [XWLY08] Chuan Xiao, Wei Wang, Xuemin Lin, and Jeffrey Xu Yu. Efficient similarity joins for near duplicate detection. In WWW, pages 131-140, 2008.
30. [ZHL+09] Shubin Zhang, Jizhong Han, Zhiyong Liu, Kai Wang, and Zhiyong Xu. SJMR: Parallelizing spatial join with MapReduce on clusters. In CLUSTER, pages 1-8, 2009.
31. [ZJ03] Yanchang Zhao and Song Junde. AGRID: An Efficient Algorithm for Clustering Large High-Dimensional Datasets. In PAKDD, pages 271-282, 2003.
32. [ZLJ12] Chi Zhang, Feifei Li, and Jeffrey Jestes. Efficient parallel kNN joins for large data in MapReduce. In EDBT, pages 38-49, 2012.