Multi-pass sorted neighborhood blocking with MapReduce

Computer Science - Research and Development

Volumn 27, Issue 1, 2012, Pages 45-63

  Kolb, Lars ^a   Thor, Andreas ^a   Rahm, Erhard ^a  

^a UNIVERSITY OF LEIPZIG   (Germany)

Author keywords

Blocking; Cloud computing; Entity resolution; Hadoop; MapReduce; Sorted Neighborhood

Indexed keywords

EID: 84857059718     PISSN: 18652034     EISSN: 18652042     Source Type: Journal    
DOI: 10.1007/s00450-011-0177-x     Document Type: Conference Paper

References (23)

1. Armbrust M, Fox A, Griffith R, Joseph AD, Katz RH, Konwinski A, Lee G, Patterson DA, Rabkin A, Stoica I, Zaharia M (2009) Above the clouds: A berkeley view of cloud computing. Tech rep, EECS Department. University of California, Berkeley
2. Batini C, Scannapieco M (2006) Data quality: concepts, methodologies and techniques. Data-centric systems and applications. Springer, Berlin
3. Baxter R, Christen P, Churches T (2003) A comparison of fast blocking methods for record linkage. In: ACM SIGKDD, vol 3, pp 25-27
4. Bilenko M, Mooney RJ (2003) Adaptive duplicate detection using learnable string similarity measures. In: KDD, pp 39-48
5. Borthakur D (2007) The hadoop distributed file system: Architecture and design. Hadoop Project Website
6. Christen P (2008) Febrl -: an open source data cleaning, deduplication and record linkage system with a graphical user interface. In: KDD, pp 1065-1068
7. Christen P, Churches T, Hegland M (2004) Febrl-a parallel open source data linkage system. In: PAKDD, pp 638-647 (Pubitemid 38824950)
8. Dean J, Ghemawat S (2004) MapReduce: Simplified data processing on large clusters. In: OSDI, pp 137-150
9. J Dean S Ghemawat 2008 MapReduce: simplified data processing on large clusters Commun ACM 51 1 107 113 10.1145/1327452.1327492
10. D DeWitt J Gray 1992 Parallel database systems: the future of high performance database systems Commun ACM 35 6 85 98 10.1145/129888.129894 (Pubitemid 23642225)
11. DeWitt DJ, Naughton JF, Schneider DA, Seshadri S (1992) Practical skew handling in parallel joins. In: VLDB, pp 27-40
12. AK Elmagarmid PG Ipeirotis VS Verykios 2007 Duplicate record detection: a survey IEEE Trans Knowl Data Eng 19 1 1 16 10.1109/TKDE.2007.250581 (Pubitemid 44955773)
13. Foundation AS (2006) Hadoop. http://hadoop.apache.org/mapreduce/
14. Hernández MA, Stolfo SJ (1995) The merge/purge problem for large databases. In: SIGMOD Conference, pp 127-138
15. Kim HS, Lee D (2007) Parallel linkage. In: CIKM, pp 283-292
16. Kirsten T, Kolb L, Hartung M, Gross A, Köpcke H, Rahm E (2010) Data partitioning for parallel entity matching. In: 8th International Workshop on Quality in Databases
17. Kolb L, Thor A, Rahm E (2011) Parallel sorted neighborhood blocking with mapreduce. In: BTW, pp 45-64
18. H Köpcke E Rahm 2010 Frameworks for entity matching: a comparison Data Knowl Eng 69 2 197 210 10.1016/j.datak.2009.10.003
19. Köpcke H, Thor A, Rahm E (2010) Evaluation of entity resolution approaches on real-world match problems. In: VLDB, pp 484-493
20. H Köpcke A Thor E Rahm 2010 Learning-based approaches for matching web data entities IEEE Internet Comput 14 23 31 10.1109/MIC.2010.58
21. J Lin C Dyer 2010 Data-intensive text processing with mapreduce Synth Lect Hum Lang Technol 3 1 1 177 10.2200/S00274ED1V01Y201006HLT007
22. E Rahm HH Do 2000 Data cleaning: problems and current approaches IEEE Data Eng Bull 23 4 3 13
23. Vernica R, Carey MJ, Li C (2010) Efficient parallel set-similarity joins using mapreduce. In: SIGMOD Conference, pp 495-506