A fast approach for parallel deduplication on multicore processors

Proceedings of the ACM Symposium on Applied Computing

Volumn , Issue , 2011, Pages 1027-1032

  Dal Bianco, Guilherme ^a   Galante, Renata ^a   Heuser, Carlos A ^a  

^a FEDERAL UNIVERSITY OF RIO GRANDE DO SUL   (Brazil)

Author keywords

data integration; deduplication; parallel systems

Indexed keywords

BLOCKING METHOD; DATA INTEGRATION; DEDUPLICATION; DISTRIBUTED ENVIRONMENTS; EMPIRICAL EVALUATIONS; LOW DEGREE; MULTI-CORE PROCESSOR; PARALLEL DATA; PARALLEL SYSTEM; PROGRAMMING MODELS; ROBUST DATUM; SUB-BLOCKS; UNBALANCED LOADS;

MULTICORE PROGRAMMING; PARALLEL PROGRAMMING;

PARALLEL PROCESSING SYSTEMS;

EID: 79959294918     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1982185.1982411     Document Type: Conference Paper

References (15)

1. A. Aizawa and K. Oyama. A fast linkage detection scheme for multi-source information integration. In Proceedings of the International Workshop on Challenges in Web Information Retrieval and Integration, pages 30-39, Washington, DC, USA, 2005.
2. R. Baxter, P. Christen, and C. F. Epidemiology. A comparison of fast blocking methods for record linkage. In Proceedings of Workshop Data Cleaning, Record Linkage, and Object Consolidation, 2003.
3. O. Benjelloun, H. Garcia-Molina, H. Gong, H. Kawai, T. E. Larson, D. Menestrina, and S. Thavisomboon. D-swoosh: A family of algorithms for generic, distributed entity resolution. Distributed Computing Systems, International Conference on, 0:37, 2007.
4. P. Christen. Probabilistic data generation for deduplication and data linkage. In Intelligent Data Engineering and Automated Learning - IDEAL 2005, pages 109-116. Springer Berlin / Heidelberg, 2005.
5. P. Christen, T. Churches, and M. Hegland. Febrl - a parallel open source data linkage system. In PAKDD, pages 638-647, 2004.
6. T. de Vries, H. Ke, S. Chawla, and P. Christen. Robust record linkage blocking using suffix arrays. In CIKM '09: Proceeding of the 18th ACM conference on Information and knowledge management, pages 305-314, New York, NY, USA, 2009. ACM.
7. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. In Proceedings of the 6th conference on Symposium on Operating Systems Design and Implementation, Berkeley, CA, USA, 2004.
8. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. Commun. ACM, 51(1):107-113, 2008.
9. A. K. Elmagarmid, P. G. Ipeirotis, Vassilios, and S. Verykios. Duplicate record detection: A survey. Transactions on Knowledge and Data Engineering.
10. H. Kim and D. Lee. Parallel linkage. In CIKM '07: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, 2007.
11. A. McCallum, K. Nigam, and L. H. Ungar. Efficient clustering of high-dimensional data sets with application to reference matching. In KDD '00: ACM SIGKDD. ACM, 2000.
12. C. Peter. Performance and scalability of fast blocking techniques for deduplication and data linkage. Proc. VLDB Endow., 1(2):1253-1264, 2007.
13. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis. Evaluating mapreduce for multi-core and multiprocessor systems. In HPCA '07, pages 13-24, 2007.
14. W. Santos, T. Teixeira, C. Machado, W. M. Jr., R. Ferreira, D. Guedes, and A. S. D. Silva. A scalable parallel deduplication algorithm. Computer Architecture and High Performance Computing, Symposium on, 0:79-86, 2007.
15. R. Vernica, M. J. Carey, and C. Li. Efficient parallel set-similarity joins using mapreduce. In SIGMOD '10: Proceedings of the 2010 international conference on Management of data, 2010.