Automatic cleaning and linking of historical census data using household information

Proceedings - IEEE International Conference on Data Mining, ICDM

Volumn , Issue , 2011, Pages 413-420

  Fu, Zhichun ^a   Christen, Peter ^a   Boot, Mac ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Data cleaning; Domain knowledge; Group linking; Historical census data; Record linkage

Indexed keywords

CENSUS DATA; DATA CLEANING; DOMAIN KNOWLEDGE; GROUP LINKING; RECORD LINKAGE;

BEHAVIORAL RESEARCH; CLEANING; DATA HANDLING; DATA MINING; POPULATION STATISTICS; SURVEYS;

INFORMATION USE;

EID: 84857170753     PISSN: 15504786     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDMW.2011.35     Document Type: Conference Paper

References (24)

1. E. Higgs, "A clearer sense of the census: Victorian censuses and historical research," in Public Record Office Handbooks, no. 28. Her Majesty's Stationery Office, London, 1996.
2. D. Quass and P. Starkey, "Record linkage for genealogical databases," in Proceedings of the 2003 ACM SIGKDD Workshop on Data Cleaning, Record Linkage, and Object Consolidation, 2003, pp. 40-42.
3. S. Ruggles, "Linking historical censuses: a new approach," History and Computing, vol. 14, no. 1+2, pp. 213-224, 2006.
4. E. Glasson, N. de Klerk, A. Bass, D. Rosman, L. Palmer, and C. Holman, "Cohort profile: The western Australian family connections genealogical project," International Journal of Epidemiology, vol. 37, pp. 30-35, 2008.
5. A. Reid, R. Davies, and E. Garrett, "Nineteenth century Scottish demography from linked censuses and civil registers: a 'sets of related individuals' approach," History and Computing, vol. 14, no. 1+2, pp. 61-86, 2006.
6. G. Bloothooft, "Multi-source family reconstruction," History and Computing, vol. 7, no. 2, pp. 90-103, 1995.
7. -, "Assessment of systems for nominal retrieval and historical record linkage," Computers and the Humanities, vol. 32, no. 1, pp. 39-56, 1998.
8. E. Fure, "Interactive record linkage: The cumulative construction of life courses," Demographic Research, vol. 3, p. 11, 2000.
9. I. Bhattacharya and L. Getoor, "Collective entity resolution in relational data," ACM Transactions on Knowledge Discovery from Data, vol. 1, no. 1, 2007.
10. M. Herschel and F. Naumann, "Scaling up duplicate detection in graph data," in Proceeding of the 17th ACM conference on Information and knowledge management, 2008, pp. 1325- 1326.
11. D. V. Kalashnikov and S. Mehrotra, "Domain-independent data cleaning via analysis of entity-relationship graph," ACM Transactions on Database Systems, vol. 31, no. 2, pp. 716- 767, 2006.
12. P. Christen, "Development and user experiences of an open source data cleaning, deduplication and record linkage system," ACM SIGKDD Explorations Newsletter, vol. 11, no. 1, pp. 39-48, 2009.
13. B.-W. On, N. Koudas, D. Lee, and D. Srivastava, "Group linkage," in Proceedings of the IEEE 23rd International Conference on Data Engineering, 2007, pp. 496-505.
14. P. Christen and D. Belacic, "Automated probabilistic address standardisation and verification," in Proceedings of Australasian Data Mining Conference, 2005, pp. 53-68.
15. T. Churches, P. Christen, K. Lim, and J. X. Zhu, "Preparation of name and address data for record linkage using hidden Markov models," BMC Medical Informatics and Decision Making, vol. 2, no. 9, 2002.
16. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proceedings of the IEEE, vol. 77, no. 2, pp. 257-286, 1999.
17. R. Vick and L. Huynh, "The effects of standardizing names for record linkage: Evidence from the United States and Norway," Historical Methods: A Journal of Quantitative and Interdisciplinary History, vol. 44, no. 1, pp. 15-24, 2011.
18. W. E. Winkler, "Overview of research linkage and current research directions," US Bureau of the Census, Statistical Research Report Series RRS2006/02, 2006.
19. R. Goeken, L. Huynh, T. A. Lynch, and R. Vick, "New methods of census record linking," Historical Methods: A Journal of Quantitative and Interdisciplinary History, vol. 44, no. 1, pp. 7-14, 2011.
20. K. Schürer and M. Woollard, The national sample of the 1881 census of Great Britain. A user Guide and workbook. Colchester, 2000.
21. T. Packer, J. Lutes, A. Stewart, D. Embley, E. Ringger, K. Seppi, and L. Jensen, "Extracting person names from diverse and noisy OCR text," in Proceedings of the fourth workshop on Analytics for noisy unstructured text data, 2010, pp. 19-26.
22. P. Christen, "A survey of indexing techniques for scalable record linkage and deduplication," IEEE Transactions on Knowledge and Data Engineering, 2011.
23. -, "A comparison of personal name matching: Techniques and practical issues," in Workshop on Mining Complex Data, held at IEEE ICDM'06, 2006, pp. 290-294.
24. W. W. Cohen, P. Ravikumar, and S. E. Fienberg, "A comparison of string distance metrics for name-matching tasks," in Proceedings of IJCAI-03 Workshop on Information Integration, 2003, pp. 73-78.