A hierarchical Bayesian approach to record linkage and population size problems

Annals of Applied Statistics

Volumn 5, Issue 2 B, 2011, Pages 1553-1585

  Tancredi, Andrea ^a   Liseo, Brunero ^a  

^a SAPIENZA UNIVERSITY OF ROME   (Italy)

Author keywords

Capture recapture methods; Conditional independence; Gibbs sampling; Metropolis hastings; Record linkage

Indexed keywords

EID: 84867545590     PISSN: 19326157     EISSN: 19417330     Source Type: Journal    
DOI: 10.1214/10-AOAS447     Document Type: Article

References (60)

1. Albert, P. S. and Dood, L. E. (2004). A cautionary note on robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics 60 427-435. Mathematical Reviews (MathSciNet): MR2066277, Digital Object Identifier: doi:10.1111/j.0006-341X.2004.00187.x
2. Alleva, G., Fortini, M. and Tancredi, A. (2007). The control of non-sampling errors on linked data: An application on population census. In Proceedings of the 2007 Intermediate Conference. Risk and Prediction. Venice.
3. Armstrong, J. and Mayda, J. E. (1993). Model-based estimation of record linkage error rates. Survey Methodology 19 137-147.
4. Belin, T. R. and Rubin, D. B. (1995). A method for calibrating false-match rates in record linkage. J. Amer. Statist. Assoc. 90 694-707.
5. Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300. Mathematical Reviews (MathSciNet): MR1325392
6. Bhattacharya, I. and Getoor, L. (2007). Collective entity resolution in relational data. ACM Transactions on Knowledge Discovery from Data (TKDD) 1 Article No. 5.
7. Bishop, Y. M. M., Fienberg, S. E. and Holland, P. W. (1975). Discrete Multivariate Analysis: Theory and Parctice. MIT Press, Cambridge. Mathematical Reviews (MathSciNet): MR381130
8. Christen, P. (2005). Probabilistic data generation for deduplication and data linkage. In IDEAL'05. LNCS 3578 109-116. Springer, Berlin.
9. Christen, P. and Pudjijono, A. (2009). Accurate synthetic generation of realistic personal information. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD'09). LNAI 5476 507-514. Springer, Berlin.
10. Copas, J. B. and Hilton, F. J. (1990). Record linkage: Statistical models for matching computer records. J. Roy. Statist. Soc. Ser. A 153 287-320.
11. Darroch, J. N. (1958). The multiple capture census I. Estimation of a closed population. Biometrika 45 343-358. Mathematical Reviews (MathSciNet): MR119360, Zentralblatt MATH: 0099.14703
12. DeGroot, M. H. and Goel, P. K. (1980). Estimation of the correlation coefficient from a broken random sample. Ann. Statist. 8 264-278. Mathematical Reviews (MathSciNet): MR560728, Zentralblatt MATH: 0446.62049, Digital Object Identifier: doi:10.1214/aos/1176344952, Project Euclid: euclid.aos/1176344952
13. Do, K. A., Mueller, P. and Tang, F. (2005). A Bayesian mixture model for differential gene expression. J. Roy. Statist. Soc. Ser. C. 54 627-644. Mathematical Reviews (MathSciNet): MR2137258, Zentralblatt MATH: 05188702, Digital Object Identifier: doi:10.1111/j.1467-9876.2005.05593.x
14. Ericson, W. A. (1969). Subjective Bayesian models in sampling finite populations. J. Roy. Statist. Soc. Ser. B 31 195-224. Mathematical Reviews (MathSciNet): MR270494
15. Erosheva, E., Fienberg, S. and Joutard, C. (2007). Describing disability through individual level mixture models for multivariate binary data. Ann. Appl. Statist. 1 502-537. Mathematical Reviews (MathSciNet): MR2415745, Zentralblatt MATH: 1126.62101, Digital Object Identifier: doi:10.1214/07-AOAS126, Project Euclid: euclid.aoas/1196438029
16. Fellegi, I. P. and Sunter, A. B. (1969). A theory of record linkage. J. Amer. Statist. Assoc. 64 1183-1210.
17. Fienberg, S. E. (2011). Bayesian models and methods in public policy and government settings. Statist. Sci. To appear. Mathematical Reviews (MathSciNet): MR2858384, Digital Object Identifier: doi:10.1214/10-STS331, Project Euclid: euclid.ss/1312204010
18. Fienberg, S. E., Johnson, M. S. and Junker, B. W. (1999). Classical multilevel and Bayesian approaches to population size estimation using multiple list, part 3. J. Roy. Statist. Soc. Ser. A 162 383-405.
19. Fienberg, S. E. and Manrique-Vallier, D. (2009). Integrated methodology for multiple systems estimation and record linkage using a missing data formulation. Advances in Statistical Analysis 93 49-60. Mathematical Reviews (MathSciNet): MR2476299, Digital Object Identifier: doi:10.1007/s10182-008-0084-z
20. Forster, J. J. and Webb, E. L. (2007). Bayesian disclosure risk assesment: Predicting small frequencies in contingency table. J. Roy. Statist. Soc. Ser. C 56 551-570. Mathematical Reviews (MathSciNet): MR2405419, Digital Object Identifier: doi:10.1111/j.1467-9876.2007.00591.x
21. Fortini, M., Liseo, B., Nuccitelli, A. and Scanu, M. (2001). On Bayesian record linkage. Research in Official Statistics 4 185-198.
22. Fortini, M., Liseo, B., Nuccitelli, A. and Scanu, M. (2002). Modelling issues in record linkage: A Bayesian perspective. In Proceedings of the Section on Survey Research Methods 1008-1013. Amer. Statist. Assoc., Alexandria, VA.
23. Genovese, C. and Wasserman, L. (2003). Bayesian and frequentist multiple testing (with discussion). In Bayesian Statistics, 7 (Tenerife, 2002) 145-161. Oxford Univ. Press, New York. Mathematical Reviews (MathSciNet): MR2003171
24. Green, P. J. and Mardia, K. V. (2006). Bayesian alignment using hierarchical models, with applicationin protein bioinformatics. Biometrika 93 235-254. Mathematical Reviews (MathSciNet): MR2278080, Zentralblatt MATH: 1153.62020, Digital Object Identifier: doi:10.1093/biomet/93.2.235
25. Herzog, T. N., Scheuren, F. J. and Winkler, W. E. (2007). Data Quality and Record Linkage Techniques. Springer, New York. Zentralblatt MATH: 05164849
26. Hoadley, B. (1969). The compound multinomial distribution and Bayesian analysis of categorical data from finite populations. J. Amer. Statist. Assoc. 64 216-229. Mathematical Reviews (MathSciNet): MR240916, Zentralblatt MATH: 0177.22303, Digital Object Identifier: doi:10.1080/01621459.1969.10500965
27. Jaro, M. (1989). Advances in record-linkage methodology as applied to matching the 1985 census of Tampa, Florida. J. Amer. Statist. Assoc. 84 414-420.
28. Judson, D. H. (2007). Information integration for constructing social statistics: History, theory and ideas towards a reserach program. J. Roy. Statist. Soc. Ser. A 170 483-501. Mathematical Reviews (MathSciNet): MR2364307, Digital Object Identifier: doi:10.1111/j.1467-985X.2007.00472.x
29. Kalashnikov, D. V. and Mehrotra, S. (2006). Domain-independent data cleaning via analysis of entity-relationship graph. ACM Transactions on Database Systems 31 716-767.
30. Kelley, P. (1986). Robustness of the Census Bureau's record linkage system. In Proceedings of the Section on Survey Research Methods, American Statistical Association 620-624.
31. Lahiri, P. and Larsen, M. D. (2005). Regression analysis with linked data. J. Amer. Statist. Assoc. 100 222-230. Mathematical Reviews (MathSciNet): MR2156832, Zentralblatt MATH: 1117.62376, Digital Object Identifier: doi:10.1198/016214504000001277
32. Larsen, M. D. (1999). Multiple imputation analysis of records linked using mixture model. In Proceedings of Survey Methods Section, Statistical Society of Canada 65-71. Statistical Society of Canada, Ottawa.
33. Larsen, M. D. (2004). Record Linkage using finite mixture models. In Applied Bayesian Modeling and Causal Inference from Incomplete Data Perspectives (A. Gelman and X.-L. Meng, eds.) 309-318. Wiley, New York. Mathematical Reviews (MathSciNet): MR2138266, Zentralblatt MATH: 05274827
34. Larsen, M. D. (2005). Advances in record linkage theory: Hierarchical Bayesian record linkage theory. In Proceedings of the Section on Survey Research Methods 3277-3283. Amer. Statist. Assoc., Alexandria, VA.
35. Larsen, M. D. and Rubin, D. B. (2001). Iterative automated record linkage using mixture models. J. Amer. Statist. Assoc. 96 32-41. Mathematical Reviews (MathSciNet): MR1973781, Digital Object Identifier: doi:10.1198/016214501750332956
36. Lindley, D. V. (1977). A problem in forensic science. Biometrika 64 207-213. Mathematical Reviews (MathSciNet): MR518935, Digital Object Identifier: doi:10.1093/biomet/64.2.207
37. Link, W. A., Yoshizaki, J., Bailey, L. L. and Pollock, K. H. (2009). Uncovering a latent multinomial: Analysis of mark-recapture data with misidentication. Biometrics 66 178-185. Mathematical Reviews (MathSciNet): MR2756704, Digital Object Identifier: doi:10.1111/j.1541-0420.2009.01244.x
38. Liseo, B. and Tancredi, A. (2009). Bayesian estimation of population size via linkage of multivariate Normal data sets. Working Paper 66. Dept. Methods and Models for Economics, Territory and Finance, Sapienza Univ. Rome.
39. Manrique-Vallier, D. and Fienberg, S. E. (2008). Population size estimation using individual level mixture models. Biom. J. 50 1051-1063. Mathematical Reviews (MathSciNet): MR2649394, Digital Object Identifier: doi:10.1002/bimj.200810448
40. Marin, J. M. and Robert, C. P. (2007). Bayesian Core. A Practical Approach to Computational Statistics. Springer, New York. Mathematical Reviews (MathSciNet): MR2289769, Zentralblatt MATH: 1137.62013
41. McGlincy, M. (2004). A Bayesian record linkage methodology for multiple imputation of missing data. In Proceedings of the Section on Survey Research Methods 4001-4008. Amer. Statist. Assoc., Alexandria, VA.
42. Newcombe, H. B. (1967). Record linkage: The design of efficient systems for linking records into individual and family histories. American Journal of Human Genetics 9 335-359.
43. Newcombe, H. B., Kennedy, J. M., Axford, S. J. and James, A. P. (1959). Automatic linkage of vital records. Science 130 954-959.
44. Norén, G. N., Orre, R. and Bate, A. (2005). A hit-miss model for duplicate detection in the WHO drug safety database. In KDD'05: Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining 459-468. Canada.
45. O'Hagan, A. and Forster, J. (2004). Kendall's Advanced Theory of Statistics. Volume 2B. Bayesian Inference. Arnold, London. Mathematical Reviews (MathSciNet): MR1285356
46. Pepe, M. S. (2003). The Statistical Evaluation of Medical Test for Classification and Prediction. Oxford Univ. Press, London. Mathematical Reviews (MathSciNet): MR2260483
47. Pepe, M. S. and Janes, H. (2007). Insights into latent class analysis of diagnostic test performance. Biostatistics 8 474-484.
48. Perez, C. J., Giron, F. J., Martin, J., Ruiz, M. and Rojano, C. (2007). Misclassified multinomial data: A Bayesian approach. Rev. R. Acad. Cien. Serie A. Mat 101 71-80. Mathematical Reviews (MathSciNet): MR2324581, Zentralblatt MATH: 1137.62317
49. R Development Core Team (2009). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
50. Robert, C. P. and Casella, G. (2004). Monte Carlo Statistical Methods, 2nd ed. Springer, New York. Mathematical Reviews (MathSciNet): MR2080278
51. Ruffieux, Y. and Green, P. J. (2009). Alignment of multiple configurations using hierachical models. J. Comput. Graph. Statist. 18 756-773. Mathematical Reviews (MathSciNet): MR2572636, Digital Object Identifier: doi:10.1198/jcgs.2009.07048
52. Seber, G. A. F. (1986). A review of estimating animal abundance. Biometrics 42 267-292.
53. Swartz, T., Haitovsky, Y., Vexler, A. and Yang, T. (2004). Bayesian identifiability and misclassification in multinomial data. Canad. J. Statist. 32 285-302. Mathematical Reviews (MathSciNet): MR2101757, Digital Object Identifier: doi:10.2307/3315930
54. Tancredi, A. and Liseo, B. (2011). Supplement to "A hierarchical Bayesian approach to record linkage and population size problems." DOI: 10.1214/10-AOAS447SUPP.
55. Winkler, W. E. (1993). Improved decision rules in the Fellegi-Sunter model of record linkage. In Proceedings of the Section on Survey Research Methods 274-279. Amer. Statist. Assoc., Alexandria, VA.
56. Winkler, W. E. (1995). Matching and record linkage. In Buisness Survey Methods (B. G. Cox, D. A. Binder, B. N. Chinnappa, A. Christianson, M. J. Colledge and P. S. Kott, eds.) 355-384. Wiley, New York.
57. Winkler, W. E. (2000). Machine learning, information retrieval and record linkage. In Proceedings of the Section on Survey Research Methods 20-29. Amer. Statist. Assoc., Alexandria, VA.
58. Winkler, W. E. (2004). Approximate string comparator search strategies for very large administrative lists. In Proceedings of the Section on Survey Research Methods 4595-4602. Amer. Statist. Assoc., Alexandria, VA.
59. Wolter, K. M. (1986). Some coverage error models for census data. J. Amer. Statist. Assoc. 81 338-345. Mathematical Reviews (MathSciNet): MR845875, Zentralblatt MATH: 0629.62018, Digital Object Identifier: doi:10.2307/2289222
60. Wright, J. A., Baker, R. J., Schofield, M. R., Frantz, A. C., Byrom, A. E. and Gleeson, D. M. (2009). Incorporating genotype uncertainty into mark-recapture-type models for estimating abundance using DNA samples. Biometrics 65 833-840. Mathematical Reviews (MathSciNet): MR2649856,Digital Object Identifier: doi:10.1111/j.1541-0420.2008.01165.x