Detection of multivariate outliers in business survey data with incomplete information

Advances in Data Analysis and Classification

Volumn 5, Issue 1, 2011, Pages 37-56

  Todorov, Valentin ^a   Templ, Matthias ^b   Filzmoser, Peter ^b  

^a UNITED NATIONS INDUSTRIAL DEVELOPMENT ORGANIZATION   (Austria)

^b VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

Missing values; Multivariate outlier detection; Robust statistics

Indexed keywords

SURVEYS;

DATA EDITING; DATA PROBLEMS; EPIDEMIC ALGORITHM; FALSE DISCOVERY RATE; GNU GENERAL PUBLIC LICENSE; INCOMPLETE DATA; INCOMPLETE INFORMATION; MINIMUM COVARIANCE DETERMINANT; MISSING VALUES; MULTIVARIATE DATA; MULTIVARIATE OUTLIER DETECTION; MULTIVARIATE OUTLIERS; OUTLIER DETECTION; RANK CORRELATION; ROBUST ESTIMATE; ROBUST METHODS; ROBUST STATISTICS; S-ESTIMATORS; SIMULATION STUDIES; STATISTICAL DATAS; SURVEY DATA;

ESTIMATION;

EID: 79952184460     PISSN: 18625347     EISSN: 18625355     Source Type: Journal    
DOI: 10.1007/s11634-010-0075-2     Document Type: Article

References (54)

1. Barnett V, Lewis T (1994) Outliers in statistical data. Wiley, New York.
2. Béguin C, Hulliger B (2004) Multivariate outlier detection in incomplete survey data: the epidemic algorithm and transformed rank correlations. J R Stat Soc Ser B (Stat Methodol) 127(2): 275-294.
3. Béguin C, Hulliger B (2008) The BACON-EEM algorithm for multivariate outlier detection in incomplete survey data. Surv Methodol 34(1): 91-103.
4. Billor N, Hadi AS, Vellemann PF (2000) Bacon: blocked adaptative computationally-efficient outlier nominators. Comput Stat Data Anal 34(3): 279-298.
5. Campbell NA (1989) Bushfire maping using NOAA AVHRR data. Technical report, CSIRO.
6. Cerioli A, Riani M, Atkinson AC (2009) Controlling the size of multivariate outlier tests with the MCD estimator of scatter. Stat Comput 19(3): 341-353.
7. Chambers RL (1986) Outlier robust finite population estimation. J Am Stat Assoc 81: 1063-1069.
8. Copt S, Victoria-Feser MP (2004) Fast algorithms for computing high breakdown covariance matrices with missing data. In: Hubert M, Pison G, Struyf A, van Aelst S (eds) Theory and applications of recent robust methods, statistics for industry and technology series. Birkhauser, Basel.
9. Croux C, Haesbroeck G (1999) Influence function and efficiency of the minimum covariance determinant scatter matrix estimator. J Multivariate Analy 71: 161-190.
10. De Waal T (2003) Processing of erroneous and unsafe data. PhD thesis, Erasmus University, Rotterdam.
11. De Waal T (2009) Statistical data editing. In: Peffermann D, Rao C (eds) Handbook of statistics 29A. Sample surveys: design, methods and applications. Elsevier B. V., Amsterdam, pp 187-214.
12. Dempster AP, Laird MN, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B (Stat Methodol) 39: 1-22.
13. Dinges G, Haitzmann M (2009) Modellbasierte Ergänzung der Konjunkturstatistik im Produzierenden Bereich; Darstellung der statistischen Grundgesamtheit im Produzierenden Bereich. Statistische Nachrichten 9: 1153-1166. http://www. stat. at/web_de/downloads/methodik/kjp. pdf.
14. Donoho DL (1982) Breakdown properties of multivariate location estimators. Technical report, Harvard University, Boston. http://www-stat. stanford. edu/~donoho/Reports/Oldies/BPMLE. pdf.
15. EUREDIT Project (2004) Towards effective statistical editing and imputation strategies-findings of the Euredit project, vols 1 and 2. EUREDIT consortium. http://www. cs. york. ac. uk/euredit/results/results. html.
16. Eurostat (2008) NACE Rev. 2. Statistical classification of economic activites in the European community. Eurostat, methodologies and working papers, ISBN 978-92-79-04741-1.
17. Fellegi I, Holt D (1976) A systematic approach to automatic edit and imputation. J Am Stat Assoc 71: 17-35.
18. Filzmoser P, Garrett RG, Reimann C (2005) Multivariate outlier detection in exploration geochemistry. Comput Geosci 31: 579-587.
19. Filzmoser P, Maronna R, Werner M (2008) Outlier identification in high dimensions. Comput Stat Data Anal 52(3): 1694-1711.
20. Franklin S, Brodeur M (1997) A practical application of a robust multivariate outlier detection method. In: Proceedings of the survey research methods section. American Statistical Association, pp 186-191. http://www. amstat. org/sections/srms/proceedings.
21. Franklin S, Brodeur M, Thomas S (2000) Robust multivariate outlier detection using Mahalanobis' distance and Stahel-Donoho estimators. In: ICES II, international conference on establishment surveys II.
22. Granquist L (1990) A review of some macro-editing methods for rationalizing the editing process. In: Proceedings of the statistics Canada symposium, Ottawa, Canada, pp 225-234.
23. Granquist L (1997) The new view on editing. Int Stat Rev 65: 381-387.
24. Hampel FR, Ronchetti EM, Rousseeuw PJ, Stahel WA (1986) Robust statistics, the approach based on infuence functions. Wiley, New York.
25. Hardin J, Rocke DM (2005) The distribution of robust distances. J Comput Graph Stat 14: 910-927.
26. Hidiroglou MA, Lavallée P (2009) Sampling and estimation in business surveys. In: Peffermann D, Rao C (eds) Handbook of statistics 29A, sample surveys: design, methods and applications. Elsevier B. V., Amsterdam, pp 441-470.
27. Huber PJ (1981) Robust statistics. Wiley, New York.
28. Hubert M, Rousseeuw PJ, Vanden Branden K (2005) Robpca: a new approach to robust principal component analysis. Technometrics 47: 64-79.
29. Hubert M, Rousseeuw PJ, van Aelst S (2008) High-breakdown robust multivariate methods. Stat Sci 23: 92-119.
30. Johnson RA, Wichern DW (2002) Applied multivariate statistical analysis. 5th edn. Prentice Hall, New Jersey.
31. Lawrence D, McKenzie R (2000) The general application of significance editing. J Official Stat 16: 243-253.
32. Little RJA, Rubin DB (1987) Statistical analysis with missing data. Wiley, New York.
33. Little RJA, Smith PJ (1987) Editing and imputation for quantitative data. J Am Stat Assoc 82: 58-69.
34. Lopuhaä HP (1999) Asymptotics of reweighted estimators of multivariate location and scatter. Ann Stat 27: 1638-1665.
35. Lopuhaä HP, Rousseeuw PJ (1991) Breakdown points of affine equivariant estimators of multivariate location and covariance matrices. Ann Stat 19: 229-248.
36. Luzi O, De Waal T, Hulliger B, Di Zio M, Pannekoek J, Kilchmann D, Guarnera U, Hoogland J, Manzari A, Tempelman C (2007) Recommended practices for editing and imputation in cross-sectional business surveys. Report.
37. Maronna RA, Yohai VJ (1995) The behaviour of the Stahel-Donoho robust multivariate estimator. J Am Stat Assoc 90: 330-341.
38. Maronna RA, Zamar RH (2002) Robust estimation of location and dispersion for high-dimensional datasets. Technometrics 44: 307-317.
39. Maronna RA, Martin D, Yohai V (2006) Robust statistics: theory and methods. Wiley, New York.
40. R Development Core Team (2009) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. http://www. R-project. org/, ISBN 3-900051-07-0.
41. Riani M, Atkinson AC, Cerioli A (2009) Finding an unknown number of multivariate outliers. J R Stat Soc Ser B (Stat Methodol) 71(2): 447-466.
42. Rousseeuw PJ, Leroy AM (1987) Robust Regression and outlier detection. Wiley, New York.
43. Rousseeuw PJ, van Zomeren BC (1990) Unmasking multivariate outliers and leverage points. J Am Stat Assoc 85: 633-651.
44. Rubin DB (1993) Discussion: statistical disclosure limitation. J Official Stat 9: 462-468.
45. Schafer J (1997) Analysis of incomplete multivariate data. Chapman and Hall, London.
46. Schafer JL, Graham JW (2002) Missing data: our view of the state of the art. Psychol Methods 7: 147-177.
47. Stahel WA (1981a) Breakdown of covariance estimators. Research Report 31, ETH Zurich, Fachgruppe für Statistik.
48. Stahel WA (1981b) Robuste schätzungen: Infinitesimale optimalität und schätzungen von kovarianzmatrizen. PhD thesis no. 6881, Swiss Federal Institute of Technology (ETH), Zürich. http://www. e-collection. ethbib. ethz. ch/view/eth: 21890.
49. Templ M, Filzmoser P (2008) Visualization of missing values using the R-package VIM. Reserach report cs-2008-1, Department of Statistics and Probability Therory, Vienna University of Technology, Vienna.
50. Todorov V, Filzmoser P (2009) An object oriented framework for robust multivariate analysis. J Stat Softw 32(3): 1-47. http://www. jstatsoft. org/v32/i03/.
51. Vanden Branden K, Verboven S (2009) Robust data imputation. Comput Biol Chem 33(1): 7-13.
52. Venables WN, Ripley BD (2003) Modern applied statistics with S. 4th edn. Springer, Berlin.
53. Verboven S, Vanden Branden K, Goos P (2007) Sequential imputation for missing values. Comput Biol Chem 31(5-6): 320-327.
54. Wegman E (1990) Hyperdimensional data analysis using parallel coordinates. J Am Stat Assoc 85: 664-675.