A robust missing value imputation method for noisy data

Applied Intelligence

Volumn 36, Issue 1, 2012, Pages 61-74

  Zhu, Bing ^a   He, Changzheng ^a   Liatsis, Panos ^b  

^a Sichuan University   (China)

^b CITY UNIVERSITY   (United Kingdom)

Author keywords

Group method of data handling (GMDH); Missing data imputation; Noise

Indexed keywords

BENCHMARK DATASETS; GROUP METHOD OF DATA HANDLING; IMPACT OF NOISE; IMPUTATION METHODS; IMPUTATION TECHNIQUES; INCOMPLETE DATA; MISSING DATA; MISSING VALUE IMPUTATION; NOISE; NOISY DATA; REAL WORLD DATA; RESEARCH TOPICS;

BENCHMARKING; DATA MINING;

DATA HANDLING;

EID: 84856282936     PISSN: 0924669X     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10489-010-0244-1     Document Type: Article

References (50)

1. RE Abdel-Aal 2005 GMDH-based feature ranking and selection for improved classification of medical data J Biomed Inf 38 6 456 468 10.1016/j.jbi.2005.03. 003 (Pubitemid 41758095)
2. TI Aksenova YP Yurachkovsky 1988 A characterisation at unbiased structure and conditions of their J-optimality Sov J Autom Inf Sci 21 4 36 42 0706.62064
3. Asuncion A, Newman DJ (2007) UCI machine learning repository. Irvine, CA: University of California, School of Information and Computer Science. http://www.ics.uci.edu/mlearn/MLRepository.html
4. Aussem A, de Morais SR (2008) A conservative feature subset selection algorithm with missing data. In: Kellenberger P (ed) Proc eighth IEEE int conf on data mining, ICDM'08, Pisa, Italy, pp 725-730
5. Barron AR, Barron RL (1988) Statistical learning networks: A unifying view. In: Wegman E (ed) Proc the 20th symposium on the interface: computing science and statistics. American Statistical Association, Washington, pp 192-203
6. G Batista MC Monard 2003 An analysis of four missing data treatment methods for supervised learning Appl Artif Intell 17 5-6 519 533 10.1080/713827181
7. Beaumont JF (2000) On regression imputation in the presence of nonignorable nonresponse. In: Proc of the survey research methods section, ASA, pp 580-585
8. S Chen C Huang 2003 Generating weighted fuzzy rules from relational database systems for estimating null values using genetic algorithms IEEE Trans Fuzzy Syst 11 4 495 506 10.1109/TFUZZ.2003.814837
9. S Chen C Huang 2008 A new approach to generate weighted fuzzy rules using genetic algorithms for estimating null values Expert Syst Appl 35 3 905 917 10.1016/j.eswa.2007.07.033
10. AP Dempster NM Laird DB Rubin 1977 Maximum likelihood from incomplete data via the EM algorithm (with discussion) J R Stat Soc B 39 1 38 0364.62022 501537
11. Elder JF, Brown DE (2000) Induction and polynomial networks. In: Fraser MD (ed) Proc network models for control and processing, induction and polynomial networks. Intellect Books, Exeter, pp 143-198
12. A Farhangfar L Kurgan W Pedrycz 2007 A novel framework for imputation of missing values in databases IEEE Trans Syst Man Cybern Part A, Syst Humans 37 5 692 709 10.1109/TSMCA.2007.902631 (Pubitemid 47323534)
13. A Farhangfar L Kurgan J Dy 2008 Impact of imputation of missing values on classification error for discrete data Pattern Recogn 41 12 3692 3705 10.1016/j.patcog.2008.05.019 1173.68479
14. Ford BL (1983) An overview of hot-deck procedures. In: Madow WG, OIkin I, Rubin DB (eds) Incomplete data in sample surveys, vol II: theory and bibliographies. Academic Press, New York, pp 85-207
15. O Harel XH Zhou 2007 Multiple imputation: Review of theory, implementation and software Stat Med 26 16 3057 3077 10.1002/sim.2787 2380504 (Pubitemid 47019007)
16. RJ Hathaway JC Bezdek 2002 Clustering incomplete relational data using the non-Euclidean relational fuzzy c-means algorithm Pattern Recogn Lett 23 1-3 151 160 10.1016/S0167-8655(01)00115-5 0993.68108 (Pubitemid 33119508)
17. ER Hruschka Jr ER Hruschka N Ebecken 2007 Bayesian networks for imputation in classification problems J Intell Inf Syst 29 3 231 252 10.1007/s10844-006-0016-x (Pubitemid 350172228)
18. CC Huang HM Lee 2004 A grey-based nearest neighbor approach for missing attribute value prediction Appl Intell 20 239 252 10.1023/B:APIN.0000021416. 41043.0f 1069.68577
19. AG Ivakhnenko 1968 The group method of data handling-a rival of the method of stochastic approximation Sov Autom Control 1-3 43 55
20. AG Ivakhnenko 1971 Polynomial theory of complex systems IEEE Trans Syst Man Cybern 1 4 364 378 10.1109/TSMC.1971.4308320 309583
21. AG Ivakhnenko YL Kocherga 1983 Theory of two-level GMDH algorithms for long-range quantitative prediction Sov Autom Control 16 6 7 12 (Pubitemid 14647063)
22. Ivakhnenko AG, Stepashko VS (1985) Noise stability of modeling. Naukova Dumka, Kiev
23. K Lakshminarayan SA Harp T Samad 1999 Imputation of missing data in industrial databases Appl Intell 11 3 259 275 10.1023/A:1008334909089 (Pubitemid 30517149)
24. F Lemke J Mueller 2003 Self-organising data mining Syst Anal Model Simul 43 2 231 240 10.1080/0232929031000136135 (Pubitemid 44112342)
25. Little R, Rubin D (2002) Statistical analysis with missing data, 2nd edn. Wiley, New York
26. Madala HR, Ivakhnenko AG (1994) Inductive learning algorithms for complex systems modeling. CRC Press, Boca Raton
27. S Mani M Valtorta S McDermott 2005 Building Bayesian network models in medicine: The MENTOR experience Appl Intell 22 2 93 108 10.1007/s10489-005-5599- 3 (Pubitemid 40400728)
28. M Mannino Y Yang Y Ryu 2009 Classification algorithm sensitivity to training data with non representative attribute noise Decis Support Syst 46 3 743 751 10.1016/j.dss.2008.11.021
29. M Mehrara, et al. 2009 Investigating the efficiency in oil futures market based on GMDH approach Expert Syst Appl 36 4 7479 7483 10.1016/j.eswa.2008.09. 055
30. Miller RG (1997) Beyond ANOVA: basics of applied statistics. Chapman & Hall, Boca Raton
31. Mueller JA, Lemke F (2000) Self-organizing data mining: an intelligent approach to extract knowledge from data. Libri Books, Berlin
32. I Myrtveit E Stensrud U Olsson 2001 Analyzing data sets with missing data: An empirical evaluation of imputation methods and likelihood-based methods IEEE Trans Softw Eng 27 11 999 1013 10.1109/32.965340 (Pubitemid 33106028)
33. S Oh W Pedrycz 2002 The design of self-organizing polynomial neural networks Inf Sci 141 3-4 237 258 10.1016/S0020-0255(02)00175-5 1008.68098 (Pubitemid 34524130)
34. A Olinsky S Chen L Harlow 2003 The comparative efficacy of imputation methods for missing data in structural equation modeling Eur J Oper Res 151 1 53 79 10.1016/S0377-2217(02)00578-7 1113.62361 1997008
35. V Puig, et al. 2007 A GMDH neural network-based approach to passive robust fault detection using a constraint satisfaction backward test Eng Appl Artif Intell 20 7 886 897 10.1016/j.engappai.2006.12.005 (Pubitemid 47432842)
36. Y Qin, et al. 2007 Semi-parametric optimization for missing data imputation Appl Intell 27 1 79 88 10.1007/s10489-006-0032-0 1189.68124 (Pubitemid 46960845)
37. Quinlan JR (1993) C4. 5: Programs for machine learning. Morgan Kauffman, Los Altos
38. DB Rubin 1976 Inference and missing data Biometrika 63 3 581 592 10.1093/biomet/63.3.581 0344.62034 455196
39. M Saar-Tsechansky F Provost 2007 Handling missing values when applying classification models J Mach Learn Res 8 1625 1657 (Pubitemid 47210270)
40. JL Schafer 1999 Multiple imputation: A primer Stat Methods Med Res 8 1 3 15 10.1191/096228099671525676 (Pubitemid 29222784)
41. VS Stepashko YP Yurachkovskiy 1986 The present state of the theory of the group method of data handling Sov J Autom Inf Sci c/c of Avtomatika 19 4 36 46 0636.93072
42. Tan PN, Steinbach M, Kumar V (2005) Introduction to data mining. Addison-Wesley, Boston
43. N Tsikriktsis 2005 A review of techniques for treating missing data in OM survey research J Oper Manag 24 1 53 62 (Pubitemid 41570132)
44. B Twala 2009 An empirical comparison of techniques for handling incomplete data when using decision trees Appl Artif Intell 23 5 373 405 10.1080/08839510902872223
45. F Ungaro C Calzolari E Busoni 2005 Development of pedotransfer functions using a group method of data handling for the soil of the Pianura Padano-Veneta region of North Italy: Water retention properties Geoderma 124 3-4 293 317 10.1016/j.geoderma.2004.05.007 (Pubitemid 39568972)
46. S Van Buuren, et al. 2006 Fully conditional specification in multivariate imputation J Stat Comput Simul 76 12 1049 1064 10.1080/10629360600810434 1144.62332 2307507 (Pubitemid 44843816)
47. J Van Hulse TM Khoshgoftaar 2008 A comprehensive empirical evaluation of missing value imputation in noisy software measurement data J Syst Softw 81 5 691 708 (Pubitemid 351389565)
48. D Williams, et al. 2007 On classification with incomplete data IEEE Trans Pattern Anal Mach Intell 29 3 427 436 10.1109/TPAMI.2007.52 (Pubitemid 46336418)
49. X Wu X Zhu 2008 Mining with noise knowledge: Error-aware data mining IEEE Trans Syst Man Cybern Part A 38 4 917 932 10.1109/TSMCA.2008.923034
50. X Zhu X Wu 2004 Class noise vs. attribute noise: A quantitative study Artif Intell Rev 22 3 177 210 10.1007/s10462-004-0751-8 1069.68587 2083952