K nearest neighbours with mutual information for simultaneous classification and missing data imputation

Neurocomputing

Volumn 72, Issue 7-9, 2009, Pages 1483-1493

  García Laencina, Pedro J ^a   Sancho Gómez, José Luis ^a   Figueiras Vidal, Aníbal R ^b   Verleysen, Michel ^c  

^a TECHNICAL UNIVERSITY OF CARTAGENA   (Spain)

^b UNIVERSIDAD CARLOS III DE MADRID   (Spain)

^c UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

Author keywords

Imputation; K nearest neighbours; Missing data; Mutual information; Pattern classification

Indexed keywords

IMPUTATION; K NEAREST NEIGHBOURS; MISSING DATA; MUTUAL INFORMATION; PATTERN CLASSIFICATION;

REAL TIME SYSTEMS;

ACCURACY; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; CLASSIFICATION; CLASSIFICATION ALGORITHM; CLASSIFIER; DATA BASE; INFORMATION PROCESSING; K NEAREST NEIGHBOR; MACHINE LEARNING; PRIORITY JOURNAL; PROBLEM SOLVING;

EID: 61849150502     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2008.11.026     Document Type: Article

References (44)

1. Acuna E., and Rodriguez C. The treatment of missing values and its effect in the classifier accuracy. In: Banks D., House L., McMorris F.R., Arabie P., and Gaul W. (Eds). Classification, Clustering and Data Mining Applications (2004), Springer, Berlin 639-648
2. D.W. Aha (Ed.), Lazy learning, Kluwer Academic Publishers, Norwell, MA, USA, 1997.
3. Aha D.W., Kibler D.F., and Albert M.K. Instance-based learning algorithms. Machine Learning 6 (1991) 37-66
4. P.D. Allison, Missing data, Sage University Papers Series on Quantitative Applications in the Social Sciences, Thousand Oaks, California, USA, 2001.
5. G.E. Batista, M.C. Monard, A study of k-nearest neighbour as an imputation method, in: Second International Conference on Hybrid Intelligent Systems, vol. 87,, Santiago, Chile, 2002, pp. 251-260.
6. Batista G.E., and Monard M.C. An analysis of four missing data treatment methods for supervised learning. Applied Artificial Intelligence 17 5-6 (2003) 519-533
7. Bishop C.M. Neural Networks for Pattern Recognition (1995), Oxford University Press, Oxford, UK
8. Brown J.G. Using a multiple imputation technique to merge data sets. Applied Economics Letters 9 5 (2002) 311-314
9. Cover T.M. Estimation by the nearest neighbor rule. IEEE Transactions on Information Theory 14 (1968) 50-55
10. Cover T.M., and Hart P.E. Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13 1 (1967) 21-27
11. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), Wiley-Interscience, New York
12. Cristianini N., and Shawe-Taylor J. Support Vector Machines and Other Kernel-Based Learning Methods (2000), Cambridge University Press, Cambridge
13. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification (2000), Wiley-Interscience, New York
14. Dudani S.A. The distance-weighted k-nearest-neighbor rule. IEEE Transactions on Systems, Man and Cybernetics 6 4 (1976) 325-327
15. Francois D., Rossi F., Wertz V., and Verleysen M. Resampling methods for parameter-free and robust feature selection with mutual information. Neurocomputing 70 7-9 (2007) 1276-1288
16. Gabrys B. Neuro-fuzzy approach to processing inputs with missing values in pattern recognition problems. International Journal of Approximate Reasoning 30 3 (2002) 149-179
17. Ghahramani Z., and Jordan M.I. Supervised learning from incomplete data via an EM approach. In: Cowan J.D., Tesauro G., and Alspector J. (Eds). Advances in NIPS vol. 6 (1994), Morgan Kaufmann Publishers, Inc., Los Altos, CA 120-127
18. M. Halatchev, L. Gruenwald, Estimating missing values in related sensor data streams, in: Jayant, R. Haritsa, T.M. Vijayaraman (Eds.), COMAD, Computer Society of India, 2005, pp. 83-94.
19. Hammer B., and Villmann T. Generalized relevance learning vector quantization. Neural Networks 15 8-9 (2002) 1059-1068
20. K. Hechenbichler, K. Schliep, Weighted k-nearest-neighbor techniques and ordinal classification. Technical Report, Ludwig-Maximilians University Munich, 2007.
21. Ishibuchi H., Miyazaki A., Kwon K., and Tanaka H. Learning from incomplete training data with missing values and medical application. Proceedings of IEEE International Joint Conference on Neural Networks (1993) 1871-1874
22. J.M. Jerez, I. Molina, J.L. Subirats, L. Franco, Missing data imputation in breast cancer prognosis, in: BioMed'06: Proceedings of the 24th IASTED International Conference on Biomedical Engineering, ACTA Press, Anaheim, CA, USA, 2006, pp. 323-328.
23. H. Kim, G.H. Golub, H. Park, Imputation of missing values in DNA microarray gene expression data, in: IEEE Computational Systems Bioinformatics Conference, 2004.
24. Kim H., Golub G.H., and Park H. Missing value estimation for DNA microarray gene expression data: local least squares imputation. Bioinformatics 21 2 (2005) 187-198
25. Kullback S. Information Theory and Statistics (1959), Wiley, New York
26. Kwak N., and Choi C.-H. Input feature selection by mutual information based on Parzen window. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 12 (2002) 1667-1671
27. Little R.J.A. Methods for handling missing values in clinical trials. Journal of rheumatology 26 8 (1999) 1654-1656
28. Little R.J.A., and Rubin D.B. Statistical Analysis with Missing Data. second ed. (2002), Wiley, NJ, USA
29. M.K. Markey, A. Patel, Impact of missing data in training artificial neural networks for computer-aided diagnosis, in: International Conference on Machine Learning and Applications, December 2004, pp. 351-354.
30. McLachlan G.J., and Krishnan T. The EM Algorithm and Extensions (1997), Wiley, New York
31. Mitchell T.M. Machine Learning (1997), McGraw-Hill, New York
32. S. Narayanan, R.J. Marks II, J.L. Vian, J.J. Choi, M.A. El-Sharkawi, B.B. Thompson, Set constraint discovery: missing sensor data restoration using auto-associative regression machines, in: Proceedings of the 2002 International Joint Conference on Neural Networks, Honolulu, May 2002, pp. 2872-2877.
33. D.J. Newman, S. Hettich, C.L. Blake, C.J. Merz, UCI repository of machine learning databases, 1998.
34. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann, Los Altos, CA
35. Rossi F., Lendasse A., Francois D., Wertz V., and Verleysen M. Mutual information for the selection of relevant variables in spectrometric nonlinear modeling. Chemometrics and Intelligent Laboratory Systems 80 (2006) 215-226
36. Rubin D.B. Multiple Imputation for Nonresponse in Surveys (1987), Wiley, New York
37. Schafer J.L. Analysis of Incomplete Multivariate Data (1997), Chapman & Hall, Florida, USA
38. Sehgal M.S.B., Gondal I., and Dooley L. Collateral missing value imputation: a new robust missing value estimation algorithm for microarray data. Bioinformatics 21 10 (2005) 2417-2423
39. Y. Song, J. Huang, D. Zhou, H. Zha, C.L. Giles, IKNN: Informative k-nearest neighbor pattern classification, in: 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, Warsaw, Poland, 2007, pp. 248-264.
40. Troyanskaya O., Cantor M., Alter O., Sherlock G., Brown P., Botstein D., Tibshirani R., Hastie T., and Altman R. Missing value estimation methods for DNA microarrays. Bioinformatics 17 6 (2001) 520-525
41. S. Tsumoto, Problems with mining medical data, in: 24th International Computer Software and Applications Conference, IEEE Computer Society, Washington, DC, USA, 2000, pp. 467-468.
42. Villmann T., Schleif F.-M., and Hammer B. Comparison of relevance learning vector quantization with other metric adaptive classification methods. Neural Networks 19 5 (2006) 610-622
43. Weinberger K., Blitzer J., and Saul L. Distance metric learning for large margin nearest neighbor classification. In: Weiss Y., Schölkopf B., and Platt J. (Eds). Advances in NIPS 18 (2006), MIT Press, Cambridge, MA 1473-1480
44. Wettschereck D., Aha D.W., and Mohri T. A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Artificial Intelligence Review 11 1-5 (1997) 273-314