Improving performance of the k-nearest neighbor classifier by combining feature selection with feature weighting

Transactions of the Japanese Society for Artificial Intelligence

Volumn 17, Issue 3, 2002, Pages 209-216

  Bao, Yongguang ^a   Du, Xiaoyong ^b   Ishii, Naohiro ^a  

^a NAGOYA INSTITUTE OF TECHNOLOGY   (Japan)

^b RENMIN UNIVERSITY OF CHINA   (China)

Author keywords

Classification; Feature selection; Feature weighting; Machine learning; Nformation theory; Rough sets

Indexed keywords

CLASSIFICATION; FEATURE SELECTION; FEATURE WEIGHTING; ROUGH SETS;

ARTIFICIAL INTELLIGENCE; FEATURE EXTRACTION; INFORMATION THEORY; LEARNING ALGORITHMS; LEARNING SYSTEMS; SENSITIVITY ANALYSIS; SET THEORY;

CLASSIFICATION (OF INFORMATION);

EID: 18444396928     PISSN: 13460714     EISSN: 13468030     Source Type: Journal    
DOI: 10.1527/tjsai.17.209     Document Type: Article

References (45)

1. [Aha 91] D. W. Aha, D. Kibler & M. K. Albert, "Instance-Based Learning Algorithms", Machine Learning, Vol. 6, 1991, pp. 37-66.
2. [Aha 92] D. W. Aha, "Tolerating Noisy, Irrelevant and Novel Attributes in Instance-Based Learning Algorithm", International Journal of Man-Machine Studies, Vol.36, No.1, 1992, pp. 267-287.
3. [Aha 94] D.W. Aha & R.L. Bankert: Feature Selection for case-based classification of Cloud Types: An Empirical Comparison, In D.W. Aha (Ed.) Case-Based Reasoning: Papers from the 1994 Workshop ((TR WS-94-10), Menlo Park, CA: AAAI press.
4. [Aha 98] D.W. Aha: Feature weighting for lazy learning algorithms, In H. Liu & H. Motoda (Eds.) Feature Extraction, Construction and Selection: A Data Mining Perspective, Norwell MA: Kluwer, 1998.
5. [Alien 97] L. B. Allen: Selection of Distance Metrics and Feature Subsets for k-Nearest Neighbor Classifiers, PhD thesis, University of Virginia, 1997.
6. [Ayan 99] N.F. Ayan: Using Information Gain as feature Weight, 8th Turkish Symposium on Artificial Intelligence and Neural Networks (TAINN'99), Istanbul, Turkey, June 1999.
7. [Bao 00] Y. Bao, X. Du & N. Ishii: Using Rough Sets and Class Mutual Information for Feature Selection, 6th international Conference on soft computing, IIZUKA2000, pp.452-458.
8. [Bao 01] Y. Bao, X. Du & N. Ishii: Improving Performance of the K-Nearest Neighbor Classification by Tolerant Rough Sets, Proc. third International Symposium on Cooperative Database Systems for Advanced Applications (CODS'2001), Beijing, China, 2001, pp. 167-171.
9. [Battit 94] R. Battiti: Using Mutual Information for Selection Features in Supervised Neural Net Learning, IEEE Trans. On Neural Networks, Vol. 5 No. 4, pp.537-550 (1994).
10. [Cardie 93] C. Cardie: Using Dicision Trees to improve Case-Based learning, In Proceedings of the 10th International Conference on Machine learning, 1993, pp.25-32.
11. [Cardie 00] C. Cardie: A Cognitive Bias Approach to Feature Selection and Weighting for Case-Based Learners, Machine Learning, Vol. 41, pp.85-116, (2000).
12. [Cost 93] S. Cost & S. Salzberg: A Weighted Nearest Neighbor Algorithm for Learning with Symbolic Features, Machine Learning, Vol. 10, pp.57-78, (1993).
13. [Cover 67] T.M. Cover & P.E. Hart: Nearest Neighbor Pattern classification, IEEE Transactions on Information Theory, Vol. 13, No.1, pp.21-27, (1967).
14. [Cover 91] T.M. Cover & J.A. Thomas: Elements of Information Theory, New York: Wiley, 1991.
15. [Creey 92] R.H. Creey, B.M. Masand, S.J. Smith & D.L. Waltz: Trading MIPS and Memory for Knowledge Engineering, Communications of the ACM, Vol. 35, pp.48-64, (1992).
16. [Dasarathy 91] B.V. Dasarathy: Nearest Neighbor Norms, NN Pattern Classification Techniques, IEEE Computer Society Press, 1991.
17. [Devijver 82] P-.A. Devijver & J. Kittler: Pattern Recognition: A Statistical Approach. Englewood Cliffs, NJ: Prentice-Hall, 1982.
18. [Dudani 76] Dudani & A. Sahibsingh: The Distance Weighted k-Nearest Neighbor Rule, IEEE Transactions on Systems, Man and Cybernetics, Vol.6, No. 4, pp.325-327, (1976).
19. [Hall 98] M. Hall: Correlation-based Feature Selection for Machine Learning, PhD Thesis, Waikato Univer-sity, 1998.
20. [Kelly 9la] J.D. Kelly & L. Davis: A Hybrid Genetic Algorithm for Classification, in: proceeding of the 12th International joint Conference on Artificial Intelligence, 1991, pp.645-650.
21. [Kelly 91b] J.D. Kelly & L. Davis: Hybridizing the Genetic Algorithm and the K Nearest Neighbors Classification Algorithm, Proc. 4th International Conference on Genetic Algorithms, pp.377-383, (1998).
22. [Kohavi 97a] R. Kohavi, D. Sommerfield & J. Dougherty: Data mining using MLC++: A machine learning library in C ++, International Journal of Artificial Intelligence Tools, Vol.6, No.4, pp.537-566, (1997).
23. [Kohavi 97b] R. Kohavi, P. Langley & Y. Yun: The Utility of Feature Weighting in Nearest Neighbor Algorithms, ECML-97 (poster).
24. [Lee 99] K.C. Lee: A Technique of dynamic Feature Selection Using the Feature Group Mutual Information, Third Pacific Asia Conference, PAKDD-99, LNAI 1574, 1999, pp. 138-142.
25. [Liu 98] H. Liu & H. Motoda: Feature Selection For Knowledge Discovery and Data Mining, Kluwer Academic Publishers, 1998.
26. [Merz 98] C.J. Merz & P.M. Murphy: UCI Repository of Machine Learning Databases, Irvine, CA: University of California Irvine, Department of Information and Computer Science, Internet: http://www.ics.uci.edu/~mlearn/MLRepos itory.html, (1998).
27. [Okamoto 94] S. Okamoto & K. Satoh: An Average Predictive Accuracy of the Nearest Neighbor Classifier, Proceedings of Second European Workshop on Case-Based Reasoning (EWCBR'94), 1994, pp.101-112.
28. [Okamoto 95] S. Okamoto & K. Satoh: An Average-Case Analysis of k-Nearest Neighbor Classifier, Proceedings of the First International Conference on Case-Based Reasoning (ICCBR'95), 1995, pp.253-264.
29. [Okamoto 96] S. Okamoto & N. Yugami: Theoretical Analysis of the Nearest Neighbor Classifier in Noise Domains, Proceedings of the 13th International Conference on Machine Learning (ICML'96), 1996, pp.355-363.
30. [Okamoto 97] S. Okamoto & N. Yugami: An Average-Case Analysis of k-Nearest Neighbor Classifier for Noise Domains, Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI97), 1997, pp.238-243.
31. [Pawlak 91] Z. Pawlak: ROUGH SETS (Theoretical Aspects of Reasoning about Data), Kluwer, 1991.
32. [Quinlan 86] J.R. Quinlan: Induction of Decision Trees. Machine Learning, Vol. 1 No. 1, pp.81-106, (1986).
33. [Quinlan 93] J.R. Quinlan: C4.5: Programs for Machine Learning, Morgan Kaufmann, 1993.
34. [Rachlin 94] J. Rachlin, S. Kasif, S. Salzberg & D.W. Aha: Towards a Better Understanding of Memory-Based and Bayesian Classifiers, In Proceedings of the Eleventh International Machine Learning Conference, New Brunswick, NJ: Morgan Kaufmann, 1994, pp.242-250.
35. [Salzberg 90] S. Salzberg: Learning with Nested Exemplars, Kluwer Academic Publishers, Norwell, MA, 1990.
36. [Salzberg 91] S. Salzberg: A Nearest Hyperectangle Learning Method, Machine Learning, Vol. 6, pp.251-276, (1991).
37. [Stanfillm 86] CC. Stanfill & D. Waltz, Toward Memory-Based Reasoning, Communications of the ACM, Vol. 29, pp.1213-1228, (1986).
38. [Skalak 94] D. Skalak: Prototype and feature selection by sampling and random mutation hill climbing algorithms, In Proceedings of the Eleventh International Machine Learning Conference, 1994, pp.293-301.
39. [Watson 94] I. Watson & Mark: Case-Based Reasoning: A Review, The Knowledge Engineering Review, Vol. 9 No. 4, pp.355-381, (1994).
40. [Wilson 00a] D.R. Wilson & T.R. Martinez: An Integrated Instance-based Learning Algorithm, Computational Intelligence, Vol. 16 No. 1, pp.1-28, (2000).
41. [Wilson 00b] D.R. Wilson & T.R. Martinez: Reduction Techniques for Instance-Based Learning Algorithms, Machine Learning, Vol. 38 No. 3, pp.257-28, (2000).
42. [Wettschereck 94a] D. Wettschereck, A Study of Distance-Based Machine learning Algorithms, PhD Thesis, Oregon State University, 1994.
43. [Wettschereck 95a] D. Wettschereck & D.W. Aha: Weighting Features, in proceedings of the First International Conference on Case-Based reasoning (ICCBR-95).
44. [Wettschereck 95b] D. Wettschereck & T.G.Dietterich: An experimental Comparison of the Nearest Neighbor and Nearest Hyperrectangle Algorithm, Machine learning,Vol. 19, pp. 19-28, (1995).
45. [Zhang 92] J.P. Zhang: Selecting Typical Instances in Instance-Based Learning, Proceedings of the Ninth International Conference on Machine Learning, 1992, pp.470-479.