Overfitting in making comparisons between variable selection methods

Journal of Machine Learning Research

Volumn 3, Issue , 2003, Pages 1371-1382

  Reunanen, Juha ^a  

^a ABB OY   (Finland)

Author keywords

Algorithm comparison; Cross validation; Knearest neighbors; Overfitting; Variable selection

Indexed keywords

ALGORITHM COMPARISON; CROSS VALIDATION; K-NEAREST NEIGHBORS; OVERFITTING; VARIABLE SELECTION;

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

LEARNING ALGORITHMS;

EID: 84890445089     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (20)

1. Jennifer Healey and Rosalind Picard. Smartcar: Detecting driver stress. In Proc. of the 15th Int. Conf. on Pattern Recognition (ICPR'2000), pages 218-221, Barcelona, Spain, 2000.
2. Anil K. Jain, Robert P.W. Duin, and Jianchang Mao. Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. Mach. Intell., 22(1):4-37, 2000.
3. Anil K. Jain and Douglas Zongker. Feature selection: Evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Mach. Intell., 19(2):153-158, 1997.
4. George H. John, Ron Kohavi, and Karl Pfleger. Irrelevant features and the subset selection problem. In Proc. of the 11th Int. Conf. on Machine Learning (ICML-94), pages 121-129, New Brunswick, NJ, USA, 1994.
5. Josef Kittler. Feature set search algorithms. In C. H. Chen, editor, Pattern Recognition and Signal Processing, pages 41-60. Sijthoff & Noordhoff, 1978.
6. Ron Kohavi and George John. Wrappers for feature subset selection. Artificial Intelligence, 97 (1-2):273-324, 1997.
7. Ron Kohavi and Dan Sommerfield. Feature subset selection using the wrapper model: Overfitting and dynamic search space topology. In Proc. of the 1st Int. Conf. on Knowledge Discovery and Data Mining (KDD-95), pages 192-197, Montreal, Canada, 1995.
8. Mineichi Kudo and Jack Sklansky. Comparison of algorithms that select features for pattern classifiers. Pattern Recognition, 33(1):25-41, 2000.
9. T. Marill and D. M. Green. On the effectiveness of receptors in recognition systems. IEEE Trans. Information Theory, 9(1):11-17, 1963.
10. Pavel Pudil, Jana Pavel Pudil, Jana Novovičová, and Josef Kittler. Floating search methods in feature selection. Pattern Recognition Letters, 15(11):1119-1125, 1994., and Josef Kittler. Floating search methods in feature selection. Pattern Recognition Letters, 15(11):1119-1125, 1994.
11. J. Ross Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
12. Robert J. Schalkoff. Pattern Recognition: Statistical, Structural and Neural Approaches. John Wiley & Sons, Inc., 1992.
13. Tobias Scheffer and Ralf Herbrich. Unbiased assessment of learning algorithms. In Proc. of the 15th Int. Joint Conf. on Artificial Intell. (IJCAI 97), pages 798-803, Nagoya, Japan, 1997.
14. Wojciech Siedlecki and Jack Sklansky. On automatic feature selection. Int. J. Pattern Recognition and Artificial Intell., 2(2):197-220, 1988.
15. Petr Somol and Pavel Pudil. Oscillating search algorithms for feature selection. In Proc. of the 15th Int. Conf. on Pattern Recognition (ICPR'2000), pages 406-409. IEEE Computer Society, 2000.
16. Petr Somol, Pavel Pudil, Jana Novovičová, and Pavel Paclík. Adaptive floating search methods in feature selection. Pattern Recognition Letters, 20(11-13):1157-1163, 1999.
17. Stephen D. Stearns. On selecting features for pattern classifiers. In Proc. of the 3rd Int. Joint Conf. on Pattern Recognition, pages 71-75, Coronado, CA, USA, 1976.
18. Elias Vyzas and Rosalind Picard. Offline and online recognition of emotion expression from physiological data. In Emotion-Based Agent Architectures Workshop (EBAA'99) at 3rd Int. Conf. on Autonomous Agents, pages 135-142, Seattle, WA, USA, 1999.
19. A.W.Whitney. A direct method of nonparametric measurement selection. IEEE Trans. Computers, 20(9):1100-1103, 1971.
20. Klaus Wiltschi, Alex Pinz, and Tony Lindeberg. An automatic assessment scheme for steel quality inspection. Machine Vision and Applications, 12(3):113-128, 2000.