Fast binary feature selection with conditional mutual information

Journal of Machine Learning Research

Volumn 5, Issue , 2004, Pages 1531-1555

  Fleuret, Francois ^a  

^a EPFL   (Switzerland)

Author keywords

Classification; Fast learning; Feature selection; Information theory; Mutual information; Naive Bayes

Indexed keywords

FEATURE EXTRACTION; INFORMATION THEORY;

CONDITIONAL MUTUAL INFORMATION; FAST FEATURE SELECTIONS; FAST LEARNING; FEATURE SELECTION METHODS; MUTUAL INFORMATIONS; NAIVE BAYES; NAIVE BAYESIAN CLASSIFIER; STATE-OF-THE-ART METHODS;

CLASSIFICATION (OF INFORMATION);

EID: 33645690579     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (34)

1. Y. Amit, D. Geman, and K. Wilder. Joint induction of shape features and tree classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(11):1300-1305, November 1997.
2. R. Battiti. Using mutual information for selecting features in supervised neural net learning. In IEEE Transactions on Neural Networks, volume 5, 1994.
3. B. V. Bonnlander and A. S. Weigend. Selecting input variables using mutual information and nonparametric density estimation. In Proceedings of ISANN, 1996.
4. B. E. Boser, I. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers. in Proceedings of the Fifth Annual Workshop on Computational Learning Theory, 5:144-152, 1992.
5. L. Breiman. Bagging predictors. Machine Learning, 24(2):123-140, 1996.
6. L. Breiman. Random forests - random features. Technical Report 567, Department of Statistics, University of California, Berkeley, 1999.
7. L. Breiman. Some infinity theory for predictors ensembles. Technical Report 579, Department of Statistics, University of California, Berkeley, 2000.
8. N. Christiani and J. Shawe-Taylor. An Introduction to Support Vector Machines and other kernelbased learning methods. Cambridge University Press, 2000.
9. T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley-Interscience, 1991.
10. S. Das. Filters, wrappers and a boosting-based hybrid for feature selection. In Proceedings of the International Conference on Machine Learning 2001, pages 74-81, 2001.
11. R. Duda and P. Hart. Pattern classification and scene analysis. John Wiley & Sons, 1973.
12. F. Fleuret. Binary feature selection with conditional mutual information. Technical Report RR-4941, INRIA, October 2003.
13. F. Fleuret and D. Geman. Coarse-to-fine visual selection. International Journal of Computer Vision, 41(1/2):85-107, 2001.
14. F. Fleuret and D. Geman. Fast face detection with precise pose estimation. In Proceedings of ICPR2002, volume 1, pages 235-238, 2002.
15. Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. InProc. 13th International Conference on Machine Learning, pages 148-156. Morgan Kaufmann, 1996a.
16. Y. Freund and R. E. Schapire. Game theory, on-line prediction and boosting. In Proc. 9th Annu. Conf. on Comput. Learning Theory, pages 325-332. ACM Press, New York, NY, 1996b.
17. J. H. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annal of Statistics, 28:337-407, 2000.
18. T. S. Furey, N. Cristianini, N. Duffy, D. W. Bednarski, M. Schummer, and D. Haussler. Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics, 16(10), 2000.
19. I. Guyon and A. Elisseeff An introduction to variable and feature selection. Journal of Machine Learning Research, 3:1157-1182, 2003.
20. I. Guyon, S. Gunn, S. Ben Hur, and G. Dror. Result analysis of the NIPS2003 feature selection challenge. In Proceedings of the NIPS2004 conference, 2004.
21. T. Hastie and R. Tibshirani. Classification by pairwise coupling. In Michael I. Jordan, Michael J. Kearns, and Sara A. Solla, editors, Advances in Neural Information Processing Systems, volume 10. The MIT Press, 1998.
22. R. Kohavi and G. John. Wrappers for feature subset selection. Artificial Intelligence, pages 273-324, 1997.
23. D. Koller and M. Sahami. Toward optimal feature selection. In Proceedings of the 13th International Conference on Machine Learning 1996, pages 284-292, 1996.
24. P. Langley, W. Iba, and K. Thompson. An analysis of bayesian classifiers. In Proceedings of AAAI-92, pages 223-228, 1992.
25. L. Mason, J. Baxter, P. Bartlett, and M. Frean. Boosting algorithms as gradient descent. In Neural Information Processing Systems, pages 512-518. MIT Press, 2000.
26. K. Miyahara and M. J. Pazzani. Collaborative filtering with the simple bayesian classifier. In Pacific Rim International Conference on Artificial Intelligence, pages 679-689, 2000.
27. A. Novikoff On convergence proofs for perceptrons. In In Symposium on Mathematical Theory of Automata, pages 615-622, 1962.
28. C. A. Ratanamahatana and D. Gunopulos. Feature selection for the naive bayesian classifier using decision trees. Applied Artificial Intelligence, 17(5-6):475-487, 2003.
29. G. Ratsch, T. Onoda, and K.-R. Muller. Regularizing AdaBoost. In Proceedings of NIPS, pages 564-570, 1998.
30. F. Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65:386-408, 1958.
31. K. Torkkola. Feature extraction by non-parametric mutual information maximization. Journal of Machine Learning Research, 3:1415-1438, 2003.
32. V. N. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, 1998.
33. M. Vidal-Naquet and S. Ullman. Object recognition with informative features and linear classification. In Proceedings of International Conference on Computer Vision 2003, pages 281-288, 2003.
34. L. Yu and H. Liu. Feature selection for high-dimensional data: A fast correlation-based filter solution. In Proceedings of the International Conference on Machine Learning 2003, pages 856-863, 2003.