Feature selection in MLPs and SVMs based on maximum output information

IEEE Transactions on Neural Networks

Volumn 15, Issue 4, 2004, Pages 937-948

  Sindhwani, Vikas ^a   Rakshit, Subrata ^b   Deodhare, Dipti ^b   Erdogmus, Deniz ^c   Principe, Jose C ^c   Niyogi, Partha ^a  

^a THE UNIVERSITY OF CHICAGO   (United States)

^b and Virtual Reality Groups   (India)

^c UNIVERSITY OF FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BACKPROPAGATION; FEATURE EXTRACTION; INFORMATION THEORY; LEARNING SYSTEMS; MULTILAYER NEURAL NETWORKS; PROBABILITY DENSITY FUNCTION;

FEATURE SELECTION; MUTUAL INFORMATION; SUPPORT VECTOR MACHINES;

LEARNING ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; BREAST TUMOR; CLASSIFICATION; CLUSTER ANALYSIS; COMPARATIVE STUDY; COMPUTER ANALYSIS; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; EVALUATION; HUMAN; INFORMATION RETRIEVAL; INFORMATION SCIENCE; LEARNING; METHODOLOGY; STATISTICAL MODEL; VALIDATION STUDY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BREAST NEOPLASMS; CLUSTER ANALYSIS; COMPUTER SIMULATION; COMPUTING METHODOLOGIES; DECISION SUPPORT TECHNIQUES; HUMANS; INFORMATION STORAGE AND RETRIEVAL; INFORMATION THEORY; LIKELIHOOD FUNCTIONS; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; PROBABILITY LEARNING;

EID: 3843072084     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2004.828772     Document Type: Article

References (28)

1. R. Battiti, "Using mutual information for selecting features in supervised neural net training," IEEE Trans. Neural Networks, vol. 5, pp. 537-550, July 1994.
2. K. D. Bollacker and J. Ghosh, "Linear feature extractors based on mutual information," in Proc. ICNN 1996, vol. 3, Washington DC, June 1996, pp. 1528-1533.
3. J. S. Bridle, "Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters," in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kauffman, 1990, vol. 2, pp. 211-217.
4. M. Dash and H. Liu, "Feature selection for classification," Intell. Data Anal., vol. 1, pp. 131-156, 1997.
5. L. Devroye, L. Gyorfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
6. D. Erdogmus and J. Principe, "Lower and upper bounds for misclassification probability based on Renyi's information," J. VLSI Signal Process. Syst. (Special Issue on Wireless Communications and Blind Signal Processing), 2003.
7. R. M. Fano, Transmission of Information: A Statistical Theory of Communications. New York: Wiley, 1961.
8. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, "Gene selection for cancer classification using su ort vector machines," Mach. Learn., 2000.
9. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn.g Res. (Special Issue on Variable and Feature Selection), 2003.
10. G. E. Hinton, "Connectionist learning procedures," Artif. Intell., vol. 40, no. 1, pp. 143-150, 1989.
11. C. N. Hsu, H. J. Huang, and D. Schuschel, "The ANNIGMA-wrapper approach to fast feature selection for neural nets," IEEE Trans. Syst., Man, Cybern. B, vol. 32, pp. 207-212, Apr. 2002.
12. G. H. John, R. Kohavi, and K. Pfleger, "Irrelevant features and the subset selection problem," in Proc. 11th Int. Conf. Machine Learning, San Mateo, CA, 1994, pp. 121-129.
13. K. Kira and L. Rendell, "The feature selection problem: traditional methods and a new algorithm," in Proc. 10th Nat. Conf. Artificial Intelligence (AAAI-92), Menlo Park, CA, 1992, pp. 129-134.
14. R. Linsker, "Self-organization in a perceptual network," Comput. Mag., vol. 21, pp. 105-117, 1988.
15. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann, 1993.
16. A. Renyi, Probability Theory. New York: Elsevier, 1970.
17. http://www.daviddlewis.com/resources/testcollections/reuters21578 [Online]
18. D. E. Rumelhart and J. L. McClelland, Eds., Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press, 1986, vol. 1.
19. B. Sch″olkopf and A. J. Smola, Learning With Kernels. Cambridge, MA: MIT Press, 2002.
20. R. Setiono and H. Liu, "Neural network feature selector," IEEE Trans. Neural Networks, vol. 8, pp. 645-662, May 1997.
21. C. E. Shannon and W. Weaver, The Mathematical Theory of Communications. Urbana, IL: Univ. Illinois Press, 1949.
22. V. Sindhwani, P. Bhattacharyya, and S. Rakshit, "Information theoretic feature crediting in multiclass support vector machines," in Proc 1st SIAM Int. Conf. Data Mining, 2001.
23. K. Torkkola and W. M. Campbell, "Mutual information in learning feature transformations," in Proc. 17th Int. Conf. Machine Learning, San Francisco, CA, 2000, pp. 1015-1022.
24. J. Weston, A. Elisseeff, M. Tipping, and B. Scholkopf, "Use of the zero norm with linear models and kernel methods," JMLR Special Issue on Variable and Feature Selection, 2002.
25. A. White and W. Z. Liu, "Bias in information-based measures in decision tree induction," Mach. Learn., vol. 15, pp. 321-329, 1994.
26. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
27. H. Yang and J. Moody, "Data visualization and feature selection: new algorithms for nongaussian data," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2000, vol. 12, pp. 687-693.
28. Y. Yang and J. O. Pedersen, "A comparative study on feature selection in text categorization," in Proc. of ICML'97, 1997, pp.