Feature selection for MLP neural network: The use of random permutation of probabilistic outputs

IEEE Transactions on Neural Networks

Volumn 20, Issue 12, 2009, Pages 1911-1922

  Yang, Jian Bo ^a   Shen, Kai Quan ^a   Ong, Chong Jin ^a   Li, Xiao Ping ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Feature ranking; Feature selection; Multilayer perceptrons (MLPs); Probabilistic outputs; Random permutation

Indexed keywords

FEATURE RANKING; FEATURE SELECTION; MULTI-LAYER PERCEPTRONS; PROBABILISTIC OUTPUT; RANDOM PERMUTATIONS;

CYBERNETICS; ELECTRIC LOADS; FEATURE EXTRACTION; IMAGE SEGMENTATION; MULTILAYERS; NUMERICAL METHODS; PATTERN RECOGNITION SYSTEMS;

MULTILAYER NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; DATA BASE; DECISION SUPPORT SYSTEM; HUMAN; PERCEPTION; PHYSIOLOGY; PROBABILITY;

ALGORITHMS; COMPUTER SIMULATION; DATABASE MANAGEMENT SYSTEMS; DECISION SUPPORT TECHNIQUES; HUMANS; NEURAL NETWORKS (COMPUTER); PERCEPTION; PROBABILITY;

EID: 72149094052     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2032543     Document Type: Article

References (27)

1. A. Blum and P. Langley, "Selection of relevant features and examples in machine learning," Artif. Intell., vol.97, no.1-2, pp. 245-271, 1997.
2. V. N. Vapnik, Statistical Learning Theory. New York: Wiley-Interscience, 1998.
3. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol.3, pp. 1157-1182, 2003.
4. R. Kohavi and G. H. John, "Wrappers for feature selection," Artif. Intell., vol.97, no.1-2, pp. 273-324, 1997.
5. G. H. John, R. Kohavi, and K. Pfleger, "Irrelevant features and subset selection problem," in Proc. Int. Conf. Mach. Learn., San Mateo, CA, 1994, pp. 121-129.
6. K.-Q. Shen, C.-J. Ong, X.-P. Li, and E. P. Wilder-Smith, "Feature selection via sensitivity analysis of SVM probabilistic outputs," Mach. Learn., vol.70, no.1, pp. 1-20, 2008.
7. J. S. Bridle, "Probabilistic interpretation of feedfoward classification network outputs with relationships to statistical pattern recognition," in Neurocomputing: Algorithms, Architectures and Applications. New York: Springer-Verlag, 1989, pp. 227-236.
8. G. Cybenko, "Continuous valued neural networks with two hidden layers are sufficient," Dept. Comput. Sci., Tufts Univ., Medford, MA, Tech. Rep., 1988.
9. K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural Netw., vol.4, no.2, pp. 251-257, 1991.
10. F. H. Long, H. C. Peng, and C. Ding, "Feature selection based on mutual information: Criteria of max-dependency ,max-relevance, and min-redundancy," IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.8, pp. 1226-1238, Aug. 2005.
11. R. Battiti, "Using mutual information for selection features in supervised neural net learning," IEEE Trans. Neural Netw., vol.5, no.4, pp. 537-550, Jul. 1994.
12. N. Kwak and C. H. Choi, "Input feature selection for classification problems," IEEE Trans. Neural Netw., vol.13, no.1, pp. 143-159, Jan. 2002.
13. T. W. Chow and D. Huang, "Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information," IEEE Trans. Neural Netw., vol.16, no.1, pp. 1045-9227, Jan. 2005.
14. V. Sindhwani, S. Rakshit, D. Deodhare, D. Erdogmus, J. Principe, and P. Niyogi, "Feature selection in MLPs and SVMs based on maximum output information," IEEE Trans. Neural Netw., vol.15, no.4, pp. 937-948, Jul. 2004.
15. R. Setiono and H. Liu, "Neural-network feature selector," IEEE Trans. Neural Netw., vol.8, no.1, pp. 29-44, Jan. 1997.
16. C.-N. Hsu, H.-J. Huang, and S. Dietrich, "The annigma-wrapper approach to fast feature selection for neural nets," IEEE Trans. Syst. Man Cybern. B, Cybern., vol.32, no.2, pp. 207-212, Apr. 2002.
17. L. Breiman, "Random forests," Mach. Learn., vol.45, no.1, pp. 5-32, Oct. 2001.
18. E. S. Page, "A note on generating random permutations," Appl. Statist., vol.16, no.3, pp. 273-274, 1967.
19. M. Fodslette, "A scaled conjugate gradient algorithm for fast supervised learning," Neural Netw., vol.6, no.4, pp. 525-533, 1993.
20. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, "Gene selection for cancer classification using support vector machines," Mach. Learn., vol.46, no.1-3, pp. 389-422, 2002.
21. A. Asuncion and D. J. Newman, UCI Machine Learning Repository, Univ. California, Irvine, CA, 2007 [Online]. Available: http://www.ics. uci.edu/~mlearn/MLRepository.html
22. A. Rakotomamonjy, "Variable selection using SVM-based criteria," J. Mach. Learn. Res., vol.3, pp. 1357-11320, 2003.
23. I. Nabney and C. Bishop, Netlab Neural Network Software, [Online]. Available: http://www.ncrg.aston.ac.uk/netlab/
24. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik, "Feature selection for SVMs," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2000, pp. 668-674.
25. L. Yu and H. Liu, "Efficient feature selection via analysis of relevance and redundancy," J. Mach. Learn. Res., vol.5, pp. 1205-1224, 2004.
26. D. Clark, Z. Schreter, and A. Adams, "A quantitative comparison of dystal and backpropagation," in Proc. Australian Conf. Neural Netw., 1996, pp. 132-137.
27. G. Rätsch, Benchmark Repository, 2005 [Online]. Available: http://ida. first.fhg.de/projects/bench/benchmarks.htm