Performing feature selection with multilayer perceptrons

IEEE Transactions on Neural Networks

Volumn 19, Issue 3, 2008, Pages 431-441

  Romero, Enrique ^a   Sopena, Josep María ^b  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

^b UNIVERSITY OF BARCELONA   (Spain)

Author keywords

Experimental work; Feature selection (FS); Multilayer perceptrons; Wrapper approach

Indexed keywords

COMPUTATIONAL METHODS; DECISION THEORY; FEATURE EXTRACTION;

SEQUENTIAL BACKWARD SELECTION; STOPPING CRITERION; WRAPPER APPROACH;

MULTILAYER NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; DATA BASE; HUMAN; INFORMATION RETRIEVAL;

DATABASE MANAGEMENT SYSTEMS; HUMANS; INFORMATION STORAGE AND RETRIEVAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 40949143180     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.909535     Document Type: Article

References (48)

1. B. Baesens, S. Viaene, J. Vanthienen, and G. Dedene, "Wrapped feature selection by means of guided neural network optimisation," in Proc. Int. Conf. Pattern Recognit., 2000, vol. 2, pp. 113-116.
2. C. M. Bishop, Neural Networks for Pattern Recognition. New York: Oxford Univ. Press, 1995.
3. C. L. Blake and C. J. Merz, "UCI repository of machine learning databases," Dept. Inf. Comput. Sci., Univ. California, Irvine, CA, 1998 [Online]. Available: http://www.ics.uci.edu/mlearn/MLRepository.html
4. A. L. Blum and P. Langley, "Selection of relevant features and examples in machine learning," Artif. Intell., vol. 97, no. 1-2, pp. 245-271, 1997.
5. S. Chen, S. A. Billings, and W. Luo, "Orthogonal least squares methods and their applications to non-linear system identification," Int. J. Control, vol. 50, no. 5, pp. 1873-1896, 1989.
6. T. Cibas, F. F. Soulié, P. Gallinari, and Š. Raudys, "Variable selection with optimal cell damage," in Proc. Int. Conf. Artif. Neural Netw., 1994, vol. 1, pp. 727-730.
7. T. Cibas, F. F. Soulié, P. Gallinari, and Š. Raudys, "Variable selection with neural networks," Neurocomputing, vol. 12, no. 2-3, pp. 223-248, 1996.
8. P. Domingos, "A unified bias-variance decomposition and its applications," in Proc. 17th Int. Conf. Mach. Learn., 2000, pp. 231-238.
9. M. Egmont-Petersen, W. R. M. Dassen, and J. H. C. Reiber, "Sequential selection of discrete features for neural networks - A Bayesian approach to building a cascade," Pattern Recognit. Lett., vol. 20, no. 11-13, pp. 1439-1448, 1999.
10. M. Egmont-Petersen, J. L. Talmon, A. Hasman, and A. W. Ambergen, "Assessing the importance of features for multi-layer perceptrons," Neural Netw., vol. 11, no. 4, pp. 623-635, 1998.
11. A. P. Engelbrecht, "A new pruning heuristic based on variance analysis of sensitivity information," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1386-1399, Nov. 2001.
12. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias/variance dilemma," Neural Comput., vol. 4, no. 1, pp. 1-58, 1992.
13. Y. Grandvalet, "Anisotropic noise injection for input variables relevance determination," IEEE Trans. Neural Netw., vol. 11, no. 6, pp. 1201-1212, Nov. 2000.
14. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol. 3, pp. 1157-1182, 2003.
15. I. Guyon, S. Gunn, A. Ben-Hur, and G. Dror, "Result analysis of the NIPS 2003 feature selection challenge," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2005, vol. 17, pp. 545-552.
16. B. Hassibi and D. G. Stork, "Second order derivatives for network pruning: Optimal brain surgeon," in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 1993, vol. 5, pp. 164-171.
17. G. H. John, R. Kohavi, and K. Pfleger, "Irrelevant features and the subset selection problem," in Proc. 11th Int. Conf. Mach. Learn., 1994, pp. 121-129.
18. M. Kantrowitz, "CMU artificial intelligence repository," Schl. Comput. Sci., Univ. Carnegie Mellon, Pittsburgh, PA, 1993 [Online]. Available: http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/ai-repository/ ai/areas/n% eural/bench/cmu
19. J. Kittler, "Feature set search algorithms," in Pattern Recognition and Signal Processing, C. H. Chen, Ed. Alphen ann den Rijn, The Netherlands: Sijthoff & Noordhoff, 1978, pp. 41-60.
20. N. Kwak and C. H. Choi, "Input feature selection for classification problems," IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 143-159, Jan. 2002.
21. H. C. Lac and D. A. Stacey, "Feature subset selection via multi-objective genetic algorithm," in Proc. Int. Joint Conf. Neural Netw., 2005, vol. 3, pp. 1349-1354.
22. Y. Le Cun, J. S. Denker, and S. A. Solla, "Optimal brain damage," in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 1990, vol. 2, pp. 598-605.
23. J. Li, M. T. Manry, P. L. Narasimha, and C. Yu, "Feature selection using a piecewise linear network," IEEE Trans. Neural Netw., vol. 17, no. 5, pp. 1101-1115, Sep. 2006.
24. H. Liu and H. Motoda, Feature Selection for Knowledge Discovery and Data Mining. Norwell, MA: Kluwer, 1998.
25. J. Mao, K. Mohiuddin, and A. K. Jain, "Parsimonious network design and feature selection through node pruning," in Proc. Int. Conf. Pattern Recognit., 1994, vol. 2, pp. 622-624.
26. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Machine Learning, Neural and Statistical Classification. Chichester, U.K.: Ellin Horwood, 1994 [Online]. Available: http://www.amsta.leeds.ac.uk/~charles/ statlog
27. J. Moody and J. Utans, "Principled architecture selection for neural networks: Application to corporate bond rating prediction," in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 1992, vol. 4, pp. 683-690.
28. M. C. Mozer and P. Smolensky, "Skeletonization: A technique for trimming the fat from a network via relevance assessment," in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 1989, vol. 1, pp. 107-115.
29. V. Onnia, M. Tico, and J. Saarinen, "Feature selection method using neural network," in Proc. Int. Conf. Image Process., 2001, vol. 1, pp. 513-516.
30. D. Opitz and R. Maclin, "Popular ensemble methods: An empirical study," J. Artif. Intell. Res., vol. 11, pp. 169-198, 1999.
31. K. L. Priddy, S. E. Rogers, D. W. Ruck, and G. L. Tarr, "Bayesian selection of important features for feedforward neural networks," Neurocomputing, vol. 5, no. 2-3, pp. 91-103, 1993.
32. P. Pudil, J. Novovičová, and J. Kittler, "Floating search methods in feature selection," Pattern Recognit. Lett., vol. 15, no. 11, pp. 1119-1125, 1994.
33. R. Reed, "Pruning algorithms - A survey," IEEE Trans. Neural Networks, vol. 4, no. 5, pp. 740-747, Sep. 1993.
34. B. D. Ripley, "Statistical ideas for selecting network architectures," in Neural Networks: Artificial Intelligence and Industrial Applications, B. Kappen and S. Gielen, Eds. London, U.K.: Springer-Verlag, 1995, pp. 183-190.
35. E. Romero, J. M. Sopena, G. Navarrete, and R. Alquézar, "Feature selection forcing overtraining may help to improve performance," in Proc. Int. Joint Conf. Neural Netw., 2003, vol. 3, pp. 2181-2186.
36. D. W. Ruck, S. K. Rogers, and M. Kabrisky, "Feature selection using a multilayer perceptron," J. Neural Netw. Comput., vol. 2, no. 2, pp. 40-48, 1990.
37. R. Setiono and H. Liu, "Neural-network feature selector," IEEE Trans. Neural Netw., vol. 8, no. 3, pp. 654-662, May 1997.
38. W. Siedlecki and J. Sklansky, "On automatic feature selection," Int. J. Pattern Recognit. Artif. Intell., vol. 2, no. 2, pp. 197-220, 1988.
39. J. M. Sopena, E. Romero, and R. Alquézar, "Neural networks with periodic and monotonic activation functions: A comparative study in classification problems," in Proc. 9th Int. Conf. Artif. Neural Netw., 1999, vol. 1, pp. 323-328.
40. A. Stahlberger and M. Riedmiller, "Fast network pruning and feature extraction using the Unit-OBS algorithm," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1997, vol. 9, pp. 655-661.
41. J. M. Steppe and K. W. Bauer, "Improved feature screening in feedforward neural networks," Neurocomputing, vol. 13, no. 1, pp. 47-58, 1996.
42. J. M. Steppe, K. W. Bauer, and S. K. Rogers, "Integrated feature and architecture selection," IEEE Trans. Neural Netw., vol. 7, no. 4, pp. 1007-1013, Jul. 1996.
43. G. Taguchi, Taguchi on Robust Technology Development: Bringing Quality Engineering Upstream. New York: ASME, 1993.
44. P. Van de Laar, T. Heskes, and S. Gielen, "Partial retraining: A new approach to input relevance determination," Int. J. Neural Syst., vol. 9, no. 1, pp. 75-85, 1999.
45. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
46. A. Verikas and M. Bacauskiene, "Feature selection with neural networks," Pattern Recognit. Lett., vol. 23, no. 11, pp. 1323-1335, 2002.
47. P. Zhang, B. Verma, and K. Kumar, "A neural-genetic algorithm for feature selection and breast abnormality classification in digital mammography," in Proc. Int. Joint Conf. Neural Netw., 2004, vol. 3, pp. 2303-2308.
48. J. M. Zurada, A. Malinowski, and S. Usui, "Perturbation method for deleting redundant inputs of perceptron networks," Neurocomputing vol. 14, no. 2, pp. 177-193, 1997.