Feature selection using a neural framework with controlled redundancy

IEEE Transactions on Neural Networks and Learning Systems

Volumn 26, Issue 1, 2015, Pages 35-50

  Chakraborty, Rudrasis ^a   Pal, Nikhil R ^a  

^a INDIAN STATISTICAL INSTITUTE   (India)

Author keywords

Dimensionality reduction; feature selection; neural network; redundancy control.

Indexed keywords

DECISION MAKING; NEURAL NETWORKS; PICKUPS; PROBLEM SOLVING; REDUNDANCY;

CONNECTION WEIGHTS; DECISION-MAKING SYSTEMS; DIMENSIONALITY REDUCTION; FEATURE SELECTION METHODS; FUNCTION APPROXIMATION; MULTILAYER PERCEPTRON NEURAL NETWORKS; MUTUAL INFORMATIONS; REDUNDANCY CONTROL;

FEATURE EXTRACTION;

ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; HUMAN; INFORMATION PROCESSING; LEARNING; PERCEPTION;

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DATASETS AS TOPIC; DECISION SUPPORT TECHNIQUES; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER); PERCEPTION;

EID: 84919630495     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2014.2308902     Document Type: Article

References (63)

1. K. Fukunaga, Introduction to Statistical Pattern Recognition. New York, NY, USA: Academic, 1990.
2. P. A. Devijver and J. Kittler, Pattern Recognition, A Statistical Approach. London, U.K.: Prentice-Hall, 1982.
3. A. Jain and D. Zongker, "Feature selection: Evaluation, application, and small sample performance," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 2, pp. 153-158, Feb. 1997.
4. N. R. Pal and K. K. Chintalapudi, "A connectionist system for feature selection," Neural Parallel Sci. Comput., vol. 5, no. 3, pp. 359-381, 1997.
5. Y. Aksu, D. J. Miller, G. Kesidis, and Q. X. Yang, "Margin-maximizing feature elimination methods for linear and nonlinear kernel-based discriminant functions," IEEE Trans. Neural Netw., vol. 21, no. 5, pp. 701-717, May 2010.
6. C. Shen, H. Li, and M. J. Brooks, "Supervised dimensionality reduction via sequential semidefinite programming," Pattern Recognit., vol. 41, no. 12, pp. 3644-3652, 2008.
7. H. Liu and L. Yu, "Toward integrating feature selection algorithms for classification and clustering," IEEE Trans. Knowl. Data Eng., vol. 17, no. 4, pp. 491-502, Apr. 2005.
8. F. Nie, S. Xiang, Y. Jia, C. Zhang, and S. Yan, "Trace ratio criterion for feature selection," in Proc. 23rd AAAI Conf. Artif. Intell., Jul. 2008, pp. 671-676.
9. L. Wang, "Feature selection with kernel class separability," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 9, pp. 1534-1546, Sep. 2008.
10. M. Last, A. Kandel, and O. Maimon, "Information-theoretic algorithm for feature selection," Pattern Recognit. Lett., vol. 22, pp. 799-811, May 2001.
11. D. Chakraborty and N. R. Pal, "Selecting useful groups of features in a connectionist framework," IEEE Trans. Neural Netw., vol. 19, no. 3, pp. 381-396, Mar. 2008.
12. D. Chakraborty and N. R. Pal, "Integrated feature analysis and fuzzy rule based system identification in a neuro-fuzzy paradigm," IEEE Trans. Syst. Man, Cybern. B, Cybern., vol. 31, no. 3, pp. 391-400, Jun. 2001.
13. W. Siedlick and J. Sklansky, "A note on genetic algorithms for large-scale feature selection," Pattern Recognit. Lett., vol. 10, no. 5, pp. 335-347, 1989.
14. D. W. Ruck, S. K. Rogers, and M. Kabrisky, "Feature selection using a multilayer perceptron," J. Neural Netw., vol. 2, no. 2, pp. 40-48, 1990.
15. L. Zhou, L. Wang, and C. Shen, "Feature selection with redundancyconstrained class separability," IEEE Trans. Neural Netw., vol. 21, no. 5, pp. 853-858, May 2010.
16. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, "Gene selection for cancer classification using support vector machines," Mach. Learn., vol. 46, nos. 1-3, pp. 389-422, 2002.
17. X. Zhou and D. P. Tuck, "MSVM-RFE: Extensions of SVM-RFE for multiclass gene selection on DNA microarray data," Bioinformatics, vol. 23, no. 9, pp. 1106-1114, 2007.
18. W. Dinkelbach, "On nonlinear fractional programming," Manag. Sci., vol. 13, no. 7, pp. 492-498, 1967.
19. H. Peng, F. Long, and C. Ding, "Feature selection based on mutual information of criteria of max-dependency, max-relevance, and minredundancy," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1226-1238, Aug. 2005.
20. L. Yu and H. Liu, "Efficient feature selection via analysis of relevance and redundancy," J. Mach. Learn. Res., vol. 5, no. 10, pp. 1205-1224, 2004.
21. Y. Hong, S. Kwong, Y. Chang, and Q. Ren, "Unsupervised feature selection using clustering ensembles and population based incremental learning algorithm," Pattern Recognit., vol. 41, no. 9, pp. 2742-2756, 2008.
22. S. Chatterjee and A. S. Hadi, "Influential observations, high leverage points, and outliers in linear regression," Statist. Sci., vol. 1, no. 3, pp. 379-393, 1986.
23. K. Z. Mao, "Identifying critical variables of principal components for unsupervised feature selection," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 2, pp. 339-344, Apr. 2005.
24. J. G. Dy, C. E. Brodle, A. C. Kak, L. S. Broderick, and A. M. Aisen, "Unsupervised feature selection applied to content-based retrieval of lung images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 3, pp. 373-378, Mar. 2003.
25. P. A. Estévez, M. Tesmer, C. A. Perez, and J. M. Zurada, "Normalized mutual information feature selection," IEEE Trans. Neural Netw., vol. 20, no. 2, pp. 189-201, Feb. 2009.
26. G. Qu, S. Hariri, and M. Yousif, "A new dependency and correlation analysis for features," IEEE Trans. Knowl. Data Eng., vol. 17, no. 9, pp. 1199-1207, Sep. 2005.
27. R. Liu, N. Yang, X. Ding, and L. Ma, "An unsupervised feature selection algorithm: Laplacian score combined with distance-based entropy measure," in Proc. 3rd Int. Symp. Intell. Inf. Technol. Appl., Nov. 2009, pp. 65-68.
28. D. Koller and M. Sahami, "Toward optimal feature selection," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 284-292.
29. X. He, D. Cai, and P. Niyogi, "Laplacian score for feature selection," in Proc. NIPS, 2005, p. 189.
30. C. Boutsidis, M. W. Mahoney, and P. Drineas, "Unsupervised feature selection for the k-means clustering problem," in Proc. NIPS, 2009, pp. 153-161.
31. M. W. Mahoney and P. Drineas, "CUR matrix decompositions for improved data analysis," Proc. Nat. Acad. Sci., vol. 106, no. 3, pp. 697-702, 2009.
32. C. Ding and H. Peng, "Minimum redundancy feature selection from microarray gene expression data," in Proc. IEEE Bioinf. Conf., Aug. 2003, pp. 523-529.
33. K. Kira and L. A. Rendell, "A practical approach to feature selection," in Proc. 9th Int. Conf. Mach. Learn., 1992, pp. 249-256.
34. M. A. Hall, "Correlation-based feature selection for machine learning," Ph.D. dissertation, Dept. Comput. Sci., Univ. Waikato, Hamilton, New Zealand, 1999.
35. K. Fakunaga and W. L. G. Koontz, "Application of the Karhunen-Loeve expansion to feature selection and ordering," IEEE Trans. Comput., vol. 19, no. 4, pp. 311-318, Apr. 1970.
36. T. Matsui, Y. Saruwatari, and M. Shigeno, "An analysis of Dinkelbach's algorithm for 0-1 fractional programming problems," Dept. Math. Eng. Inf. Phys., Univ. Tokyo, Tokyo, Japan, Tech. Rep. METR92-14, 1992.
37. A. Schrijver, Theory of Linear and Integer Programming. New York, NY, USA: Wiley, 1986.
38. P. Kohavi and G. H. John, "Wrappers for feature subset selection," Artif. Intell., vol. 97, nos. 1-2, pp. 273-324, 1997.
39. D. P. Muni, N. R. Pal, and J. Das, "Genetic programming for simultaneous feature selection and classifier design," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 1, pp. 106-117, Feb. 2006.
40. N. R. Pal, "Fuzzy logic approaches to structure preserving dimensionality reduction," IEEE Trans. Fuzzy Syst., vol. 10, no. 3, pp. 277-286, Jun. 2002.
41. S. K. Pal, R. K. De, and J. Basak, "Unsupervised feature evaluation: A neuro-fuzzy approach," IEEE Trans. Neural Netw., vol. 11, no. 2, pp. 366-376, Mar. 2000.
42. N. R. Pal, "Soft computing for feature analysis," Fuzzy Sets Syst., vol. 103, no. 2, pp. 201-221, 1999.
43. D. P. Muni, N. R. Pal, and J. Das, "A novel approach for designing classifiers using genetic programming," IEEE Trans. Evol. Comput., vol. 8, no. 2, pp. 183-196, Apr. 2004.
44. S. K. Das, "Feature selector with a linear dependence measure," IEEE Trans. Comput., vol. 20, no. 9, pp. 1106-1109, Sep. 1971.
45. S. Nijima and Y. Okuno, "Laplacian linear discriminant analysis approach to unsupervised feature selection," IEEE/ACM Trans. Comput. Biol. Bioinf., vol. 6, no. 4, pp. 605-614, Dec. 2009.
46. N. R. Pal, A. Sharma, S. Sanadhya, and Karmeshu, "On identifying marker genes from gene expression data in a neural framework through online feature analysis," Int. J. Intell. Syst., vol. 21, no. 4, pp. 453-467, Apr. 2006.
47. C. D. Huang, C. T. Lin, and N. R. Pal, "Hierarchical learning architecture with automatic feature selection for multiclass protein fold classification," IEEE Trans. Nanobiosci., vol. 2, no. 4, pp. 221-232, Dec. 2003.
48. N. R. Pal, T. Cahoon, J. C. Bezdek, and K. Pal, "A new approach to target recognition for LADAR data," IEEE Trans. Fuzzy Syst., vol. 9, no. 1, pp. 44-52, Feb. 2001.
49. S. Kumar, Neural Networks: A Classroom Approach, 2nd ed. New York, NY, USA: McGraw-Hill, 2012.
50. S. S. Haykin, Neural Networks: A Comprehensive Foundation. Englewood Cliffs, NJ, USA: Prentice-Hall, 1999.
51. N. R. Pal and M. Malpani, "Redundancy-constrained feature selection with radial basis function networks," in Proc. IJCNN, Jun. 2012, pp. 1-8.
52. (2012). UCI Machine Learning Repository [online]. Available: http://archive.ics.uci.edu/ml/datasets.html
53. A. Tsanas, M. A. Little, P. E. McSharry, and L. O. Ramig, "Accurate telemonitoring of Parkinson's disease progression by noninvasive speech tests," IEEE Trans. Biomed. Eng., vol. 57, no. 4, pp. 884-893, Apr. 2010.
54. Q. Song, J. Ni, and G. Wang, "A fast clustering-based feature subset selection algorithm for high-dimensional data," IEEE Trans. Knowl. Data Eng., vol. 25, no. 1, pp. 1-14, Jan. 2013.
55. W. H. Press, B. P. Flannery, S. A. Teuklosky, and W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing. Cambridge, U.K.: Cambridge Univ. Press, 1988.
56. Y. Yang, H. T. Shen, Z. Ma, Z. Huang, and X. Zhou, "2,1-norm regularized discriminative feature selection for unsupervised learning," in Proc. 22nd Int. Joint Conf. Artif. Intell., vol. 2. 2011, pp. 1589-1594.
57. H. Lee, A. Battle, R. Raina, and A. Y. Ng, "Efficient sparse coding algorithms," in Proc. Adv. Neural Inf. Process. Syst., vol. 19. 2007, p. 801.
58. Z. Li, Y. Yang, J. Liu, X. Zhou, and H. Lu, "Unsupervised feature selection using nonnegative spectral analysis," in Proc. AAAI Conf. Artif. Intell., 2012, pp. 1-7.
59. R. Jenatton, J.-Y. Audibert, and F. Bach, "Structured variable selection with sparsity-inducing norms," J. Mach. Learn. Res., vol. 12, pp. 2777-2824, Feb. 2011.
60. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
61. S. Dudoit, J. Fridlyand, and T. P. Speed, "Comparison of discrimination methods for the classification of tumors using gene expression data," J. Amer. Statist. Assoc., vol. 97, no. 457, pp. 77-87, 2002.
62. M. Slawski, W. zu Castell, and G. Tutz, "Feature selection guided by structural information," Ann. Appl. Statist., vol. 4, no. 2, pp. 1056-1080, 2010.
63. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik, "Feature selection for SVMs," in Proc. NIPS, vol. 12. 2000, pp. 668-674.