Dimensionality reduction by feature clustering for regression problems

Information Sciences

Volumn 299, Issue , 2015, Pages 42-57

  Xu, Rong Fang ^a   Lee, Shie Jue ^a  

^a NATIONAL SUN YAT SEN UNIVERSITY   (Taiwan)

Author keywords

Correlation coefficient; Feature clustering; Feature extraction; Machine learning; Mutual information

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); COVARIANCE MATRIX; FEATURE EXTRACTION; LEARNING SYSTEMS; MATRIX ALGEBRA; REDUCTION; REGRESSION ANALYSIS; VIRTUAL REALITY;

CORRELATION COEFFICIENT; COVARIANCE MATRICES; DIMENSIONALITY REDUCTION; FEATURE CLUSTERING; GENERALIZATION CAPABILITY; MUTUAL INFORMATIONS; NUMBER OF CLUSTERS; TRAINING DATA SETS;

DATA REDUCTION;

EID: 84961291920     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2014.12.003     Document Type: Article

References (63)

1. D. Asteriou, and S.G. Hall Applied Econometrics 2011 Palgrave MacMillan
2. L.D. Baker, A. McCallum, Distributional clustering of words for text classification, in: Proc. 21st Annual International ACM SIGIR, 1998, pp. 96-103.
3. R. Battiti Using mutual information for selecting features in supervised neural net learning IEEE Trans. Neural Networks 5 4 1994 537 550
4. R. Bekkerman, R. El-Yaniv, N. Tishby, and Y. Winter Distributional word clusters vs. words for text categorization J. Mach. Learn. Res. 3 2003 1183 1208
5. BP source codes in the matlab toolbox, 2013.
6. A. Bravais, Analyse mathématique sur les probabilités des erreurs de situation d'un point, Memoirés presentes par divers savants à l'Académie Royale des Sciences de l'Institut de France, vol. 9, pp. 255-332.
7. P.L. Carmona, J.M. Sotoca, F. Pla, F.K.H. Phoa, J.B. Dias, Feature selection in regression tasks using conditional mutual information, in: Proc. 5th Iberian Conf. on Pattern Recognition and Image Analysis, 2011, pp. 224-231.
8. R.D. Cook, and S. Weisberg Sliced inverse regression for dimension reduction: comment J. Am. Stat. Assoc. 86 414 1991 328 332
9. M.C. Dalmau, O.W.M. Flórez, Experimental results of the signal processing approach to distributional clustering of terms on reuters-21578 collection, in: Proc. 29th European Conf. IR Research, 2007, pp. 678-681.
10. I.S. Dhillon, S. Mallela, and R. Kumar A divisive information-theoretic feature clustering algorithm for text classification J. Mach. Learn. Res. 3 2003 1265 1287
11. G. Doquire, and M. Verleysen A graph Laplacian based approach to semi-supervised feature selection for regression problems Neurocomputing 121 2013 5 13
12. P.A. Estévez, M. Tesmer, C.A. Perez, and J.M. Zurada Normalized mutual information feature selection IEEE Trans. Neural Networks 20 2 2009 189 201
13. Z.Z. Feng, X. Yang, S. Subedi, and P.D. McNicholas The LASSO and sparse least squares regression methods for SNP selection in predicting quantitative traits IEEE/ACM Trans. Comput. Biol. Bioinformatics 9 2 2012 629 636
14. B. Frénay, G. Doquire, and M. Verleysen Is mutual information adequate for feature selection in regression? Neural Networks 48 2013 1 7
15. M. Hall, Correlation-Based Feature Selection for Machine Learning, Ph.D. Thesis, University of Waikato, 1999.
16. J. Han, M. Kamber, and J. Pei Data Mining: Concepts and Techniques 3rd edition. 2011 Morgan Kaufmann Publishers
17. D. Hand, H. Mannila, and P. Smyth Principles of Data mining 2001 The MIT Press
18. W. Härdle, and L. Simar Applied Multivariate Statistical Analysis 2003 Springer-Verlag Berlin Heidelberg
19. J. Hausser, and K. Strimmer Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks J. Mach. Learn. Res. 10 2009 1469 1484
20. X. He, D. Cai, and P. Niyogi Laplacian Score for Feature Selection Advances in Neural Information Processing Systems, vol. 18 2005 The MIT Press
21. X. He, M. Ji, C. Zhang, and H. Bao A variance minimization criterion to feature selection using laplacian regularization IEEE Trans. Pattern Anal. Mach. Intell. 33 10 2011 2013 2025
22. J.F. Healey Statistics: A Tool for Social Research 9th edition. 2011 Cengage Learning
23. H.-H. Hsu, and C.-W. Hsieh Feature selection via correlation coefficient clustering J. Software 5 12 2010 1371 1377
24. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew Extreme learning machine: theory and applications Neurocomputing 70 2006 489 501
25. J.-Y. Jiang, S.-J. Lee, A weight-based feature extraction approach for text classification, in: Proc. 2nd International Conf. on Innovative Computing, Information and Control, 2007.
26. J.-Y. Jiang, R.-J. Liou, and S.-J. Lee A fuzzy self-constructing feature clustering algorithm for text classification IEEE Trans. Knowl. Data Eng. 23 3 2011 335 349
27. I.T. Jolliffe Principal Component Analysis 1986 Springer-Verlag
28. N. Kwak, and J.-W. Lee Feature extraction based on subspace methods for regression problems Neurocomputing 73 2010 1740 1751
29. A. Kyrillidis, V. Cevher, Combinatorial selection and least absolute shrinkage via the CLASH algorithm, in: Proc. IEEE International Symposium on Information Theory, 2012, pp. 2216-2220.
30. D.D. Lewis, Feature selection and feature extraction for text categorization, in: Proc. Workshop Speech and Natural Language, 1992, pp. 212-217.
31. F. Li, C. Sminchisescu, Feature selection in kernel regression via L1 regularization, in: Proc. 26th Internationl Conf. on Machine Learning, 2009.
32. K.-C. Li Sliced inverse regression for dimension reduction J. Am. Stat. Assoc. 86 414 1991 316 327
33. K.-C. Li On principal Hessian directions for data visualization and dimension reduction: another application of Stein's lemma J. Am. Stat. Assoc. 87 420 1992 1025 1039
34. W. Li Mutual information functions versus correlation functions J. Stat. Phys. 60 1990 823 837
35. G.-F. Lu, and Y. Wang Feature extraction using a fast null space based linear discriminant analysis algorithm Inf. Sci. 193 2012 72 80
36. S. Maldonado, R. Weber, Feature selection for support vector regression via kernel penalization, in: Proc. International Joint Conf. on Neural Networks, 2010, pp. 1-7.
37. A.M. Martinez, and A.C. Kak PCA versus LDA IEEE Trans. Pattern Anal. Mach. Intell. 23 2 2001 228 233
38. S. Menard Coefficients of determination for multiple logistic regression analysis Am. Stat. 54 1 2000 17 24
39. S. Mitra, P.P. Kundu, and W. Pedrycz Feature selection using structural similarity Inf. Sci. 198 2012 48 61
40. M.K. Pakhira, S. Bandyopadhyay, and U. Maulik Validity index for crisp and fuzzy clusters Pattern Recogn. 37 3 2004 487 501
41. D. Paul, E. Bair, T. Hastie, and R. Tibshirani "Preconditioning" for feature selection and regression in high-dimensional problems Ann. Statist. 36 4 2008 1595 1618
42. K. Pearson Notes on the history of correlation Biometrika 13 1 1920 25 45
43. R. Pedace Econometrics for Dummies 2013 Wiley
44. H. Peng, F. Long, and C. Ding Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy IEEE Trans. Pattern Anal. Mach. Intell. 27 8 2005 1226 1238
45. B. Peralta, and A. Soto Embedded local feature selection within mixture of experts Inf. Sci. 269 2014 176 187
46. J.O. Rawlings, S.G. Pantula, and D.A. Dickey Applied Regression Analysis: A Research Tool 2nd edition. 1998 Springer
47. Regression Data Set, 2013. < http://www.dcc.fc.up.pt/ltorgo/Regression/DataSets.html >.
48. F. Rossi, A. Lendasse, D. Francois, V. Wertz, and M. Verleysen Mutual information for the selection of relevant variables in spectrometric nonlinear modelling Chemometr. Intell. Lab. Syst. 80 2005 215 226
49. S.T. Roweis, and L.K. Saul Nonlinear dimensionality reduction by locally linear embedding Science 290 5500 2000 2323 2326
50. G. Silviu Information Theory with Applications 1977 McGraw-Hil
51. J.B. Tenenbaum, V. de Silva, and J.C. Langford A global geometric framework for nonlinear dimensionality reduction Science 290 5500 2000 2319 2323
52. S. Theodoridis, and K. Koutroumbas Pattern Recognition 4th edition. 2009 Academic Press Canada
53. R. Tibshirani Regression shrinkage and selection via the lasso J. R. Stat. Soc. Ser. B 58 1 1996 267 288
54. R. Tibshirani Regression shrinkage and selection via the lasso: a retrospective J. R. Stat. Soc. Ser. B 73 3 2011 273 282
55. J.-A. Ting, A.D. Souza, S. Vijayakumar, and S. Schaal Efficient learning and feature selection in high-dimensional regression Neural Comput. 22 2010 831 886
56. UCI Data Set, 2013. < http://archive.ics.uci.edu/ml/ >.
57. O. Valenzuela, I. Rojas, L.J. Herrera, A. Guillén, F. Rojas, L. Marquez, and M. Pasadas Feature selection using mutual information and neural networks Monografías del Seminario Matemático García de Galdeano 33 2006 331 340
58. S. Weisberg. dr Package. < http://cran.r-project.org/web/packages/dr/index.html >.
59. Y. Xu, G. Jones, J.-T. Li, B. Wang, and C.-M. Sun A study on mutual information-based feature selection for text categorization J. Comput. Inf. Syst. 3 3 2007 1007 1012
60. J. Yan, B. Zhang, N. Liu, S. Yan, Q. Cheng, W. Fan, Q. Yang, W. Xi, and Z. Chen Effective and efficient dimensionality reduction for large-scale and streaming data preprocessing IEEE Trans. Knowl. Data Eng. 18 3 2006 320 333
61. M. Yuan, and Y. Lin Model selection and estimation in regression with grouped variables J. R. Stat. Soc. Ser. B 68 1 2006 49 67
62. B. Zamani, A. Akbari, and B. Nasersharif Evolutionary combination of kernels for nonlinear feature transformation Inf. Sci. 274 2014 95 107
63. L.W. Zhong, and J.T. Kwok Efficient sparse modeling with automatic feature grouping IEEE Trans. Neural Networks Learn. Syst. 23 9 2012 1436 1447