Embedded local feature selection within mixture of experts

Information Sciences

Volumn 269, Issue , 2014, Pages 176-187

  Peralta, Billy ^a   Soto, Alvaro ^a  

^a PONTIFICIA UNIVERSIDAD CATÓLICA DE CHILE   (Chile)

Author keywords

Embedded feature selection; Local feature selection; Mixture of experts; Regularization

Indexed keywords

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

DIVIDE AND CONQUER; EMBEDDED FEATURE SELECTIONS; EMBEDDED PROCESS; HIGH DIMENSIONAL DATA; LOCAL FEATURE SELECTIONS; MIXTURE OF EXPERTS; REAL-WORLD DATASETS; REGULARIZATION;

MIXTURES;

EID: 84897053423     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2014.01.008     Document Type: Article

References (55)

1. J. Aguilar, Dataset Repository in arff, 2008. < http://www.upo.es/eps/ aguilar/datasets.html >.
2. A. Asuncion, D. Newman, UCI Machine Learning Repository, 2007. < http://www.ics.uci.edu/∼mlearn/MLRepository.html >.
3. R. Battiti Using mutual information for selecting features in supervised neural net learning IEEE Trans. Neural Netw. 5 1994 537 550
4. C. Bishop Pattern Recognition and Machine Learning (Information Science and Statistics) second ed. 2007 Springer New York, USA
5. C. Bishop, M. Svensén, Bayesian hierarchical mixtures of experts, in: Conference on Uncertainty in Artificial Intelligence, 2003, pp. 57-64.
6. S. Boyd, and L. Vandenberghe Convex Optimization 2004 Cambridge University Press Cambridge, United Kingdom
7. J. Bradley, A. Kyrola, D. Bickson, and C. Guestrin Parallel coordinate descent for L1-regularized loss minimization Int. Conf. Mach. Learn. 2011 321 328
8. L. Breiman Random forests Mach. Learn. 45 2001 5 32
9. A. Dempster, N. Laird, and D. Rubin Maximum likelihood from incomplete data via the EM algorithm J. Roy. Stat. Soc. Ser. B (Methodological) 39 1977 1 38
10. R. Duda, P. Hart, and D. Stork Pattern Classification second ed. 2001 Wiley-Interscience USA
11. R. Ebrahimpour, and F.M. Jafarlou View-independent face recognition with hierarchical mixture of experts using global eigenspaces J. Commun. Comput. 7 2010 1103 1107
12. Y. Freund, and R. Schapire A decision-theoretic generalization of on-line learning and an application to boosting Proceedings of the European Conference on Computational Learning Theory 1995 Springer-Verlag London, UK 23 37
13. B. Geng, D. Tao, T. Xu, L. Yang, and X. Hua Ensemble manifold regularization IEEE Trans. Pattern Anal. Mach. Intell. 34 2012 1227 1233
14. I. Guyon, and A. Elisseeff An introduction to variable and feature selection J. Mach. Learn. Res. 3 2003 1157 1182
15. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik Gene selection for cancer classification using support vector machines J. Mach. Learn. 46 2002 389 422
16. M. Hall, Correlation-based Feature Selection for Machine Learning, Ph.D. Thesis, University of Waikato, 1999.
17. J. Hampshire, and A. Waibel The meta-pi network: building distributed knowledge representations for robust multisource pattern recognition IEEE Trans. Pattern Anal. Mach. Intell. 14 1992 751 769
18. T. Ho The random subspace method for constructing decision forests IEEE Trans. Pattern Anal. Mach. Intell. 20 1998 832 844
19. R. Jacobs, M. Jordan, S. Nowlan, and G. Hinton Adaptive mixtures of local experts Neural Comput. 3 1991 79 87
20. M. Jordan, and R. Jacobs Hierarchical mixtures of experts and the EM algorithm Neural Comput. 6 1994 181 214
21. A. Khalili New estimation and feature selection methods in mixture-of-experts models Can. J. Stat. 38 2010 519 539
22. R. Kohavi, and G. John Wrappers for feature subset selection Artif. Intell. 97 1997 273 324
23. S.I. Lee, H. Lee, P. Abbeel, A.Y. Ng, Efficient L1 regularized logistic regression, in: Proceedings of the 21st National Conference on Artificial Intelligence (AAAI), 2006, pp. 401-408.
24. C. Lima, A. Coelho, and F. Von Zuben Hybridizing mixtures of experts with support vector machines: investigation into nonlinear dynamic systems identification Inform. Sci. 177 2007 2049 2074
25. H. Liu, Arizona State University: Feature Selection Datasets, 2012. < http://featureselection.asu.edu/datasets.php >.
26. H. Liu, and R. Setiono Chi2: Feature selection and discretization of numeric attributes J. Vassilopoulos, Proceedings of the International Conference on Tools with Artificial Intelligence 1995 IEEE Computer Society Herndon, Virginia 388 391
27. D. MacKay Probable networks and plausible predictions - a review of practical Bayesian methods for supervised neural networks Netw.: Comput. Neural Syst. 6 1995 469 505
28. S. Maldonado, R. Weber, and J. Basak Simultaneous feature selection and classification using kernel-penalized support vector machines Inform. Sci. 181 2011 115 128
29. E. Meeds, S. Osindero, An alternative infinite mixture of Gaussian process experts, in: Advances In Neural Information Processing Systems, 2005, pp. 883-890.
30. P. Moerland, Some Methods for Training Mixtures of Experts, Technical Report, IDIAP Research Institute, 1997.
31. S.K. Murthy, S. Kasif, and S. Salzberg A system for induction of oblique decision trees J. Artif. Intell. Res. 2 1994 1 32
32. M. Nguyen, H. Abbass, and R. McKay A novel mixture of experts model based on cooperative coevolution Neurocomputing 70 2006 155 163
33. R. Ñanculef, C. Valle, H. Allende, and C. Moraga Training regression ensembles by sequential target correction and resampling Inform. Sci. 195 2012 154 174
34. W. Pan, and X. Shen Penalized model-based clustering with application to variable selection J. Mach. Learn. Res. 8 2007 1145 1164
35. N. Pinto, D. Cox, and J. DiCarlo Why is real-world visual object recognition hard? PLoS Comput. Biol. 4 2008 151 156
36. J. Quinlan C4.5: Programs for Machine Learning 1993 Morgan Kaufmann Publishers Inc. California, USA
37. C. Rasmussen, Z. Ghahramani, Infinite mixtures of gaussian process experts, in: Advances in Neural Information Processing Systems, 2001, pp. 881-888.
38. J. Rice Mathematical Statistics and Data Analysis second ed. 1994 Duxbury Press
39. J. Saragih, S. Lucey, and J. Cohn Deformable model fitting with a mixture of local experts Int. Conf. Comput. Vision 2009 2248 2255
40. D. Scott, and S. Sain Multi-dimensional Density Estimation 2004 Elsevier Amsterdam pp. 229-263
41. A. Statnikov, I. Tsamardinos, Y. Dosbayev, and C. Aliferis GEMS: a system for automated cancer diagnosis and biomarker discovery from microarray gene expression data Int. J. Med. Inform. 74 2005 491 503
42. R. Tibshirani Regression shrinkage and selection via the Lasso J. Roy. Stat. Soc. (Ser. B) 58 1996 267 288
43. M. Titsias, and A. Likas Mixture of experts classification using a hierarchical mixture model Neural Comput. 14 2002 2221 2244
44. P. Tseng Convergence of block coordinate descent method for nondifferentiable maximization J. Optim. Theory Appl. 109 2001 475 494
45. A. Ulas, O. Taner, and E. Alpaydin Eigenclassifiers for combining correlated classifiers Inform. Sci. 187 2012 109 120
46. C. Van-Rijsbergen Information Retrieval second ed. 1979 Butterworth-Heinemann London, UK
47. J. Vogdrup Combining predictors: comparison of five meta machine learning methods Inform. Sci. 119 1999 91 105
48. M. Wang, X. Hua, R. Hong, J. Tang, J. Guo-Jung, and Y. Song Unified video annotation via multigraph learning IEEE Trans. Circ. Syst. Video Technol. 19 2009 733 746
49. M. Wang, H. Li, D. Tao, K. Lu, and X. Wu Multimodal graph-based reranking for web image search IEEE Trans. Image Process. 21 2012 4649 4661
50. S. Wang, and J. Zhu Variable selection for model-based high dimensional clustering and its application to microarray data Biometrics 64 2008 440 448
51. F. Wu, Y. Yuan, and Y. Zhuang Heterogeneous feature selection by group Lasso with logistic regression Int. Conf. Multimedia 2010 983 986
52. J. Xiao, C. He, X. Jiang, and D. Liu A dynamic classifier ensemble selection approach for noise data Inform. Sci. 180 2010 3402 3421
53. L. Xu, M. Jordan, G. Hinton, An alternative model for mixtures of experts, in: Advances in Neural Information Processing Systems, 1994, pp. 633-640.
54. K. Yang, M. Wang, X. Hua, S. Yan, and H. Zhang Assemble new object detector with few examples IEEE Trans. Image Process. 20 2011 3341 3349
55. A. Yuille, and D. Geiger Winner-take-all mechanisms M.A. Arbib, The Handbook of Brain Theory and Neural Networks 1998 MIT Press Cambridge, MA, USA 1056