Simultaneous feature selection and Gaussian mixture model estimation for supervised classification problems

Pattern Recognition

Volumn 47, Issue 8, 2014, Pages 2582-2595

  Kersten, Jens ^a  

^a GERMAN AEROSPACE CENTER DLR   (Germany)

Author keywords

Clustering; Expectation maximization; Feature saliency; Feature selection; Gaussian mixture models; Remote sensing; Supervised learning

Indexed keywords

ALGORITHMS; COMMUNICATION CHANNELS (INFORMATION THEORY); EXPERIMENTS; FEATURE EXTRACTION; IMAGE SEGMENTATION; MAXIMUM PRINCIPLE; OBJECT RECOGNITION; REMOTE SENSING; SUPERVISED LEARNING;

CLUSTERING; COMPLEX DATA DISTRIBUTIONS; EXPECTATION - MAXIMIZATIONS; EXPECTATION-MAXIMIZATION ALGORITHMS; FEATURE SALIENCY; GAUSSIAN MIXTURE MODEL; STATE-OF-THE-ART METHODS; SUPERVISED CLASSIFICATION;

ESTIMATION;

EID: 84899480198     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2014.02.015     Document Type: Article

References (45)

1. R.O. Duda, and P.E. Hart Pattern Classification and Scene Analysis 1973 John Wiley & Sons New York
2. J. Ratsaby, S.S. Venkatesh, Learning from a mixture of labeled and unlabeled examples with parametric side information, in: Proceedings of the Eighth Annual Conference on Computational Learning Theory, COLT '95, ACM, New York, NY, USA, 1995, pp. 412-417.
3. X. Zhu, Semi-Supervised Learning Literature Survey, Technical Report 1530, Computer Sciences, University of Wisconsin-Madison, 2005.
4. A.P. Dempster, N.M. Laird, and D.B. Rubin Maximum likelihood from incomplete data via the EM-algorithm J. R. Stat. Soc. Ser. B 39 1 1977 1 38
5. G.J. McLachlan, and T. Krishnan The EM Algorithm and Extensions, Wiley Series in Probability and Statistics 2nd ed. 2008 Wiley-Interscience Hoboken, New Jersey
6. J.A. Bilmes, A Gentle Tutorial on the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models, Technical Report, International Computer Science Institute, Department of Electrical Engineering and Computer Science, 1997.
7. C.F.J. Wu On the convergence properties of the EM algorithm Ann. Stat. 11 1 1983 95 103
8. M. Dash, and H. Liu Feature selection for classification Intell. Data Anal. 1 1997 131 156
9. M. Koeppen, The curse of dimensionality, in: 5th Online World Conference on Soft Computing in Industrial Applications (WSC5), 2000.
10. I. Guyon An introduction to variable and feature selection J. Mach. Learn. Res. 3 2003 1157 1182
11. P. Pudil, J. Novovicova, and J. Kittler Floating search methods in feature selection Pattern Recognit. Lett. 15 11 1994 1119 1125
12. Y. Peng, Z. Wu, and J. Jiang A novel feature selection approach for biomedical data classification J. Biomed. Inf. 43 1 2010 15 23
13. M. Kudo, and J. Sklansky Comparison of algorithms that select features for pattern classifiers Pattern Recognit. 33 2000 25 41 (Pubitemid 32081876)
14. A.K. Jain, R.P.W. Duin, and J. Mao Statistical pattern recognition a review IEEE Trans. Pattern Anal. Mach. Intell. 22 1 2000 4 37
15. F. Ferri, P. Pudil, M. Hatef, J. Kittler, Comparative study of techniques for large scale feature selection, in: E. Gelsema, L. Kanal (Eds.), Pattern Recognition in Practice IV, 1994, pp. 403-413.
16. J. Kersten, M. Gaehler, and S. Voigt A general framework for fast and interactive classification of optical VHR satellite imagery using hierarchical and planar Markov random fields PFG Photogramm. Fernerkund. Geoinformation. 6 2010 439 449
17. J. Kersten, Ein Rahmenwerk zur interaktiven Klassifikation hochaufloesender optischer Satellitenbilder mittels graphenbasierter Bildmodellierung. (Dissertation), Technical University Berlin, Faculty of Electrical Engineering and Computer Science, Department of Computer Engineering and Microelectronics: Computer Vision and Remote Sensing, 2011.
18. D. Ververidis, and C. Kotropoulos Gaussian mixture modeling by exploiting the Mahalanobis-distance IEEE Trans. Signal Process. 56 2008 2797 2811
19. M.H.C. Law, M.A.T. Figueiredo, and A.K. Jain Simultaneous feature selection and clustering using mixture models IEEE Trans. Pattern Anal. Mach. Intell 26 2004 1154 1166
20. J.G. Dy, C.E. Brodley, Feature subset selection and order identification for unsupervised learning, in: Proceedings of the 17th International Conference on Machine Learning, Morgan Kaufmann, 2000, pp. 247-254.
21. K. Mardia, J. Kent, and J. Bibby Multivariate Analysis, Probability and Mathematical Statistics 1979 Academic Press London
22. P. Langley, Selection of relevant features in machine learning, in: Proceedings of the AAAI Fall Symposium on Relevance, 1994, pp. 1-5.
23. P.M. Narendra, and K. Fukunaga A branch and bound algorithm for feature subset selection IEEE Trans. Comput. 26 1977 917 922 (Pubitemid 8550870)
24. A.W. Whitney A direct method of nonparametric measurement selection IEEE Trans. Comput. 20 9 1971 1100 1103
25. W. Siedlecki, and J. Sklansky A note on genetic algorithm for large-scale feature selection Pattern Recognit. Lett. 10 5 1989 335 347
26. G.J. McLachlan, D. Peel, Finite Mixture Models, Wiley Series in Probability and Statistics, New York, 2000.
27. Q. Zhao, V. Hautamäki, I. Kärkkäinen, and P. Fränti Random swap {EM} algorithm for gaussian mixture models Pattern Recognit. Lett. 33 16 2012 2120 2126
28. Z. Zhang EM algorithms for Gaussian mixtures with split-and-merge operation Pattern Recognit. 36 9 2003 1973 1983 (Pubitemid 36726817)
29. M.A.T. Figueiredo, S. Member, and A.K. Jain Unsupervised learning of finite mixture models IEEE Trans. Pattern Anal. Mach. Intell. 24 3 2002 381 396 (Pubitemid 35289353)
30. C. Constantinopoulos, M.K. Titsias, and A. Likas Bayesian feature and model selection for Gaussian mixture models IEEE Trans. Pattern Anal. Mach. Intell. 28 2006 1013 1018
31. P. Pudil, J. Novovicová, N. Choakjarernwanit, and J. Kittler Feature selection based on the approximation of class densities by finite mixtures of special type Pattern Recognit. 28 9 1995 1389 1398
32. Y. Li, M. Dong, and J. Hua Simultaneous localized feature selection and model detection for Gaussian mixtures IEEE Trans. Pattern Anal. Mach. Intell. 31 2009 953 960
33. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik Gene selection for cancer classification using support vector machines Mach. Learn. 46 1-3 2002 389 422 (Pubitemid 34129977)
34. T. Marill, and D.M. Green On the effectiveness of receptors in recognition system IEEE Trans. Inf. Theory 9 1963 11 17
35. P. Somol, P. Pudil, J. Novovicová, and P. Paclík Adaptive floating search methods in feature selection Pattern Recognit. Lett. 20 1999 1157 1163 (Pubitemid 32261893)
36. H. Roemer, J. Kersten, R. Kiefl, S. Plattner, A. Mager, and S. Voigt Airborne near-real-time monitoring of assembly and parking areas in case of large-scale public events and natural disasters Int. J. Geogr. Inf. Sci. 2013 1 18
37. H.C. Law, Clustering, dimensionality reduction, and side information (Ph.D. thesis), East Lansing, MI, USA, 2006.
38. J. Novovicova, P. Pudil, and J. Kittler Divergence based feature selection for multimodal class densities IEEE Trans. Pattern Anal. Mach. Intell. 18 2 1996 218 223 (Pubitemid 126772809)
39. S. Vaithyanathan, B. Dom, Generalized model selection for unsupervised learning in high dimensions, in: Proceedings of Neural Information Processing Systems, MIT Press, 1999, pp. 970-976.
40. G. Celeux, S. Chretien, F. Forbes, A. Mkhadri, A Component-wise EM-Algorithm for Mixtures, Technical Report RR-3746, INRIA, Rhone-Alpes, Montbonnot St. Martin, France, August 1999.
41. S. Aksoy, and R.M. Haralick Feature normalization and likelihood-based similarity measures for image retrieval Pattern Recognit. Lett. 22 2000 563 582
42. A. Asuncion, D.J. Newman, UCI Machine Learning Repository, 2007. URL: 〈 http://www.ics.uci.edu/~mlearn/MLRepository.html 〉.
43. 〈 http://vabene.dlr.de/vabene 〉, VABENE homepage, 2012 (accessed 23 November 2012).
44. F. Kurz, R. Mueller, M. Stephani, P. Reinartz, M. Schroeder, Calibration of a wide-angle digital camera system for near real time scenarios, in: High Resolution Earth Imaging for Geospatial Information, ISPRS Workshop 2007, Hannover, Germany, 2007.
45. L. Rokach A survey of clustering algorithms O. Maimon, L. Rokach, Data Mining and Knowledge Discovery Handbook 2010 Springer USA 269 298