Sparse ensembles using weighted combination methods based on linear programming

Pattern Recognition

Volumn 44, Issue 1, 2011, Pages 97-106

  Zhang, Li ^a   Zhou, Wei Da ^a  

^a XIDIAN UNIVERSITY   (China)

Author keywords

Classifier ensemble; k nearest neighbor; Linear programming; Linear weighted combination; Sparse ensembles

Indexed keywords

CLASSIFIER ENSEMBLES; COMBINATION METHOD; DATA SETS; ENSEMBLE CLASSIFIERS; ENSEMBLE LEARNING; HIGH RESOLUTION RANGE PROFILES; K-NEAREST NEIGHBORS; LINEAR PROGRAMMING PROBLEM; LINEAR WEIGHTED COMBINATION; MACHINE-LEARNING; MULTIPLE CLASSIFIERS; SPARSE ENSEMBLES; STRUCTURE RISK MINIMIZATION; TRAINING ERRORS; WEIGHT COEFFICIENTS; WEIGHT VECTOR;

LINEAR PROGRAMMING; MOTION COMPENSATION; OPTIMIZATION;

CLASSIFIERS;

EID: 77956456333     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2010.07.021     Document Type: Article

References (48)

1. L.I. Kuncheva Combining Pattern Classifiers: Methods and Algorithms 2004 Wiley Hoboken, NJ
2. T. Windeatt, F. Roli (Eds.), Multiple Classifier Systems, in: Lecture Notes in Computer Science, vol. 2709, Springer, 2003.
3. F. Roli, J. Kittler, T. Windeatt (Eds.), Multiple Classifier Systems, in: Lecture Notes in Computer Science, vol. 3077, Springer, 2004.
4. E.K. Tang, P.N. Suganthan, and X. Yao An analysis of diversity measures Machine Learning 65 2006 247 271
5. L. Breiman Bagging predictors Machine Learning 24 1996 123 140
6. Y. Freund, R. Shapire, Experiments with a new boosting algorithm, in: 13th International Conference on Machine Learning, Bary, Italy, Morgan Kaufmann, 1996, pp. 148156.
7. R. Duda, P. Hart, and D. Stork Pattern Classification second ed. 2000 John Wiley & Sons
8. T.K. Ho, J.J. Hull, and S.N. Srihari Decision combination in multiple classifier systems IEEE Transactions on Pattern Analysis and Machine Intelligence 16 1 1994 66 75
9. S.D. Bay, Combining nearest neighbor classifiers through multiple feature subsets, in: Proceedings of the 15th International Conference on Machine Learning, Madison, WI, 1998, pp. 3745.
10. J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas On combining classifiers IEEE Transactions on Pattern Analysis and Machine Intelligence 20 3 1998 226 239
11. J.A. Benediktsson, J.R. Sveinsson, O.K. Ersoy, and P.H. Swain Parallel consensual neural networks IEEE Transactions on Neural Networks 8 1 1997 54 64
12. N. Ueda Optimal linear combination of neural networks for improving classification performance IEEE Transactions on Pattern Analysis and Machine Intelligence 22 2 2000 207 215
13. L.I. Kuncheva A theoretical study on six classifier fusion strategies IEEE Transactions on Pattern Analysis and Machine Intelligence 24 2002 281 286
14. V. Tresp, and M. Taniguchi Combining estimators using non-constant weighting functions Advances in Neural Information Processing Systems vol. 7 1995 419 426
15. L. Breiman Stacked regressions Machine Learning 24 1996 49 64
16. G. Fumera, and F. Roli A theoretical and experimental analysis of linear combiners for multiple classifier systems IEEE Transactions on Pattern Analysis and Machine Intelligence 27 6 2005 942 956
17. X. Yao, and Y. Liu Making use of population information in evolutionary artificial neural networks IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics 28 3 1998 417 425
18. A.J. Grove, D. Schuurmans, Boosting in the limit: maximizing the margin of learned ensembles, in: Proceedings of the 15th National Conference on Artificial Intelligence, American Association for Artificial Intelligence, Menlo Park, CA, USA, 1998, pp. 692699.
19. T. Graepel, R. Herbrich, and J. Shawe-Taylor Generalization error bounds for sparse linear classifiers 13th Annual Conference on Computational Learning Theory 2000 298 303
20. S. Floyd, and M. Marmuth Sample compression learnability, and the VapnikChervonenkis dimension Machine Learning 21 3 1995 269 304
21. L. Zhang, and W.D. Zhou On the sparseness of 1-norm support vector machines Neural Networks 23 3 2010 373 385
22. G. Martnez-Muoz, D. Hernndez-Lobato, and A. Surez An analysis of ensemble pruning techniques based on ordered aggregation IEEE Transactions on Pattern Analysis and Machine Intelligence 31 2 2009 245 259
23. Z.H. Zhou, J.X. Wu, and W. Tang Ensembling neural networks: many could be better than all Artificial Intelligence 137 12 2002 239 263
24. Y. Zhang, S. Burer, and W.N. Street Ensemble pruning via semi-definite programming Journal of Machine Learning Research 7 2006 1315 1338
25. D.D. Margineantu, and T.G. Dietterich Pruning adaptive boosting Proceedings of the 14th International Conference on Machine Learning 1997 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA 211 218
26. G. Martnez-Muoz, and A. Surez Aggregation ordering in bagging Proceedings of the International Conference on Artificial Intelligence and Applications 2004 258 263
27. G. Martnez-Muoz, and A. Surez Pruning in ordered bagging ensembles Proceedings of the 23th International Conference on Machine Learning 2006 609 616
28. G. Martnez-Muoz, and A. Surez Using boosting to prune bagging ensembles Pattern Recognition Letters 28 1 2007 156 165
29. H. Chen, P. Tino, and X. Yao Predictive ensemble pruning by expectation propagation IEEE Transactions on Knowledge and Data Engineering 21 7 2009 999 1013
30. G. Tsoumakas, I. Katakis, and I.P. Vlahavas Effective voting of heterogeneous classifiers Proceedings of the 11th European Conference on Machine Learning, Lecture Notes in Artificial Intelligence vol. 3201 2004 465 476
31. G. Tsoumakas, L. Angelis, and I. Vlahavas Selective fusion of heterogeneous classifiers Intelligent Data Analysis 9 2005 511 525
32. Z.H. Zhou, and Y. Yu Adapt bagging to nearest neighbor classifier Journal of Computer Science and Technology 20 2005 48 54
33. V. Vapnik The Nature of Statistical Learning Theory 1995 Springer-Verlag New York
34. C.J.C. Burges A tutorial on support vector machines for pattern recognition Data Mining and Knowledge Discovery 2 2 1998 121 167
35. F. Girosi, M. Jones, and T. Poggio Regularization theory and neural networks architectures Neural Computation 7 2 1995 219 269
36. F. Girosi An equivalence between sparse approximation and support vector machines Neural Computation 10 6 1998 1455 1480
37. O. Mangasarian Exact 1-norm support vector machines via unconstrained convex differentiable minimization Journal of Machine Learning Research 7 2006 1517 1530
38. J. Zhu, S. Rosset, T. Hastie, and R. Tibshirani 1-norm support vector machines Advances in Neural Information Processing Systems, Lecture Notes in Computer Science vol. 16 2004 MIT Press Cambridge, MA 49 56
39. J. Bi, Y. Chen, and J. Wang A sparse support vector machine approach to region-based image categorization Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Lecture Notes in Computer Science 2005 MIT Press Cambridge, MA 1121 1128
40. J. Bi, K.P. Bennett, M. Embrechts, C.M. Breneman, and M. Song Dimensionality reduction via sparse support vector machines Journal of Machine Learning Research 4 2003 1229 1243
41. G.M. Fung, and O.L. Mangasarian A feature selection Newton method for support vector machine classification Computational Optimization and Applications 28 2004 185 202
42. W.D. Zhou, L. Zhang, and L.C. Jiao Linear programming support vector machines Pattern Recognition 35 12 2002 2927 2936
43. R. Venderbei Linear Programming: Foundation and Extension 1996 Academic Press New York
44. P. Murphy, D. Aha, UCI machine learning repository, from 〈 http://www.ics.uci.edu/∼mlearn/MLRepository.html 〉, 1992.
45. L. Du, H.W. Liu, Z. Bao, and J.Y. Zhang A two-distribution compounded statistical model for radar HRRP target recognition IEEE Transactions on Signal Processing 54 6 2006 2226 2238
46. B. Chen, H. Liu, and Z. Bao Optimizing the data-dependent kernel under a unified kernel optimization framework Pattern Recognition 41 6 2008 2107 2119
47. C. Domeniconi, J. Peng, and D. Gunopulos Locally adaptive metric nearest neighbor classification IEEE Transactions on Pattern Analysis and Machine Intelligence 24 9 2002 1281 1285 (Pubitemid 35242588)
48. Y. Bao, N. Ishii, and X. Du Combining multiple k-nearest neighbor classifiers using different distance functions Proceedings of the Fifth International Conference on Intelligent Data Engineering and Automated Learning, Lecture Notes in Computer Science vol. 3177 2004 Springer-Verlag Berlin 634 641