Supervised projection approach for boosting classifiers

Pattern Recognition

Volumn 42, Issue 9, 2009, Pages 1742-1760

  García Pedrajas, Nicolás ^a  

^a UNIVERSITY OF CÓRDOBA   (Spain)

Author keywords

Boosting; Classification; Ensembles of classifiers; Supervised projections

Indexed keywords

ADABOOST; ALTERNATIVE APPROACH; BIAS VARIANCE DECOMPOSITION; BOOSTING; BOOSTING ALGORITHM; BOOSTING METHOD; CLASS LABELS; CLASSIFICATION; ENSEMBLES OF CLASSIFIERS; GENERALIZATION ERROR; NEW APPROACHES; NON-LINEAR; REAL-WORLD APPLICATION; SUPERVISED PROJECTIONS; UCI MACHINE LEARNING REPOSITORY; UNIFORM DISTRIBUTION; WEIGHTING SCHEME;

ADAPTIVE BOOSTING; CLASSIFIERS;

LEARNING SYSTEMS;

EID: 67349098005     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2008.12.023     Document Type: Article

References (59)

1. Breiman L. Stacked regressions. Machine Learning 24 1 (1996) 49-64
2. Kohavi R., and Kunz C. Option decision trees with majority voting. Proceedings of the Fourteenth International Conference on Machine Learning (1997), Morgan Kaufman, San Francisco, CA, USA 161-169
3. Bauer E., and Kohavi R. An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Machine Learning 36 1/2 (1999) 105-142
4. Webb G.I. Multiboosting: a technique for combining boosting and wagging. Machine Learning 40 2 (2000) 159-196
5. García-Pedrajas N., Hervás-Martínez C., and Ortiz-Boyer D. Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Transactions on Evolutionary Computation 9 3 (2005) 271-302
6. Merz C.J. Using correspondence analysis to combine classifiers. Machine Learning 36 1 (1999) 33-58
7. Dietterich T.G. Ensemble methods in machine learning. In: Kittler J., and Roli F. (Eds). Proceedings of the First International Workshop on Multiple Classifier Systems, Lecture Notes in Computer Science vol. 1857 (2000), Springer, Berlin 1-15
8. Freund Y., and Schapire R. Experiments with a new boosting algorithm. Proceedings of the Thirteenth International Conference on Machine Learning. Bari, Italy (1996) 148-156
9. Dzeroski S., and Zenko B. Is combining classifiers with stacking better than selecting the best one?. Machine Learning 54 (2004) 255-273
10. Fern A., and Givan R. Online ensemble learning: an empirical study. Machine Learning 53 (2003) 71-109
11. L. Breiman, Bias, variance, and arcing classifiers, Technical Report 460, Department of Statistics, University of California, Berkeley, CA, 1996.
12. Schapire R.E., Freund Y., Bartlett P.L., and Lee W.S. Boosting the margin: a new explanation for the effectiveness of voting methods. Annals of Statistics 26 5 (1998) 1651-1686
13. Freund Y., and Schapire R.E. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55 1 (1997) 119-139
14. Friedman J., Hastie T., and Tibshirani R. Additive logistic regression: a statistical view of boosting. Annals of Statistics 28 2 (2000) 337-407
15. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Machine Learning 40 (2000) 139-157
16. Schapire R.E., and Singer Y. Improved boosting algorithms using confidence-rated predictions. Machine Learning 37 (1999) 297-336
17. Sun Y., Todorovic S., and Li J. Reducing the overfitting of adaboost by controlling its data distribution skewdness. International Journal of Patterns Recognition and Artificial Intelligence 20 7 (2006) 1093-1116
18. Rätsch G., Onoda T., and Müller K.-R. Soft margins for adaboost. Machine Learning 42 (2001) 287-320
19. Vapnik V. The nature of Statistical Learning Theory (1999), Springer, New York
20. García-Pedrajas N., García-Osorio C., and Fyfe C. Nonlinear boosting projections for ensemble construction. Journal of Machine Learning Research 8 (2007) 1-33
21. Yu S., Yu K., Tresp V., and Kriegel H.-P. Multi-output regularized feature projection. IEEE Transactions on Knowledge and Data Engineering 18 12 (2006) 1600-1613
22. Ho T.K. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 8 (1998) 832-844
23. Geng X., Zhan D.-C., and Zhou Z.-H. Supervised nonlinear dimensionality reduction for visualization and classification. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics 35 6 (2005) 1098-1107
24. Li C.J., and Jansuwan C. Dynamic projection network for supervised pattern classification. International Journal of Approximate Reasoning 40 (2005) 243-261
25. Lee Y., Ahn D., and Moon K. Margin preserving projections. Electronic Letters 42 21 (2006) 1249-1250
26. Zhao H., Sun S., Jing Z., and Yang J. Local structure based supervised feature extraction. Pattern Recognition 39 (2006) 1546-1550
27. Lerner B., Guterman H., Aladjem M., Dinstein I., and Romem Y. On pattern classification with Sammon's nonlinear mapping-An experimental study. Pattern Recognition 31 4 (1998) 371-381
28. Tenenbaum J.B., de Silva V., and Langford J.C. A global geometric framework for nonlinear dimensionality reduction. Science 290 5500 (2000) 2319-2323
29. Kruskal J.B. Toward a practical method which helps uncover the structure of a set of multivariate observations by finding the linear transformation which optimizes a new 'index of condensation'. In: Milton R.C., and Nelder J.A. (Eds). Statistical Computing (1969), Academic Press, New York 427-440
30. Polzehl J. Projection pursuit discriminant analysis. Computational Statistics and Data Analysis 20 2 (1995) 141-157
31. Lee E., Cook D., Klinke S., and Lumley T. Projection pursuit for exploratory supervised classification. Journal of Computational & Graphical Statistics 14 4 (2005) 831-846
32. Lazarevic A., Fiez T., and Obradovic Z. Adaptive boosting for spatial functions with unstable driving attributes. Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining (2000), Springer, Japan 329-340
33. Gorman R.P.R.P., and Sejnowski T.J. Analysis of hidden units in a layered network trained to classify sonar targets. Neural Networks 1 (1988) 75-89
34. Mao J., and Jain A.K. Artificial neural networks for feature extraction and multivariate data projection. IEEE Transactions on Neural Networks 6 2 (1995) 296-317
35. Haykin S. Neural Networks-A Comprehensive Foundation. second ed. (1999), Prentice-Hall, Upper Saddle River, NJ
36. Lerner B., Guterman H., Aladjem M., and Dinstein I. A comparative study of neural networks based feature extraction paradigms. Pattern Recognition Letters 20 1 (1999) 7-14
37. Kuncheva L.I. Error bounds for aggressive and conservative adaboost. Proceedings of MCS, Lecture Notes in Computer Science vol. 2709 (2003), Springer, Berlin, Guilford, UK 25-34
38. S. Hettich, C. Blake, C. Merz, UCI repository of machine learning databases, 1998 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉.
39. Dietterich T.G. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10 7 (1998) 1895-1923
40. Demšar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7 (2006) 1-30
41. Rumelhart D., Hinton G., and Williams R.J. Learning internal representations by error propagation. In: Rumelhart D., and McClelland J. (Eds). Parallel Distributed Processing (1986), MIT Press, Cambridge, MA 318-362
42. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann, San Mateo
43. Cristianini N., and Shawe-Taylor J. An Introduction to Support Vector Machines (2000), Cambridge University Press, Cambridge
44. C.-C. Chang, C.-J. Lin, LIBSVM: a library for support vector machines, 2001, software available at 〈http://www.csie.ntu.edu.tw/∼cjlin/libsvm〉.
45. Zhang T., and Yu B. Boosting with early stooping: convergence and consistency. The Annals of Statistics 33 4 (2005) 1538-1579
46. Domingo C., and Watanabe O. MadaBoost: a modification of AdaBoost. Proceedings of the Thirteenth Annual Conference on Computational Learning Theory (2000), Morgan Kaufmann, San Francisco 180-189
47. Grove A.J., and Schuurmans D. Boosting in the limit: maximizing the margin of learned ensembles. Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98) (1998), Madison, WI, USA 692-699
48. Mason L., Bartlett P.L., and Baxter J. Improved generalization through explicit optimization of margins. Machine Learning 38 (2000) 243-255
49. Breiman L. Prediction games and arcing algorithms. Neural Computation 11 7 (1999) 1493-1517
50. Ortiz-Boyer D., Hervás-Martínez C., and García-Pedrajas N. Cixl2: a crossover operator for evolutionary algorithms based on population features. Journal of Artificial Intelligence Research 24 (2005) 33-80
51. Sierra A., and Echevarría A. Evolutionary discriminant analysis. IEEE Transactions on Evolutionary Computation 10 1 (2006) 81-92
52. Quinlan J.R. Bagging, boosting, and c4.5. Proceedings of the Thirteenth National Conference on Artificial Intelligence (1996), AAAI Press, MIT Press, New York, Cambridge, MA 725-730
53. J.R. Quinlan, Boosting first-order learning, in: Proceedings of the Algorithmic Learning Theory, Seventh International Workshop, ALT '96, Sydney, Australia, October 1996, vol. 1160, Springer, Berlin, 1996, pp. 143-155.
54. Rosset S., Zhu J., and Hastie T. Boosting as a regularized path to a maximum margin classifier. Journal of Machine Learning Research 5 (2004) 941-973
55. Breiman L. Random forests. Machine Learning 45 (2001) 5-32
56. Schapire R.E., Freund Y., Barlett P., and Lee W.S. Boosting the margin: a new explanation for the effectiveness of voting methods. Proceedings of the Fourteenth International Conference on Machine Learning (1997), Morgan Kaufmann 322-330
57. Weigend A.S. On overfitting and the effective number of hidden units. In: Mozer M.C., Smolensky P., Touretzky D.S., Elman J.L., and Weigend A.S. (Eds). Proceedings of the 1993 Connectionist Models Summer School (1993), Erlbaum Associates, Hillsdale, NJ 335-342
58. Kong E.B., and Dietterich T.G. Error-correcting output coding corrects bias and variance. In: Prieditis A., and Lemmer J.F. (Eds). Machine Learning: Proceedings of the Twelfth International Conference (1995), Elsevier Science Publishers, Amsterdam 275-283
59. Kohavi R., and Wolpert D.H. Bias plus variance decomposition for zero-one loss functions. In: Saitta L. (Ed). Machine Learning: Proceedings of the Thirteenth International Conference (1996), Morgan Kaufmann, Los Altos, CA 275-283