A dynamically adjusted mixed emphasis method for building boosting ensembles

IEEE Transactions on Neural Networks

Volumn 19, Issue 1, 2008, Pages 3-17

  Gómez Verdejo, Vanessa ^a   Arenas Garcia, Jerónimo ^a   Figueiras Vidal, Aníbal R ^a  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

Author keywords

Adaboost; Boosting; Convex combination; Dynamic parameter selection; Emphasis

Indexed keywords

CLASSIFICATION (OF INFORMATION); DYNAMICAL SYSTEMS; ERROR ANALYSIS; LEARNING ALGORITHMS; OPTIMIZATION; PARAMETER ESTIMATION;

DYNAMIC PARAMETER SELECTION; DYNAMICALLY ADAPTED WEIGHTED EMPHASIS REAL ADABOOST; REAL ADABOOST ENSEMBLE;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; LEARNING; NONLINEAR SYSTEM; PHYSIOLOGY; PRINCIPAL COMPONENT ANALYSIS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; LEARNING; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; PRINCIPAL COMPONENT ANALYSIS;

EID: 39549086158     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.902723     Document Type: Article

References (50)

1. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Upper Saddle River, NJ: Prentice-Hall, 1999.
2. A. J. C. Sharkey, Ed., Combining Artificial Neural Nets. Ensemble and Modular Multi-Net Systems. London, U.K.: Springer-Verlag, 1999.
3. R. A. Jacobs and M. I. Jordan, "A competitive modular connectionist architecture," in Advances in Neural Information Processing Systems 3, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1991, pp. 667-773.
4. M. I. Jordan and R. A. Jacobs, "Hierarchical mixtures of experts and the EM algorithm," Neural Comput., vol. 6, no. 2, pp. 181-214, 1994.
5. Y. Freund and R. E. Schapire, Experiments with a new boosting algorithm," in Proc. 13th Int. Conf. Mach. Learn., Bari, Italy, 1996, pp. 148-156.
6. Y. Freund and R. E. Schapire, "Game theory, on-line prediction and boosting," in Proc. 9th Annu. Conf. Comput. Learn. Theory, Desenzano di Garda, Italy, 1996, pp. 325-332.
7. V. Boyarshinov and M. Magdon-Ismali, "Efficient optimal linear boosting of a pair of classifiers," IEEE Trans. Neural Netw., vol. 18, no. 2, pp. 317-328, Mar. 2007.
8. R. E. Schapire and Y. Singer, "Improved boosting algorithms using confidence-rated predictions," Mach. Learn., vol. 37, no. 3, pp. 297-336, 1999.
9. J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting," Ann. Statist., vol. 28, pp. 337-374, 2000.
10. C. Cachm, "Pedagogical pattern selection strategies," Neural Netw. vol. 7, no. 1, pp. 175-181, 1994.
11. P. E. Hart, "The condensed nearest neighbor rule," IEEE Trans. Inf. Theory, vol. IT-14, no. 3, pp. 515-516, May 1968.
12. P. W. Munro, "Repeat until bored: A pattern selection strategy," in Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. Lippman, Eds. San Mateo, CA: Morgan Kaufmann, 1992, pp. 1001-1008.
13. M. Plutowski and H. White, "Selecting concise training sets from clean data," IEEE Trans. Neural Netw., vol. 4, no. 2, pp. 305-318, Mar. 1993.
14. M. Wann, T. Hediger, and N. N. Greenbaum, "The influence of training sets on generalization in feed-forward neural networks," in Proc. Int. Joint Conf. Neural Netw., San Diego, CA, 1990, vol. 111, pp. 137-142.
15. B. T. Zhang, "Accelerated learning by active example selection," Int. J. Neural Syst., vol. 5, no. 1, pp. 67-75, 1994.
16. R. E. Schapire, Y. Freund, P. Bartlett, and W. S. Lee, "Boosting the margin: A new explanation for the effectiveness of voting methods," Ann. Statist., vol. 26, no. 5, pp. 1651-1686, 1998.
17. J. Arenas-García, A. R. Figueiras-Vidal, and A. J. C. Sharkey, "The beneficial effects of using multi net systems that focus on hard patterns," in Lecture Notes in Computer Science, T. Windeatt and F. Rolli, Eds. Berlin, Germany: Springer-Verlag, 2003, pp. 45-54.
18. A. Lyhiyaoui, M. Martínez-Ramón,, I. Mora-Jiménez, M. Vázquez-Castro, J. L. Sancho-Gómez, and A. R. Figueiras-Vidal, "Sample selection via clustering to construct support vector-like classifiers," IEEE Trans. Neural Netw., vol. 10, no. 6, pp. 1474-1481, Nov. 1999.
19. R. Garćia-Marciel, I. Mora-Jiménez, and A. R. Figueiras-Vidal, "Improving kernel-based classifiers by guided dynamic sample selection," in Proc. 13th Int. Conf. Artif. Neural Netw. Eng., A. Press, Ed., St. Louis, MO, 2003, pp. 27-32.
20. J. Sklansky and L. Michelotti, "Locally trained piecewise linear classifiers," IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-2, no. 2, pp. 101-111, Feb. 1980.
21. L. Breiman, "Combining predictors," in Combining Artificial Neural Nets. Ensemble and Modular Multi-Net Systems, A. J. C. Sharkey, Ed. London, U.K.: Springer-Verlag, 1999, pp. 31-50.
22. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Disc., vol. 2, no. 2, pp. 121-167, 1998.
23. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
24. K. R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf, "An introduction to kernel-based learning algorithms," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 181-201, Mar. 2001.
25. M. Lehtokangas, "Cascade-correlation learning for classification," IEEE Trans. Neural Netw., vol. 11, no. 3, pp. 795-798, May 2000.
26. S. E. Fallman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems 2, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 524-532.
27. G. Rätsch, T. Onoda, and K. R. Muller, "Soft margins for Adaboost," Mach. Learn., vol. 42, no. 3, pp. 287-320, 2001.
28. C. Rudin, I. Daubechies, and R. E. Schapire, "The dynamics of AdaBoost: Cyclic behavior and convergence of margins," J. Mach. Learn. Res. vol. 5, pp. 1557-1595, 2004.
29. G. Rätsch and M. K. Warmuth, "Efficient margin maximizing with boosting," J. Mach. Learn. Res., vol. 6, pp. 2131-2152, 2005.
30. C. Rudin, R. E. Schapire, and I. Daubechies, "Boosting based on a smooth margin," in Proc. 17th Annu. Conf. Comput. Learn. Theory, Banff, AB, Canada, 2004, pp. 502-517.
31. V. Gomez-Verdejo, M. Ortega-Moral, J. Arenas-García, and A. R. Figueiras-Vidal, "Boosting by weighting critical and erroneous samples," Neurocomputing, vol. 69, pp. 679-685, 2006.
32. V. Gómez-Verdejo, M. Ortega-Moral, J. P. Cabrera, J. Arenas-García, and A. R. Figueiras-Vidal, "Boosting by emphasizing boundary samples," in Proc. Int. Conf. Learn., Elche, Spain, 2004, pp. 67-72.
33. V. Gómez-Verdejo, M. Ortega-Moral, J. Arenas-García, and A. R. Figueiras-Vidal, "Boosting by weighting boundary and erroneous samples," in Proc. 13th Eur. Symp. Artif. Neural Netw., Bruges, Belgium, 2005, pp. 85-90.
34. R. E. Schapire, "The strength of weak learnability," in Proc. 30th Annu. Symp. Found. Comput. Sci., 1989, pp. 28-33.
35. J. Lu, K. N. Plataniotis, A. N. Venetsanopoulos, and S. Z. Li, "Ensemble-based discriminant learning with boosting for face recognition," IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 166-178, Jan. 2006.
36. T. Windeatt, "Accuracy/diversity and ensemble MLP classifier design," IEEE Trans. Neural Netw., vol. 17, no. 5, pp. 1194-1211, Sep. 2006.
37. D. Opitz and R. Maclin, "Popular ensemble methods: An empirical study," J. Artif. Intell. Res., vol. 11, pp. 169-198, 1999.
38. H. Schwenk and Y. Bengio, "Boosting neural networks," Neural Comput., vol. 12, no. 8, pp. 1869-1887, 2000.
39. V. Gómez-Verdejo, J. Arenas-García, M. Ortega-Moral, and A. R. Figueiras-Vidal, "Designing RBF classifiers for weighted boosting," in Proc. Int. Joint Conf. Neural Netw., Montreal, QC, Canada, 2005, pp. 1057-1062.
40. D. G. Luenberger, Linear and Nonlinear Programming, 2nd ed. Reading, MA: Addison-Wesley, 1984.
41. R. Meir and G. Rätsch, "An introduction to boosting and leveraging," in Advanced Lectures on Machine Learning, S. Mendelson and A. Smola, Eds. New York: Springer-Verlag, 2003, pp. 119-184.
42. V. Vapnik and A. Chervonenkis, "On the uniform convergence of relative frequencies of events to their probabilities," Theory Probab. Appl. vol. 16, no. 2, pp. 264-280, 1971.
43. G. Rätsch, "Robust boosting via convex optimization: Theory and applications," Ph.D. dissertation, Dept. Comp. Sci., Univ. Potsdam, Potsdam, Germany, 2001.
44. J. von Neumann, "Zur Theorie der Gesellschaftsspiele," Math. Ann. vol. 100, pp. 295-320, 1928.
45. J. T. Kwok, "Moderating the output of support vector classifiers," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1018-1031, Sep. 1999.
46. B. D. Ripley, "Neural networks and related methods for classification (with discussion)," J. Roy. Statist. Soc. B, vol. 56, pp. 409-456, 1994.
47. C. L. Blake and C. J. Merz, Dept. Inf. Comp. Sci., Univ. California, "UCI Repository of Machine Learning Databases," Irvine, CA, 1998 [Online]. Available: http://www.ics.uci.edu/-learn/MLRepository.html
48. P. Alinat, "Periodic Progress Report 4, ROARS Project ESPRIT II - Number 5516," Tech. Thomson Rep. TS ASM 93/S/EGS/NC/079, 1993.
49. A. Ruiz and P. E. L. de Teruel, "Nonlinear kernel-based statistical pattern analysis," IEEE Trans. Neural Netw., vol. 12, no. 1, pp. 16-32, Jan. 2001.
50. F. Pérez-Cruz, "IRWLS Matlab toolbox to solve the SVM for pattern recognition and regression estimation," 2002 [Online]. Available: http://www.tsc.uc3m.es/-fernando/