Benchmarking Ensemble Classifiers with Novel Co-Trained Kernal Ridge Regression and Random Vector Functional Link Ensembles [Research Frontier]

IEEE Computational Intelligence Magazine

Volumn 12, Issue 4, 2017, Pages 61-72

  Zhang, Le ^a   Suganthan, Ponnuthurai Nagaratnam ^b  

^a ADVANCED DIGITAL SCIENCES CENTER   (Singapore)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); DECISION TREES; LEARNING SYSTEMS;

CLASSIFICATION PERFORMANCE; DIFFERENT DOMAINS; ENSEMBLE CLASSIFIERS; FUNCTIONAL-LINK NETWORK; KERNEL RIDGE REGRESSIONS; OBLIQUE DECISION TREE; RANDOM FOREST ALGORITHM; RESEARCH FRONTIERS;

REGRESSION ANALYSIS;

EID: 85035802026     PISSN: 1556603X     EISSN: None     Source Type: Journal    
DOI: 10.1109/MCI.2017.2742867     Document Type: Article

References (40)

1. T. G. Dietterich, "Ensemble methods in machine learning," in Multiple Classifier Systems. New York: Springer, 2000, pp. 1-15.
2. Y. Ren, L. Zhang, and P. N. Suganthan, "Ensemble classification and regression-recent developments, applications and future directions [review article]," IEEE Computational Intell. Mag., vol. 11, no. 1, pp. 41-53, Feb. 2016.
3. T. G. Dietterich, "Ensemble learning," The Handbook of Brain Theory and Neural Networks. 2002, vol. 2, pp. 110-125.
4. P. Domingos, "A unified bias-variance decomposition," in Proc. Int. Conf. Machine Learning, 2000, pp. 231-238.
5. L. Breiman, "Random forests," Mach. Learn., vol. 45, no. 1, pp. 5-32, Jan. 2001.
6. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, Aug. 1996.
7. Y. Freund and R. E. Schapire, "Experiments with a new boosting algorithm," in Proc. Int. Conf. Machine Learning, 1996, vol. 96, pp. 148-156.
8. M. Fernández-Delgado, E. Cernadas, S. Barro, and D. Amorim, "Do we need hundreds of classifiers to solve real world classification problems?," J. Mach. Learn. Res., vol. 15, no. 1, pp. 3133-3181, Oct. 2014.
9. G. Valentini and T. G. Dietterich, "Bias-variance analysis of support vector machines for the development of SVM-based ensemble methods," J. Mach. Learn. Res., vol. 5, pp. 725-775, July 2004.
10. M. Gönen and E. Alpaydin, "Multiple kernel learning algorithms," J. Mach. Learn. Res., vol. 12, pp. 2211-2268, Jul., 2011.
11. B. H. Menze, B. M. Kelm, D. N. Splitthoff, U. Koethe, and F. A. Hamprecht, "On oblique random forests," in Proc. Machine Learning and Knowledge Discovery in Databases Conf., Sept. 2011, pp. 453-469.
12. L. Zhang and P. N. Suganthan, "Oblique decision tree ensemble via multisurface proximal support vector machine," IEEE Trans. Cybern., vol. 45, no. 10, pp. 2165-2176, Oct. 2015.
13. L. Zhang and P. N. Suganthan, "Random forests with ensemble of feature spaces," Pattern Recogn., vol. 47, no. 10, pp. 3429-3437, Oct. 2014.
14. C. Saunders, A. Gammerman, and V. Vovk, "Ridge regression learning algorithm in dual variables," in Proc. 15th Int. Conf. Machine Learning, 1998, pp. 515-521.
15. S. An, W. Liu, and S. Venkatesh, "Face recognition using kernel ridge regression," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007, pp. 1-7.
16. J. Friedman, T. Hastie, and R. Tibshirani, The Elements of Statistical Learning. Springer Series in Statistics. Berlin, Germany: Springer, 2001, vol. 1.
17. S. Ren, X. Cao, Y. Wei, and J. Sun, "Global refinement of random forest," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 723-730.
18. L. Breiman, Classification and Regression Trees. London, U.K.: Chapman & Hall, 1984.
19. B. Scholkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA: MIT Press, 2001.
20. J. J. Rodriguez, L. I. Kuncheva, and C. J. Alonso, "Rotation forest: A new classifier ensemble method," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 10, pp. 1619-1630, Oct. 2006.
21. S. Haykin and N. Network, "Neural networks: A comprehensive foundation," Neural Netw., vol. 2, no. 2004, Dec. 2004.
22. R. A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Netw., vol. 1, no. 4, pp. 295-307, 1988.
23. Y.-H. Pao, Adaptive Pattern Recognition and Neural Networks. Boston, MA: Addison-Wesley Longman, 1989.
24. M. Gori and A. Tesi, "On the problem of local minima in backpropagation," IEEE Trans. Pattern Anal. Mach. Intell., no. 1, pp. 76-86, Jan. 1992.
25. L. Zhang and P. Suganthan, "A survey of randomized algorithms for training neural networks," Inform. Sci., vol. 364, pp. 146-155, Oct. 2016.
26. Y.-H. Pao and Y. Takefji, "Functional-link net computing: theory, system architecture, and functionalities," IEEE Comput. J., vol. 25, no. 5, pp. 76-79, May. 1992.
27. B. Igelnik and Y.-H. Pao, "Stochastic choice of basis functions in adaptive function approximation and the functional-link net," IEEE Trans. Neural Netw., vol. 6, no. 6, pp. 1320-1329, Nov. 1995.
28. Y.-H. Pao and S. M. Phillips, "The functional link net and learning optimal control," Neurocomputing, vol. 9, no. 2, pp. 149-164, Oct. 1995.
29. L. Zhang and P. Suganthan, "A comprehensive evaluation of random vector functional link networks," Inform. Sci., vol. 367, pp. 1094-1105, Nov. 2016.
30. Y. Ren, P. Suganthan, N. Srikanth, and G. Amaratunga, "Random vector functional link network for short-term electricity load demand forecasting," Inform. Sci., vol. 367, pp. 1078-1093, Nov. 2016.
31. L. Zhang and P. N. Suganthan, "Visual tracking with convolutional random vector functional link network," IEEE Trans. Cybernetics, to be published.
32. M. Alhamdoosh and D. Wang, "Fast decorrelated neural network ensembles with random weights," Inform. Sci., vol. 264, pp. 104-117, Apr. 2014.
33. Y. Liu and X. Yao, "Ensemble learning via negative correlation," Neural Netw., vol. 12, no. 10, pp. 1399-1404, Dec. 1999.
34. V. Sindhwani, P. Niyogi, and M. Belkin, "A co-regularization approach to semi-supervised learning with multiple views," in Proc. ICML Workshop on Learning with Multiple Views, 2005, pp. 74-79.
35. M. Lichman. (2013). UCI machine learning repository. [Online]. Available: http://archive.ics.uci.edu/ml
36. J. Demŝar, "Statistical comparisons of classifiers over multiple data sets," J. Mach. Learn. Res., vol. 7, pp. 1-30, Jan. 2006.
37. G. Brown, J. Wyatt, R. Harris, and X. Yao, "Diversity creation methods: a survey and categorisation," Inform. Fusion, vol. 6, no. 1, pp. 5-20, Mar. 2005.
38. F. Orabona and L. Jie, "Ultra-fast optimization algorithm for sparse multi kernel learning," in Proc. 28th Int. Conf. Machine Learning, 2011, pp. 249-256.
39. A. Rakotomamonjy, F. R. Bach, S. Canu, and Y. Grandvalet, "SimpleMKL," J. Mach. Learn. Res., vol. 9, pp. 2491-2521, Nov. 2008.
40. D. J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures. Boca Raton, FL: CRC, 2003.