An on-line weighted ensemble of regressor models to handle concept drifts

Engineering Applications of Artificial Intelligence

Volumn 37, Issue , 2015, Pages 392-406

  Gomes Soares, Symone ^a   Araújo, Rui ^a  

^a UNIVERSITY OF COIMBRA   (Portugal)

Author keywords

Concept drift; Ensemble learning; Ensemble pruning strategies; Learning in changing environments; Regression

Indexed keywords

ERRORS;

CHANGING ENVIRONMENT; CONCEPT DRIFTS; ENSEMBLE LEARNING; ENSEMBLE PRUNING; REGRESSION;

LEARNING ALGORITHMS;

EID: 84910607774     PISSN: 09521976     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.engappai.2014.10.003     Document Type: Article

References (50)

1. F. Ahmed, S. Nazir, and Y. Yeo A recursive PLS-based soft sensor for prediction of the melt index during grade change operations in HDPE plant Korean J. Chem. Eng. 26 January (1) 2009 14 20
2. Baena-García, M., del Campo-Ávila, J., Fidalgo, R., Bifet, A., Gavaldà, R., Morales-Bueno, R., September 18 2006. Early drift detection method. In: ECML/PKDD 2006 Workshop on Knowledge Discovery from Data Streams. p. 1.
3. Z. Bosnić, J. Demšar, G. Kešpret, P.P. Rodrigues, J. Gama, and I. Kononenko Enhancing data stream predictions with reliability estimators and explanation Eng. Appl. Artif. Intell. 34 September (0) 2014 178 192
4. D. Brzezinski, and J. Stefanowski Reacting to different types of concept drift the accuracy updated ensemble algorithm IEEE Trans. Neural Netw. Learn. Syst. 25 January (1) 2014 81 94
5. M. Chao, S.Z. Xin, and L.S. Min Neural network ensembles based on copula methods and distributed multiobjective central force optimization algorithm Eng. Appl. Artif. Intell. 32 June (0) 2014 203 212
6. K.-W. Chau Application of a PSO-based neural network in analysis of outcomes of construction claims Autom. Constr. 16 August (5) 2007 642 646
7. Cheng, C., Chau, K.-W., Sun, Y., Lin, J., 2005. Long-term prediction of discharges in Manwan Reservoir using artificial neural network models. In: Advances in Neural Networks (ISNN 2005). Lecture Notes in Computer Science, vol. 3498. Springer Berlin Heidelberg, pp. 1040-1045.
8. Chu, F., Zaniolo, C., 2004. Fast and light boosting for adaptive mining of data streams. In: Advances in Knowledge Discovery and Data Mining. Lecture Notes in Computer Science, Vol. 3056. Springer Berlin Heidelberg, pp. 282-292.
9. S. de Jong SIMPLS an alternative approach to partial least squares regression Chemom. Intell. Lab. Syst. 18 March (3) 1993 251 263
10. C. D'Este, G. Timms, A. Turnbull, and A. Rahman Ensemble aggregation methods for relocating models of rare events Eng. Appl. Artif. Intell. 34 September (0) 2014 58 65
11. Drucker, H., 1997. Improving regressors using boosting techniques. In: Proceedings of the Fourteenth International Conference on Machine Learning. ICML'97. Morgan Kaufmann Publishers Inc. pp. 107-115.
12. Elwell, R., Polikar, R., June 14-19 2009. Incremental learning in nonstationary environments with controlled forgetting. In: Proceedings of the International Joint Conference on Neural Networks. pp. 771-778.
13. R. Elwell, and R. Polikar Incremental learning of concept drift in nonstationary environments IEEE Trans. Neural Netw. 22 October (10) 2011 1517 1531
14. Feely, R., 2000. Predicting Stock Market Volatility Using Neural Networks (Bachelor's thesis) (unpublished). Department of Computer Science, Trinity College Dublin.
15. Fortuna, L., Graziani, S., Rizzo, A., Xibilia, M.G., 2006. Soft Sensors for Monitoring and Control of Industrial Processes (Advances in Industrial Control). Springer-Verlag New York, Inc., Secaucus, NJ, USA.
16. Y. Freund, and R.E. Schapire A decision-theoretic generalization of on-line learning and an application to boosting J. Comput. Syst. Sci. 55 August (1) 1997 119 139
17. J.H. Friedman Multivariate adaptive regression splines Ann. Stat. 19 March (1) 1991 1 67
18. Grbovic, M., Vucetic, S., 2011. Tracking concept change with incremental boosting by minimization of the evolving exponential loss. In: Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases - Vol. I. Springer-Verlag, pp. 516-532.
19. Z.-K. Huang, and K.-W. Chau A new image thresholding method based on gaussian mixture model Appl. Math. Comput. 205 November (2) 2008 899 907
20. P.H. Ibargüengoytia, M.A. Delgadillo, U.A. García, and A. Reyes Viscosity virtual sensor to control combustion in fossil fuel power plants Eng. Appl. Artif. Intell. 26 9 2013 2153 2163
21. Ikonomovska, E., October 2012. Algorithms for Learning Regression Trees and Ensembles on Evolving Data Streams (Ph.D. thesis). Jožef Stefan Int. Postgraduate School.
22. P. Kadlec, and B. Gabrys Local learning-based adaptive soft sensor for catalyst activation prediction AIChE Journal 57 August (5) 2011 1288 1301
23. P. Kadlec, R. Grbić, and B. Gabrys Review of adaptation mechanisms for data-driven soft sensors Comput. Chem. Eng. 35 January (1) 2011 1 24
24. I.B. Khediri, M. Limam, and C. Weihs Variable window adaptive kernel principal component analysis for nonlinear nonstationary process monitoring Comput. Ind. Eng. 61 October (3) 2011 437 446
25. Klinkenberg, R., October 2005. Meta-learning, model selection, and example selection in machine learning domains with concept drift. In: Proceedings of Annual Workshop of the Special Interest Group on Machine Learning, Knowledge Discovery, and Data Mining. FGML-2005. pp. 164-171.
26. Kolter, J.Z., Maloof, M.A., August 7-11 2005. Using additive expert ensembles to cope with concept drift. In: Proceedings of the Twenty-second International Conference on Machine Learning. ACM, pp. 449-456.
27. N. Littlestone, and M.K. Warmuth The weighted majority algorithm Inf. Comput. 108 February (2) 1994 212 261
28. Y. Liu, N. Hu, H. Wang, and P. Li Soft chemical analyzer development using adaptive least-squares support vector regression with selective pruning and variable moving window size Ind. Eng. Chem. Res. 48 May (12) 2009 5731 5741
29. L.L. Minku, A.P. White, and X. Yao The impact of diversity on online ensemble learning in the presence of concept drift IEEE Trans. Knowl. Data Eng. 22 May (5) 2010 730 742
30. L.L. Minku, and X. Yao DDD a new ensemble approach for dealing with concept drift IEEE Trans. Knowl. Data Eng. 24 April (4) 2012 619 633
31. Miranda, L.F.P., 2012. Metodologias de inteligência computacional para projecto de sensores virtuais em processos industriais (Master's thesis). Department of Electrical and Computer Engineering, University of Coimbra, Coimbra, Portugal.
32. W. Ni, S.K. Tan, W.J. Ng, and S.D. Brown Localized, adaptive recursive partial least squares regression for dynamic system modeling Ind. Eng. Chem. Res. 51 May (23) 2012 8025 8039
33. Nishida, K., Yamauchi, K., August 19-22 2007. Adaptive classifiers-ensemble system for tracking concept drift. In: Proceedings of the Sixth International Conference on Machine Learning and Cybernetics, Vol. 6. pp. 3607-3612.
34. Nishida, K., Yamauchi, K., Omori, T., 2005. ACE: Adaptive classifiers-ensemble system for concept-drifting environments. In: Multiple Classifier Systems. Lecture Notes in Computer Science, Vol. 3541. Springer Berlin Heidelberg, pp. 176-185.
35. Polikar, R., 2012. Ensemble learning. In: Ensemble Machine Learning: Methods and Applications. Springer, pp. 1-34.
36. S.J. Qin Recursive PLS algorithms for adaptive data modeling Comput. Chem. Eng. 22 January (4-5) 1998 503 514
37. A. Shaker, and E. Hüllermeier IBLStreams a system for instance-based classification and regression on data streams Evol. Syst. 3 December (4) 2012 235 249
38. S. Shen, G. Li, and H. Song An assessment of combining tourism demand forecasts over different time horizons J. Travel Res. 47 November (2) 2008 197 207
39. D.L. Shrestha, and D.P. Solomatine Experiments with AdaBoost.RT, an improved boosting scheme for regression Neural Comput. 18 July (7) 2006 1678 1710
40. S. Soares, C.H. Antunes, and R. Araújo Comparison of a genetic algorithm and simulated annealing for automatic neural network ensemble development Neurocomputing 121 December (0) 2013 498 511
41. Soares, S.G., Antunes, C.H., Araújo, R., July 07-11 2012. A genetic algorithm for designing neural network ensembles. In: Proc.of the 4th Int. Conf. on Genetic and Evolutionary Computation. ACM, pp. 681-688.
42. F.A.A. Souza, and R. Araújo Mixture of partial least squares experts and application in prediction settings with multiple operating modes Chemom. Intell. Lab. Syst. 130 January 2014 192 202
43. R. Taormina, K.-W. Chau, and R. Sethi Artificial neural network simulation of hourly groundwater levels in a coastal aquifer system of the Venice Lagoon Eng. Appl. Artif. Intell. 25 December (8) 2012 1670 1676
44. A. Vergara, S. Vembu, T. Ayhan, M.A. Ryan, M.L. Homer, and R. Huerta Chemical gas sensor drift compensation using classifier ensembles Sens. Actuat. B: Chem. 166-167 May (0) 2012 320 329
45. G. Widmer, and M. Kubat Learning in the presence of concept drift and hidden contexts Mach. Learn. 23 April (1) 1996 69 101
46. Wise, B.M., Gallagher, N.B., Bro, R., Shaver, J.M., 2003. PLS Toolbox 3.0. Eigenvector Research, Inc.
47. C.L. Wu, K.-W. Chau, and Y.S. Li Predicting monthly streamflow using data-driven models coupled with data-preprocessing techniques Water Resources Res. 45 August (8) 2009 1 23
48. W. Yan, H. Shao, and X. Wang Soft sensing modeling based on support vector machine and bayesian model selection Comput. Chem. Eng. 28 July (8) 2004 1489 1498
49. J. Zhang, and K.-W. Chau Multilayer ensemble pruning via novel multi-sub-swarm particle swarm optimization J. Univ. Comput. Sci. 15 February (4) 2009 840 858
50. Zliobaite, I., 2009. Learning under concept drift: an overview. Technical Report, Faculty of Mathematics and Informatics, Vilnius University.