A dynamic and on-line ensemble regression for changing environments

Expert Systems with Applications

Volumn 42, Issue 6, 2015, Pages 2935-2948

  Soares, Symone G ^a   Araújo, Rui ^a  

^a UNIVERSITY OF COIMBRA   (Portugal)

Author keywords

Concept drift; Ensemble learning; Learning in changing environments; Regression

Indexed keywords

REGRESSION ANALYSIS; SOCIAL NETWORKING (ONLINE); VIRTUAL REALITY;

ADAPTATION STRATEGIES; CHANGING ENVIRONMENT; COMPUTATIONAL COSTS; CONCEPT DRIFTS; DYNAMIC ADAPTATIONS; ENSEMBLE LEARNING; REGRESSION; STATE-OF-THE-ART APPROACH;

E-LEARNING;

EID: 84949117585     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2014.11.053     Document Type: Article

References (41)

1. Abellán, J., & Mantas, C. J. (2014). Improving experimental studies about ensembles of classifiers for bankruptcy prediction and credit scoring. Expert Systems with Applications, 41(8), 3825-3830.
2. Bifet, A., & Gavaldà, R., (2007). Learning from time-changing data with adaptive windowing. In Proc. of the 2007 SIAM int. conf. on data mining (pp. 443-448).
3. Brzezinski, D., & Stefanowski, J. (2014). Combining block-based and online methods in learning ensembles from concept drifting data streams. Information Sciences, 265(0), 50-67.
4. Caruana, R., Lawrence, S., & Giles, L., (2000). Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In Proc. neural information processing systems conf. NIPS'00 (pp. 402-408).
5. Chu, F., & Zaniolo, C. (2004). Fast and light boosting for adaptive mining of data streams. In Advances in knowledge discovery and data mining. Lecture notes in computer science (Vol. 3056, pp. 282-292). Berlin Heidelberg: Springer.
6. Elwell, R., & Polikar, R., (2009). Incremental learning in nonstationary environments with controlled forgetting. In Proc. of the int. joint conf. on neural networks (pp. 771-778).
7. Elwell, R., & Polikar, R. (2011). Incremental learning of concept drift in nonstationary environments. IEEE Transactions on Neural Networks, 22(10), 1517-1531.
8. Farid, D. M., Zhang, L., Hossain, A., Rahman, C. M., Strachan, R., Sexton, G., & Dahal, K. (2013). An adaptive ensemble classifier for mining concept drifting data streams. Expert Systems with Applications, 40(15), 5895-5906.
9. Fortuna, L., Graziani, S., Rizzo, A., & Xibilia, M. G. (2006). Soft sensors for monitoring and control of industrial processes (advances in industrial control). Secaucus, NJ, USA: Springer-Verlag New York, Inc.
10. Friedman, J. H. (1991). Multivariate adaptive regression splines. The Annals of Statistics, 19(1), 1-67.
11. Galicia, H. J., He, Q. P., & Wang, J. (2012). Comparison of the performance of a reduced-order dynamic PLS soft sensor with different updating schemes for digester control. Control Engineering Practice, 20(8), 747-760.
12. Gjerkes, H., Malensek, J., Sitar, A., & Golobic, I. (2011). Product identification in industrial batch fermentation using a variable forgetting factor. Control Engineering Practice, 19(10), 1208-1215.
13. Grbić, R., Slišković, D., & Kadlec, P. (2013). Adaptive soft sensor for online prediction and process monitoring based on a mixture of gaussian process models. Computers & Chemical Engineering, 58(0), 84-97.
14. Grbovic, M., & Vucetic, S. (2011). Tracking concept change with incremental boosting by minimization of the evolving exponential loss. In Proc. of the 2011 European conf. on machine learning and knowledge discovery in databases (pp. 516-532). Springer-Verlag.
15. Ikonomovska, E. (2012). Algorithms for learning regression trees and ensembles on evolving data streams (Phd's thesis). Jožef Stefan Int. Postgraduate School.
16. Jin, H., Chen, X., Yang, J., & Wu, L. (2014). Adaptive soft sensor modeling framework based on just-in-time learning and kernel partial least squares regression for nonlinear multiphase batch processes. Computers & Chemical Engineering, 71(0), 77-93.
17. Kadlec, P., & Gabrys, B. (2011). Local learning-based adaptive soft sensor for catalyst activation prediction. AIChE Journal, 57(5), 1288-1301.
18. Kadlec, P., Grbić, R., & Gabrys, B. (2011). Review of adaptation mechanisms for datadriven soft sensors. Computers & Chemical Engineering, 35(1), 1-24.
19. Kaneko, H., & Funatsu, K. (2014). Adaptive soft sensor based on online support vector regression and bayesian ensemble learning for various states in chemical plants. Chemometrics and Intelligent Laboratory Systems, 137(0), 57-66.
20. Kolter, J. Z., & Maloof, M. A. (2005). Using additive expert ensembles to cope with concept drift. In Proc. of the 22nd int. conf. on machine learning (pp. 449-456). New York, NY, USA: ACM.
21. Lan, Y., Soh, Y. C., & Huang, G.-B. (2009). Ensemble of online sequential extreme learning machine. Neurocomputing, 72(13-15), 3391-3395.
22. Liang, N.-Y., Huang, G.-B., Saratchandran, P., & Sundararajan, N. (2006). A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Transactions on Neural Networks, 17(6), 1411-1423.
23. Lin, B., Recke, B., Knudsen, J. K. H., & Jørgensen, S. B. (2007). A systematic approach for soft sensor development. Computers & Chemical Engineering, 31(5-6), 419-425.
24. Liu, G., Zhou, D., Xu, H., & Mei, C. (2010). Model optimization of SVM for a fermentation soft sensor. Expert Systems with Applications, 37 (4), 2708-2713.
25. Lv, Y., Liu, J., Yang, T., & Zeng, D. (2013). A novel least squares support vector machine ensemble model for NOx emission prediction of a coal-fired boiler. Energy, 55 (0), 319-329.
26. Mangat, V., & Vig, R. (2014). Novel associative classifier based on dynamic adaptive PSO: Application to determining candidates for thoracic surgery. Expert Systems with Applications, 41(18), 8234-8244.
27. Minku, L. L., & Yao, X. (2012). DDD: A new ensemble approach for dealing with concept drift. IEEE Transactions on Knowledge and Data Engineering, 24(4), 619-633.
28. Ni, W., Brown, S. D., & Man, R. (2014). A localized adaptive soft sensor for dynamic system modeling. Chemical Engineering Science, 111(0), 350-363.
29. Nishida, K., & Yamauchi, K., (2007). Adaptive classifiers-ensemble system for tracking concept drift. In Int. conf. on machine learning and cybernetics. Vol. 6 (pp. 3607-3612).
30. Nishida, K., Yamauchi, K., & Omori, T. (2005). ACE: Adaptive classifiers-ensemble system for concept-drifting environments. In Multiple classifier systems. Lecture notes in computer science (Vol. 3541, pp. 176-185). Berlin Heidelberg: Springer.
31. Oza, N. C., & Russell, S., (2001). Experimental comparisons of online and batch versions of bagging and boosting. In Proc. of the 7th ACM SIGKDD int. conf. on knowledge discovery and data mining. KDD'01 (pp. 359-364).
32. Polikar, R. (2012). Ensemble learning. In C. Zhang & Y. Ma (Eds.), Ensemble machine learning: Methods and applications (pp. 1-34). New York, NY, USA: Springer.
33. Qin, S. J. (1998). Recursive PLS algorithms for adaptive data modeling. Computers & Chemical Engineering, 22(4-5), 503-514.
34. Schimunek, B., Chalmers, J., Kettaneh-Wold, N., & Ho, T. T., (2008). Process drift recognition and solutions.
35. Shrestha, D. L., & Solomatine, D. P. (2006). Experiments with AdaBoost. RT, an improved boosting scheme for regression. Neural Computation, 18(7), 1678-1710.
36. Soares, S., Antunes, C. H., & Araújo, R. (2013). Comparison of a genetic algorithm and simulated annealing for automatic neural network ensemble development. Neurocomputing, 121(0), 498-511.
37. Soares, S. G., & Araújo, R. (2015). An on-line weighted ensemble of regressor models to handle concept drifts. Engineering Applications of Artificial Intelligence, 37(0), 392-406.
38. Soares, S., Araújo, R., Sousa, P., & Souza, F., (2011). Design and application of soft sensor using ensemble methods. In Proc. of the 16th IEEE int. conf. on emerging technologies and factory automation. ETFA'11 (pp. 1-8).
39. Tsymbal, A. (2004). The problem of concept drift: Definitions and related work. Tech. rep., The University of Dublin, Trinity College, Department of Computer Science, Dublin, Ireland.
40. Vergara, A., Vembu, S., Ayhan, T., Ryan, M. A., Homer, M. L., & Huerta, R. (2012). Chemical gas sensor drift compensation using classifier ensembles. Sensors and Actuators B: Chemical, 166-167(0), 320-329.
41. von Stosch, M., Oliveira, R., Peres, J., & de Azevedo, S. F. (2011). A novel identification method for hybrid (N)PLS dynamical systems with application to bioprocesses. Expert Systems with Applications, 38(9), 10862-10874.