Online tree-based ensembles and option trees for regression on evolving data streams

Neurocomputing

Volumn 150, Issue PB, 2015, Pages 458-470

  Ikonomovska, Elena ^a   Gama, João ^b   Džeroski, Sašo ^c  

^a Platform Applied Science Infrastructure Amobee Inc   (United States)

^b UNIVERSITY OF PORTO   (Portugal)

^c JO EF STEFAN INSTITUTE   (Slovenia)

Author keywords

Data streams; Online ensembles; Online learning; Online random forests; Online regression trees; Option trees

Indexed keywords

ARTIFICIAL INTELLIGENCE; BATCH DATA PROCESSING; DATA COMMUNICATION SYSTEMS; DATA MINING; DECISION TREES; LEARNING ALGORITHMS; LEARNING SYSTEMS; QUALITY CONTROL; REGRESSION ANALYSIS; SOCIAL NETWORKING (ONLINE);

DATA STREAM; ONLINE ENSEMBLES; ONLINE LEARNING; OPTION TREES; RANDOM FORESTS; REGRESSION TREES;

E-LEARNING;

ARTICLE; HOEFFDING TREE; LEARNING ALGORITHM; LEAST MEAN SQUARE FILTER; MEASUREMENT ACCURACY; ONLINE BAGGING; ONLINE SYSTEM; PROCESS DESIGN; RANDOM FOREST; REGRESSION ANALYSIS; STATISTICAL MODEL;

EID: 84922728924     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.04.076     Document Type: Article

References (33)

1. Kuncheva L.I., Whitaker C.J. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 2003, 51(2):181-207.
2. P. Domingos, G. Hulten, Mining high-speed data streams, in: Proceedings of 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2000, pp. 71-80.
3. G. Hulten, L. Spencer, P. Domingos, Mining time-changing data streams, in: Proceedings 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2001, pp. 97-106.
4. J. Gama, R. Rocha, P. Medas, Accurate decision trees for mining high-speed data streams, in: Proceedings of 9th International Conference on Knowledge Discovery and Data Mining, ACM, 2003, pp. 523-528.
5. J. Gama, P. Medas, R. Rocha, Forest trees for on-line data, in: Proceedings of 19th ACM Symposium on Applied Computing, ACM, 2004, pp. 632-636.
6. Ikonomovska E., Gama J., Džeroski S. Learning model trees from evolving data streams. Data Mining and Knowledge Discovery 2010, 1-41.
7. Rutkowski L., Pietruczuk L., Duda P., Jaworski M. Decision trees for mining data streams based on the McDiarmid[U+05F3]s bound. IEEE Transactions of Knowledge and Data Engineering 2013, 25(6):1272-1279.
8. E. Ikonomovska, J. Gama, B. Ženko, S. Džeroski, Speeding-up Hoeffding-based regression trees with options, in: Proceedings of 28th International Conference on Machine Learning, ACM, 2011, pp. 537-544.
9. W.N. Street, Y. Kim, A streaming ensemble algorithm (SEA) for large-scale classification, in: Proceedings of 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2001, pp. 377-382.
10. H. Wang, W. Fan, P.S. Yu, J. Han, Mining concept-drifting data streams using ensemble classifiers, in: Proceedings of 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2003, pp. 226-235.
11. Gama J., Medas P. Learning decision trees from dynamic data streams. Universal Computer Science 2005, 11(8):1353-1366.
12. Zhang Y., Jin X. An automatic construction and organization strategy for ensemble learning on data streams. ACM SIGMOD Record 2006, 35(3):28-33.
13. Scholz M., Klinkenberg R. Boosting classifiers for drifting concepts. Intelligent Data Analysis 2007, 11(1):3-28.
14. Kolter J.Z., Maloof M.A. Dynamic weighted majority: an ensemble method for drifting concepts. Machine Learning Research 2007, 8:2755-2790.
15. P. Zhang, X. Zhu, Y. Shi, Categorizing and mining concept drifting data streams, in: Proceedings of 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2008, pp. 812-820.
16. Tsymbal A., Pechenizkiy M., Cunningham P., Puuronen S. Dynamic integration of classifiers for handling concept drift. Information Fusion 2008, 9(1):56-68.
17. Li P., Wu X., Hu X., Liang Q., Gao Y. A random decision tree ensemble for mining concept drifts from noisy data streams. Applied Artificial Intelligence 2010, 24(7):680-710.
18. N.C. Oza, S.J. Russell, Experimental comparisons of online and batch versions of bagging and boosting, in: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2001, pp. 359-364.
19. A. Bifet, G. Holmes, B. Pfahringer, R. Kirkby, R. Gavaldà, New ensemble methods for evolving data streams, in: Proceedings of 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2009, pp. 139-148.
20. A. Bifet, R. Gavaldà, Learning from time-changing data with adaptive windowing, in: Proceedings of 7th SIAM International Conference on Data Mining, SIAM, 2007.
21. Freund Y., Schapire R.E. A decision-theoretic generalization of on-line learning and an application to boosting. Computer and System sciences 1997, 55(1):119-139.
22. R. Pelossof, M. Jones, I. Vovsha, C. Rudin, Online coordinate boosting, in: Proceedings of 12th International Conference on Computer Vision Workshops, IEEE Computer Society, 2009, pp. 1354-1361.http://arxiv.org/abs/0810.4553v1.
23. H. Abdulsalam, D.B. Skillicorn, P. Martin, Streaming random forests, in: Proceedings of 11th International Database Engineering and Applications Symposium, IEEE Computer Society, 2007, pp. 225-232.
24. H. Abdulsalam, D. B. Skillicorn, P. Martin, Classifying evolving data streams using dynamic streaming random forests, in: Proceedings of 19th International Conference on Database and Expert Systems Applications, Springer-Verlag, 2008, pp. 643-651.
25. A. Bifet, G. Holmes, B. Pfahringer, Leveraging bagging for evolving data streams, in: Proceedings of the European Conference on Machine learning and Knowledge Discovery in Databases, Springer-Verlag, 2010, pp. 135-150.
26. Wolpert D.H. Stacked generalization. Neural Networks 1992, 5:241-259.
27. Ting K.M., Witten I.H. Issues in stacked generalization. Artificial Intelligence Research 1999, 10(1):271-289.
28. Bifet A., Frank E., Holmes G., Pfahringer B. Ensembles of restricted Hoeffding trees. ACM Transactions on Intelligent Systems and Technology 2012, 3(2):30:1-30:20.
29. Breiman L. Random forests. Machine Learning 2001, 45:5-32.
30. R. Kohavi, C. Kunz, Option decision trees with majority votes, in: Proceedings of 14th International Conference on Machine Learning, Morgan Kaufmann Publishers Inc., 1997, pp. 161-169.
31. N. Ueda, R. Nakano, Generalization error of ensemble estimators, in:Proceedings of 1996 IEEE International Conference on Neural Networks, vol. 1, IEEE Computer Society, 1996, pp. 90-95.
32. A. Frank, A. Asuncion, UCI machine learning repository (2010). URL http://archive.ics.uci.edu/ml.
33. E. Ikonomovska, Algorithms for learning regression trees and ensembles on evolving data streams. (PhD Thesis), 2012.