Learning concept-drifting data streams with random ensemble decision trees

Neurocomputing

Volumn 166, Issue , 2015, Pages 68-83

  Li, Peipei ^a   Wu, Xindong ^a,b   Hu, Xuegang ^a   Wang, Hao ^a  

^a HEFEI UNIVERSITY OF TECHNOLOGY   (China)

^b UNIVERSITY OF VERMONT   (United States)

Author keywords

Concept drift; Data streams; Noisy data; Random decision tree

Indexed keywords

DECISION TREES; TREES (MATHEMATICS);

CONCEPT DRIFTING; CONCEPT DRIFTS; INCREMENTAL ALGORITHM; MULTIPLE SOLUTIONS; NOISY DATA; ON-LINE ALGORITHMS; ON-LINE CLASSIFICATION; RANDOM DECISION TREES;

DATA STREAMS;

ALGORITHM; ARTICLE; COMPARATIVE STUDY; INCREMENT ALALGORITHM; LEARNING CONCEPT DRIFTING DATA; MATHEMATICAL MODEL; MEASUREMENT ACCURACY; PREDICTION; PRIORITY JOURNAL; SIGNAL NOISE RATIO; STATISTICAL BIAS; VALIDATION PROCESS;

EID: 84931090810     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2015.04.024     Document Type: Article

References (63)

1. G. Johannes, G. Venkatesh, R. Raghu, L. Wei-Yin, BOAT-optimistic decision tree construction, in: Proceedings of SIGMOD'99, 1999, pp. 169-180.
2. Ross Q.J. C4.5: Programs for Machine Learning 1993, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
3. J. Shafer, R. Agrawal, M. Mehta, Sprint: a scalable parallel classifier for data mining, in: Proceedings of VLDB'96, 1996, pp. 544-555.
4. W. Fan, H. Wang, P.S. Yu, S. Ma, Is random model better? On its accuracy and efficiency, in: Proceedings of ICDM'03, 2003, pp. 51-58.
5. G. Hulten, L. Spencer, P. Domingos, Mining time-changing data streams, in: Proceedings of KDD'01, 2001, pp. 97-106.
6. W.N. Street, A streaming ensemble algorithm (SEA) for large-scale classification, in: Proceedings of KDD'01, 2001, pp. 377-382.
7. H. Wang, W. Fan, P.S. Yu, J. Han, Mining concept-drifting data streams using ensemble classifiers, in: Proceedings of KDD'03, 2003, pp. 226-235.
8. M. Woźniak, A. Kasprzak, P. Cal, Weighted aging classifier ensemble for the incremental drifted data streams, in: Proceedings of Flexible Query Answering Systems (FQAS'13), 2013, pp. 579-588.
9. J.Z. Kolter, M.A. Maloof, Using additive expert ensembles to cope with concept drift, in: Proceedings of ICML'05, 2005, pp. 449-456.
10. K.J. Zico, M.A. Maloof, Dynamic weighted majority: an ensemble method for drifting concepts, J. Mach. Learn. Res. 8 (2007) 2755-2790.
11. Gama J., Fernandes R., Rocha R. Decision trees for mining data streams. Intell. Data Anal. 2006, 10:23-45.
12. P. Domingos, G. Hulten, Mining high-speed data streams, in: Proceedings of KDD'00, 2000, pp. 71-80.
13. Scholz M., Klinkenberg R. Boosting classifiers for drifting concepts. Intell. Data Anal. 2007, 11:3-28.
14. A. Bifet, G. Holmes, B. Pfahringer, R. Kirkby, R. Gavaldà, New ensemble methods for evolving data streams, in: Proceedings of KDD'09, 2009, pp. 139-148.
15. A. Bifet, G. Holmes, B. Pfahringer, E. Frank, Fast perceptron decision tree learning from evolving data streams, in: Proceedings of PAKDD'10, 2010, pp. 299-310.
16. Pavlidis N.G., Tasoulis D.K., Adams N.M., Hand D.J. λ-perceptron. an adaptive classifier for data streams. Pattern Recognit. 2011, 44:78-96.
17. J. Gama, P. Kosina, Learning decision rules from data streams, in: Proceedings of IJCAI'11, 2011, pp. 1255-1260.
18. P. Zhang, B.J. Gao, X. Zhu, L. Guo, Enabling fast lazy learning for data streams, in: Proceedings of ICDM'11, 2011, pp. 932-941.
19. H. Yang, S. Fong, Incrementally optimized decision tree for noisy big data, in: Proceedings of BigMine Workshop of ACM SIGKDD, 2012, pp. 36-44.
20. Jackowski K. Fixed-size ensemble classifier system evolutionarily adapted to a recurring context with an unlimited pool of classifiers. Pattern Analysis and Applications 2013, 1-16. 10.1007/s10044-013-0318-x.
21. W. Fan, Streamminer: a classifier ensemble-based engine to mine concept-drifting data streams, in: Proceedings of VLDB'04, 2004, pp. 1257-1260.
22. H. Abdulsalam, D.B. Skillicorn, P. Martin, Streaming random forests, in: Proceedings of IDEAS'07, 2007, pp. 225-232.
23. Breiman L. Random forests. Mach. Learn. 2001, 45:5-32.
24. H. Abdulsalam, D.B. Skillicorn, P. Martin, Classifying evolving data streams using dynamic streaming random forests, in: Proceedings of DEXA'08, 2008, pp. 643-651.
25. Hu X., Li P., Wu X., Wu G. A semi-random multiple decision-tree algorithm for mining data streams. J. Comput. Sci. Technol. 2007, 22:711-724.
26. P. Li, X. Hu, X. Wu, Mining concept-drifting data streams with multiple semi-random decision trees, in: Proceedings of ADMA'08, 2008, pp. 733-740.
27. Li P., Wu X., Hu X., Liang Q., Gao Y. A random decision tree ensemble for mining concept drifts from noisy data streams. Appl. Artif. Intell. 2010, 24:680-710.
28. P. Li, X. Wu, Q. Liang, X. Hu, Y. Zhang, Random ensemble decision trees for learning concept-drifting data streams, in: Proceedings of PAKDD'11, 2011, pp. 313-325.
29. W. Fan, On the optimality of probability estimation by random decision trees, in: Proceedings of AAAI'04, 2004, pp. 336-341.
30. Hoeffding W. Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc. 1963, 58:13-30.
31. Y. Yang, X. Wu, X. Zhu, Combining proactive and reactive predictions for data streams, in: Proceedings of KDD'05, 2005, pp. 710-715.
32. T.K. Ho, Random decision forests, in: Proceedings of ICDAR'95, 1995, pp. 278-282.
33. Amit Y., Geman D. Shape quantization and recognition with randomized trees. Neural Comput. 1997, 9:1545-1588.
34. Ho T.K. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 1998, 20:832-844.
35. Abdulsalam H., Skillicorn D.B., Martin P. Classification using streaming random forests. IEEE Trans. Knowl. Data Eng. 2011, 23:22-36.
36. A. Wang, G. Wan, Z. Cheng, S. Li, An incremental extremely random forest classifier for online learning and tracking, in: Proceedings of ICIP'09, 2009, pp. 1433-1436.
37. G. Widmer, M. Kubat, Learning flexible concepts from streams of examples: Flora2, in: Proceedings of ECAI'92, 1992, pp. 463-467.
38. J.a. Gama, G. Castillo, Learning with local drift detection, in: Proceedings of ADMA'06, 2006, pp. 42-55.
39. J.C. Schlimmer, R.H. Granger Jr., Incremental learning from noisy data, Mach. Learn. 1 (1986) 317-354.
40. Bifet A., Holmes G., Kirkby R., Pfahringer B. MOA: Massive online analysis. Mach. Learn. Res. 2010, 11:1601-1604.
41. Wu X., Li P., Hu X. Learning from concept drifting data streams with unlabeled data. Neurocomputing 2012, 92:145-155.
42. Li P., Wu X., Hu X. Mining recurring concept drifts with limited labeled streaming data. ACM Trans. Intell. Syst. Technol. 2012, 3:29-1-32.
43. P. Li, Q. Liang, X. Wu, X. Hu, Parameter estimation in semi-random decision tree ensembling on streaming data, in: Proceedings of PAKDD'09, 2009, pp. 376-388.
44. Montgomery D.C. Introduction to Statistical Quality Control 2004, John Wiley & Sons Inc., New York.
45. G. Castillo, J. Gama, P. Medas, Adaptation to drifting concepts, in: Proceedings of EPAI03, 2003, p. 279-293.
46. J. Gama, P. Medas, G. Castillo, P. Rodrigues, Learning with drift detection, in: Proceedings of SBIA Brazilian Symposium on Artificial Intelligence, 2004, pp. 286-295.
47. J. Gama, P. Medas, P. Rodrigues, Learning decision trees from dynamic data streams, in: Proceedings of SAC'05, 2005, pp. 573-577.
48. Dümbgen L. P-values for classification. Electron. J. Stat. 2008, 2:468-493.
49. F.T. Liu, K.M. Ting, W. Fan, Maximizing tree diversity by building complete-random decision trees, in: Proceedings of PAKDD'05, 2005, pp. 605-610.
50. Angluin D., Laird P. Learning from noisy examples. Mach. Learn. 1988, 2:343-370.
51. S. Shalev-Shwartz, Y. Singer, N. Srebro, Pegasos: primal estimated sub-gradient solver for SVM, in: Proceedings of ICML'07, 2007, pp. 807-814.
52. J. Gama, R. Rocha, P. Medas, Accurate decision trees for mining high-speed data streams, in: Proceedings of KDD'03, 2003, pp. 523-528.
53. N.C. Oza, S. Russell, Online bagging and boosting, in: Artificial Intelligence and Statistics 2001, Morgan Kaufmann, 2001, pp. 105-112.
54. R. Pelossof, I. Vovsha, M. Jones, C. Rudin, Online coordinate boosting, in: Proceedings of ICCV Workshops, 2009, pp. 1354-1361.
55. M. Baena-Garća, J. del Campo-Avila, R. Fidalgo, A. Bifet, R. Gavaldà, R. Morales-Bueno, Early drift detection method, in: Proceedings of the Fourth International Workshop on Knowledge Discovery from Data Streams, 2006.
56. Demšar J. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 2006, 7:1-30.
57. Zhang M., Wu L. Lift: Multi-label learning with label-specific features. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37:107-120.
58. Demšar J. Multiple comparisons among means. J. Am. Stat. Assoc. 1961, 56:52-64.
59. J. Gama, R. Sebastião, P.P. Rodrigues, Issues in evaluation of stream learning algorithms, in: Proceedings of KDD'09, 2009, pp. 329-338.
60. Yahoo! Shopping Web Services, . http://developer.yahoo.com/everything.html.
61. Rutkowski L., Pietruczuk L., Duda P., Jaworski M. Decision trees for mining data streams based on the McDiarmid's bound. IEEE Trans. Knowl. Data Eng. 2013, 25:1272-1279.
62. Gama J., Kosina P. Recurrent concepts in data streams classification. Knowledge and Information Systems 2014, 40(3):489-507.
63. Wozniak M., Hybrid A. Decision Tree Training Method using Data Streams. Knowledge and Information Systems 2011, 29(2):335-347.