DDD: A new ensemble approach for dealing with concept drift

IEEE Transactions on Knowledge and Data Engineering

Volumn 24, Issue 4, 2012, Pages 619-633

  Minku, Leandro L ^a   Yao, Xin ^a  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

Author keywords

Concept drift; diversity; ensembles of learning machines; online learning

Indexed keywords

CONCEPT DRIFTS; DIVERSITY; DIVERSITY LEVEL; ENSEMBLE LEARNING APPROACH; ENSEMBLES OF LEARNING MACHINES; EXPERIMENTAL COMPARISON; FALSE POSITIVE; LEARNING MACHINES; ONLINE LEARNING; ONLINE LEARNING ALGORITHMS;

LEARNING ALGORITHMS; LEARNING SYSTEMS;

E-LEARNING;

EID: 84857738059     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2011.58     Document Type: Article

References (40)

1. N.C. Oza and S. Russell, "Experimental Comparisons of Online and Batch Versions of Bagging and Boosting," Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 359-364, 2001.
2. A. Fern and R. Givan, "Online Ensemble Learning: An Empirical Study," Machine Learning, vol. 53, pp. 71-109, 2003.
3. R. Polikar, L. Udpa, S.S. Udpa, and V. Honavar, "Learn++: An Incremental Learning Algorithm for Supervised Neural Networks," IEEE Trans. Systems, Man, and Cybernetics-Part C, vol. 31, no. 4, pp. 497-508, Nov. 2001. (Pubitemid 34191829)
4. F.L. Minku, H. Inoue, and X. Yao, "Negative Correlation in Incremental Learning," Natural Computing J., Special Issue on nature-Inspired Learning and Adaptive Systems, vol. 8, no. 2, pp. 289-320, 2009.
5. H. Abdulsalam, D.B. Skillicorn, and P. Martin, "Streaming Random Forests," Proc. Int'l Database Eng. and Applications Symp. (IDEAS), pp. 225-232, 2007.
6. A. Narasimhamurthy and L.I. Kuncheva, "A Framework for Generating Data to Simulate Changing Environments," Proc. 25th IASTED Int'l Multi-Conf.: Artificial Intelligence and Applications, pp. 384-389, 2007.
7. J. Gama, P. Medas, G. Castillo, and P. Rodrigues, "Learning with Drift Detection," Proc. Seventh Brazilian Symp. Artificial Intelligence (SBIA '04), pp. 286-295, 2004. (Pubitemid 39751638)
8. J. Gao, W. Fan, and J. Han, "On Appropriate Assumptions to Mine Data Streams: Analysis and Practice," Proc. IEEE Int'l Conf. Data Mining (ICDM), pp. 143-152, 2007.
9. F.L. Minku, A. White, and X. Yao, "The Impact of Diversity on On-Line Ensemble Learning in the Presence of Concept Drift," IEEE Trans. Knowledge and Data Eng., vol. 22, no. 5, pp. 730-742, http:// dx.doi.org/10.1109/TKDE.2009.156, May 2010.
10. N. Littlestone and M.K. Warmuth, "The Weighted Majority Algorithm," Information and Computation, vol. 108, pp. 212-261, 1994.
11. N. Kasabov, Evolving Connectionist Systems. Springer, 2003.
12. M. Baena-García, J. Del Campo-Avila, R. Fidalgo, and A. Bifet, "Early Drift Detection Method," Proc. Fourth ECML PKDD Int'l Workshop Knowledge Discovery from Data Streams (IWKDDS '06), pp. 77-86, 2006.
13. A. Dawid and V. Vovk, "Prequential Probability: Principles and Properties," Bernoulli, vol. 5, no. 1, pp. 125-162, 1999.
14. W. Street and Y. Kim, "A Streaming Ensemble Algorithm (SEA) for Large-Scale Classification," Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 377-382, 2001.
15. H. Wang, W. Fan, P.S. Yu, and J. Han, "Mining Concept-Drifting Data Streams Using Ensemble Classifiers," Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 226-235, 2003.
16. F. Chu and C. Zaniolo, "Fast and Light Boosting for Adaptive Mining of Data Streams," Proc. Pacific-Asia Conf. Knowledge Discovery and Data Mining (PAKDD '04), pp. 282-292, 2004. (Pubitemid 38824911)
17. M. Scholz and R. Klinkenberg, "An Ensemble Classifier for Drifting Concepts," Proc. Second Int'l Workshop Knowledge Discovery from Data Streams, pp. 53-64, 2005.
18. M. Scholz and R. Klinkenberg, "Boosting Classifiers for Drifting Concepts," Intelligent Data Analysis, Special Issue on Knowledge Discovery from Data Streams, vol. 11, no. 1, pp. 3-28, 2007.
19. S. Ramamurthy and R. Bhatnagar, "Tracking Recurrent Concept Drift in Streaming Data Using Ensemble Classifiers," Proc. Int'l Conf. Machine Learning and Applications (ICMLA '07), pp. 404-409, 2007.
20. J. Gao, W. Fan, J. Han, and P. Yu, "A General Framework for Mining Concept-Drifting Data Streams with Skewed Distributions," Proc. SIAM Int'l Conf. Data Mining (ICDM), 2007.
21. H. He and S. Chen, "IMORL: Incremental Multiple-Object Recognition and Localization," IEEE Trans. Neural Networks, vol. 19, no. 10, pp. 1727-1738, Oct. 2008.
22. K. Nishida and K. Yamauchi, "Adaptive Classifiers-Ensemble System for Tracking Concept Drift," Proc. Sixth Int'l Conf. Machine Learning and Cybernetics (ICMLC '07), pp. 3607-3612, 2007.
23. K. Nishida and K. Yamauchi, "Detecting Concept Drift Using Statistical Testing," Proc. 10th Int'l Conf. Discovery Science (DS '07), pp. 264-269, 2007.
24. K. Nishida, "Learning and Detecting Concept Drift," PhD dissertation, Hokkaido Univ., http://lis2.huie.hokudai.ac.jp/ knishida/paper/nishida2008-dissertation .pdf, 2008.
25. K.O. Stanley, "Learning Concept Drift with a Commitee of Decision Trees," Technical Report AI-TR-03-302, Dept. of Computer Sciences, Univ. of Texas, Austin, 2003.
26. J.Z. Kolter and M.A. Maloof, "Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts," J. Machine Learning Research, vol. 8, pp. 2755-2790, 2007.
27. J.Z. Kolter and M.A. Maloof, "Using Additive Expert Ensembles to Cope with Concept Drift," Proc. Int'l Conf. Machine Learning (ICML '05), pp. 449-456, 2005. (Pubitemid 43183365)
28. K. Tumer and J. Ghosh, "Error Correlation and Error Reduction in Ensemble Classifiers," Connection Science, vol. 8, no. 3, pp. 385-404, 1996.
29. L.I. Kuncheva and C.J. Whitaker, "Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy," Machine Learning, vol. 51, pp. 181-207, 2003.
30. E.K. Tang, P.N. Sunganthan, and X. Yao, "An Analysis of Diversity Measures," Machine Learning, vol. 65, pp. 247-271, 2006. (Pubitemid 44451200)
31. G. Yule, "On the Association of Attributes in Statistics," Philosophical Trans. Royal Soc. of London, Series A, vol. 194, pp. 257-319, 1900.
32. J. Schlimmer and R. Granger, "Beyond Incremental Processing: Tracking Concept Drift," Proc. Fifth Nat'l Conf. Artificial Intelligence (AAAI), pp. 502-507, 1986.
33. "The UCI KDD Archive," http://mlr.cs.umass.edu/ml/ databases/kddcup99/kddcup99.html, 1999.
34. M. Harries, "Splice-2 Comparative Evaluation: Electricity Pricing," Technical Report UNSW-CSE-TR-9905, Artificial Intelligence Group, School of Computer Science and Eng., The Univ. of New South Wales, Sydney, 1999.
35. P. Utgoff, N. Berkman, and J. Clouse, "Decision Tree Induction Based on Efficient Tree Restructuring," Machine Learning, vol. 29, no. 1, pp. 5-44, 1997. (Pubitemid 127507172)
36. F.L. Minku and X. Yao, "Using Diversity to Handle Concept Drift in On-Line Learning," Proc. Int'l Joint Conf. Neural Networks (IJCNN '09), pp. 2125-2132, 2009.
37. I.H. Witten and E. Frank, Data Mining-Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann Publishers, 2000.
38. N.C. Oza and S. Russell, "Online Bagging and Boosting," Proc. IEEE Int'l Conf. Systems, Man and Cybernetics, vol. 3, pp. 2340-2345, 2005. (Pubitemid 41674736)
39. F.L. Minku and X. Yao, "On-Line Bagging Negative Correlation Learning," Proc. Int'l Joint Conf. Neural Networks (IJCNN '08), pp. 1375-1382, 2008.
40. S. Kotsiantis, D. Kanellopoulos, and P. Pintelas, "Handling Imbalanced Datasets: A Review," GESTS Int'l Trans. Computer Science and Eng., vol. 30, no. 1, pp. 25-36, 2006.