Issues in evaluation of stream learning algorithms

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2009, Pages 329-337

  Gama, João ^a   Sebastião, Raquel ^a   Rodrigues, Pedro Pereira ^a  

^a UNIVERSITY OF PORTO   (Portugal)

Author keywords

Concept drift; Data streams; Evaluation design

Indexed keywords

CHANGE DETECTION; CONCEPT DRIFT; DATA STREAM; DATA STREAMS; DECISION MODELS; ERROR ESTIMATES; ERROR ESTIMATORS; EVALUATION DESIGN; FADING FACTORS; HYPOTHESIS TESTING; ILLUSTRATIVE EXAMPLES; MCNEMAR; NON-STATIONARY ENVIRONMENT; PAGE-HINKLEY; PERFORMANCE ASSESSMENT; RESEARCH AREAS; RESOURCE AWARE; SEQUENTIAL METHODS; SLIDING WINDOW; STREAMING APPLICATIONS;

DATA COMMUNICATION SYSTEMS; DATA MINING; DECODING; EDUCATION; ERROR ANALYSIS; ERROR DETECTION; ESTIMATION; LEARNING SYSTEMS; MODELS; WINDOWS;

LEARNING ALGORITHMS;

EID: 70350664414     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1557019.1557060     Document Type: Conference Paper

References (28)

1. B. Babcock, M. Datar, R. Motwani, and L. O'Callaghan. Maintaining variance and k-medians over data stream windows. In Proc. of the 22nd Symposium on Principles of Database Systems, pages 234-243. ACM Press, 2003.
2. Michele Basseville and Igor Nikiforov. Detection of Abrupt Changes: Theory and Applications. Prentice-Hall Inc, 1987.
3. C. Blake, E. Keogh, and C.J. Merz. UCI repository of Machine Learning Databases, 1999.
4. Gladys Castillo and Jõao Gama. Bias management of bayesian network classifiers. In A. Hoffmann, H. Motoda, and T. Scheffer, editors, Discovery Science, Proceedings of 8th International Conference, pages 70-83. LNAI 3735, Springer Verlag, 2005.
5. H. Chernoff. A measure of asymptotic eficiency for tests of a hypothesis based on the sums of observations. Annals of Mathematical Statistics, 23:493-507, 1952.
6. Graham Cormode, S. Muthukrishnan, and Wei Zhuang. Conquering the divide: Continuous clustering of distributed data streams. In ICDE, pages 1036-1045, 2007.
7. A. P. Dawid. Statistical theory: The prequential approach. Journal of the Royal Statistical Society-A, 147:278-292, 1984.
8. Janez Demsar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1-30, 2006.
9. T. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Corvallis, technical report nr. 97.331, Oregon State University, 1996.
10. Pedro Domingos and Geoff Hulten. Mining High-Speed Data Streams. In Ismail Parsa, Raghu Ramakrishnan, and Sal Stolfo, editors, Proceedings of the ACM Sixth International Conference on Knowledge Discovery and Data Mining, pages 71-80. ACM Press, 2000.
11. F.J. Ferrer-Troyano, J.S. Aguilar-Ruiz, and J.C. Riquelme. Incremental rule learning and border examples selection from numerical data streams. Journal of Universal Computer Science, 11(8):1426-1439, 2005.
12. Francisco Ferrer-Troyano, Jesus S. Aguilar-Ruiz, and Jose C. Riquelme. Discovering decision rules from numerical data streams. In Proceedings of the 2004 ACM symposium on Applied computing, pages 649-653. ACM Press, 2004.
13. João Gama, Pedro Medas, Gladys Castillo, and Pedro Rodrigues. Learning with drift detection. In Ana L. C. Bazzan and Soffane Labidi, editors, Advances in Artificial Intelligence - SBIA 2004, volume 3171 of Lecture Notes in Computer Science, pages 286-295. Springer Verlag, October 2004.
14. João Gama, Pedro Medas, and Ricardo Rocha. Forest trees for on-line data. In Proceedings of the 2004 ACM symposium on Applied computing, pages 632-636. ACM Press, 2004.
15. João Gama, Ricardo Rocha, and Pedro Medas. Accurate decision trees for mining high-speed data streams. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 523-528. ACM Press, 2003.
16. B. Ghosh and P. Sen. Handbook of Sequential Analysis. Narcel Dekker, 1991.
17. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.
18. Geoff Hulten and Pedro Domingos. Catching up with the data: research issues in mining data streams. In Proc. of Workshop on Research issues in Data Mining and Knowledge Discovery, 2001.
19. Geoff Hulten and Pedro Domingos. VFML - a toolkit for mining high-speed time-changing data streams. http://www.cs.washington.edu/dm/vfml/.2003.
20. Geoff Hulten, Laurie Spencer, and Pedro Domingos. Mining time-changing data streams. In Proceedings of the 7th ACM SIGKDD International conference on Knowledge discovery and data mining, pages 97-106. ACM Press, 2001.
21. Richard Kirkby. Improving Hoeffding Trees. PhD thesis, University of Waikato - New Zealand, 2008.
22. Ralf Klinkenberg. Learning drifting concepts: Example selection vs. example weighting. Intelligent Data Analysis, 8(3):281-300, 2004.
23. I. Koychev. Gradual forgetting for adaptation to concept drift. In Proceedings of ECAI 2000 Workshop Current Issues in Spatio-Temporal Reasoning. Berlin, Germany, pages 101-106, 2000.
24. I. Mierswa, M. Wurst, R. Klinkenberg, M. Scholz, and T. Euler. Yale: Rapid prototyping for complex data mining tasks. In ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pages 935-940. ACM Press, 2006.
25. H. Mouss, D. Mouss, N. Mouss, and L. Sefouhi. Test of Page-Hinkley, an approach for fault detection in an agro-alimentary production system. In Proceedings of the 5th Asian Control Conference, volume 2, pages 815-818, 2004.
26. E. S. Page. Continuous inspection schemes. Biometrika, 41(1/2):100-115, 1954.
27. W. Nick Street and YongSeog Kim. A streaming ensemble algorithm SEA for large-scale classification. In Proc. seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 377-382. ACM Press, 2001.
28. Gerhard Widmer and Miroslav Kubat. Learning in the presence of concept drift and hidden contexts. Machine Learning, 23:69-101, 1996.