Compose: A semisupervised learning framework for initially labeled nonstationary streaming data

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 1, 2014, Pages 12-26

  Dyer, Karl B ^a   Capo, Robert ^a   Polikar, Robi ^a  

^a Rowan University   (United States)

Author keywords

Alpha shape; concept drift; nonstationary environment; semisupervised learning (SSL); verification latency

Indexed keywords

ALPHA-SHAPES; CONCEPT DRIFTS; NON-STATIONARY ENVIRONMENT; SEMI-SUPERVISED LEARNING (SSL); VERIFICATION LATENCIES;

COMPUTATIONAL GEOMETRY; SUPERVISED LEARNING; VIRTUAL REALITY;

LEARNING ALGORITHMS;

EID: 84891150042     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2277712     Document Type: Article

References (55)

1. C. Alippi and M. Roveri, "Just-in-time adaptive classifiers-Part I: Detecting nonstationary changes," IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1145-1153, Jul. 2008.
2. A. Bifet, "Adaptive learning and mining for data streams and frequent patterns," Ph.D. dissertation, Dept. Llenguatges Sistemes Inf., Univ. Politecnica Catalunya, Barcelona, Spain, 2009.
3. S. Chen and H. He, "Towards incremental learning of nonstationary imbalanced data stream: A multiple selectively recursive approach," Evolv. Syst., vol. 2, no. 1, pp. 35-50, 2011.
4. R. Elwell and R. Polikar, "Incremental learning of concept drift in nonstationary environments," IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1517-1531, Oct. 2011.
5. H. Haibo, C. Sheng, L. Kang, and X. Xin, "Incremental learning from stream data," IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 1901-1914, Dec. 2011.
6. J. Kolter and M. Maloof, "Dynamic weighted majority: An ensemble method for drifting concepts," J. Mach. Learn. Res., vol. 8, pp. 2755-2790, Jul. 2007.
7. J. Quinonero-Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence, Dataset Shift in Machine Learning. Cambridge, MA, USA: MIT Press, 2009.
8. P. P. Rodrigues, J. Gama, and J. P. Pedroso, "Hierarchical clustering of time-series data streams," IEEE Trans. Knowl. Data Eng., vol. 20, no. 5, pp. 615-627, May 2008.
9. W. N. Street and Y. Kim, "A streaming ensemble algorithm (SEA) for large-scale classification," in Proc. 7th ACM SIGKDD Int. Conf. KDD, 2001, pp. 377-382.
10. A. Tsymbal, M. Pechenizkiy, P. Cunningham, and S. Puuronen, "Dynamic integration of classifiers for handling concept drift," Inf. Fusion, vol. 9, no. 1, pp. 56-68, Jan. 2008. (Pubitemid 47589061)
11. G. Widmer and M. Kubat, "Learning in the presence of concept drift and hidden contexts," Mach. Learn., vol. 23, no. 1, pp. 69-101, 1996. (Pubitemid 126737384)
12. I. Zliobaite, "Combining similarity in time and space for training set formation under concept drift," Intell. Data Anal., vol. 15, no. 4, pp. 589-611, 2011.
13. G. Marrs, R. Hickey, and M. Black, "The impact of latency on online classification learning with concept drift," in Knowledge Science, Engineering and Management, vol. 6291, Y. Bi and M. A. Williams, Eds. Berlin Heidelberg, Germany: Springer-Verlag, 2010, pp. 459-469.
14. H. Daume and D. Marcu, "Domain adaptation for statistical classiers," J. Artif. Intell. Res., vol. 26, pp. 101-126, Jun. 2006.
15. S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira, "Analysis of representations for domain adaptation," in Advances in Neural Information Processing Systems, vol. 19, B. Scholkopf, J. Platt, and T. Hoffman, Eds. Cambridge, MA, USA: MIT Press, 2007, pp. 137-144.
16. J. P. Sinno and Y. Qiang, "A survey on transfer learning," IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345-1359, Oct. 2010.
17. H. Shimodaira, "Improving predictive inference under covariate shift by weighting the log-likelihood function," J. Stat. Planning Inference, vol. 90, no. 2, pp. 227-244, 2000.
18. R. Caruana, "Multitask learning," Mach. Learn., vol. 28, no. 41, pp. 41-75, 1997. (Pubitemid 127507169)
19. J. J. Heckman, "Sample selection bias as a specification error," Econometrica, vol. 47, no. 1, pp. 153-161, 1979.
20. M. Sugiyama, M. Krauledat, and K.-R. Muller, "Covariate shift adaptation by importance weighted cross validation," J. Mach. Learn. Res., vol. 8, pp. 985-1005, May 2007. (Pubitemid 46798411)
21. J. Huang, A. J. Smola, A. Gretton, K. M. Borgwardt, and B. Scholkopf, "Correcting sample selection bias by unlabeled data," in Advances in Neural Information Processing Systems, vol. 19, B. Scholkoph, J. Platt, and T. Hoffman, Eds. Cambridge, MA, USA: MIT Press, 2007, pp. 601-608.
22. J. Blitzer, R. McDonald, and F. Pereira, "Domain adaptation with structural correspondence learning," in Proc. Conf. EMNLP, 2006, pp. 120-128.
23. S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan, "A theory of learning from different domains," Mach. Learn., vol. 79, nos. 1-2, pp. 151-175, Oct. 2009.
24. A. Margolis, "Automatic annotation of spoken language using out-ofdomain resources and domain adaptation," Ph.D. dissertation, Dept. Comput. Sci., Univ. Washington, Seattle, WA, USA, 2011.
25. O. Chapelle, B. Scholkopf, and A. Zien, Semi-Supervised Learning. Cambridge, MA, USA: MIT Press, 2006.
26. K. Chen and S. Wang, "Semi-supervised learning via regularized boosting working on multiple semi-supervised assumptions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 1, pp. 129-143, Jan. 2011.
27. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. R. Stat. Soc. Ser. B, Methodol., vol. 39, no. 1, pp. 1-38, 1977.
28. X. Zhu and A. Goldberg, Introduction to Semi-Supervised Learning. San Rafael, CA, USA: Morgan & Claypool, 2009, pp. 31-32.
29. T. Joachims, "Transductive inference for text classification using support vector machines," in Proc. 16th Int. Conf. Mach. Learn., 1999, pp. 200-209.
30. V. Vapnik and A. Sterin, "On structural risk minimization for overall risk in a problem of pattern recognition," Autom. Remote Control, vol. 10, no. 3, pp. 1495-1503, 1977.
31. D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Scholkopf, "Learning with local and global consistency," in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2004, pp. 321-328.
32. X. Zhu and Z. Ghahramani, "Learning from labeled and unlabeled data with label propagation," School Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. CMU-CALD-02-107, 2002.
33. M. Masud, J. Gao, L. Khan, J. Han, and X. Li, "Classification and novel class detection in data streams with active mining," in Proc. Pacific-Asia Conf. Knowl. Discovery Data Mining, 2010, pp. 311-324.
34. A. Goldberg, X. Zhu, A. Furger, and J. Xu, "OASIS: Online active semi-supervised learning," in Proc. 25th Conf. Artif. Intell., 2011, pp. 1-6.
35. I. Zliobaite, A. Bifet, B. Pfahringer, and G. Holmes, "Active learning with evolving streaming data," in Machine Learning and Knowledge Discovery in Databases, vol. 6913. Berlin Heidelberg, Germany: Springer- Verlag, 2011, pp. 597-612.
36. R. Capo, K. Dyer, and R. Polikar, "Active learning in nonstationary environments," in Proc. IJCNN, Houston, TX, USA, 2013.
37. A. Kuh, T. Petsche, and R. L. Rivest, "Learning time-varying concepts," in Advances in Neural Information Processing Systems, Denver, CO, USA: MIT Press, 1990, pp. 183-189.
38. L. L. Minku, A. P. White, and X. Yao, "The impact of diversity on online ensemble learning in the presence of concept drift," IEEE Trans. Knowl. Data Eng., vol. 22, no. 5, pp. 730-742, May 2010.
39. A. Goldberg, M. Li, and X. Zhu, "Online manifold regularization: A new learning setting and empirical study," Computer, vol. 5211, no. 1, pp. 393-407, 2008.
40. P. Li, X. Wu, and X. Hu, "Mining recurring concept drifts with limited labeled streaming data," in Proc. 2nd Asian Conf. Mach. Learn., Tokyo, Japan, 2010, pp. 241-252.
41. G. Ditzler and R. Polikar, "Semi-supervised learning in nonstationary environments," in Proc. Int. Joint Conf. Neural Netw., 2011, pp. 2741-2748.
42. M. Masud, J. Gao, L. Khan, and J. Han, "A practical approach to classify evolving data streams: Training with limited amount of labeled data," in Proc. IEEE 8th Int. Conf. Data Mining, Washington, DC, USA, Dec. 2008, pp. 929-934.
43. P. Zhang, X. Zhu, and L. Guo, "Mining data streams with labeled and unlabeled training examples," in Proc. IEEE 9th Int. Conf. Data Mining, Miami, FL, USA, Dec. 2009, pp. 627-636.
44. I. Katakis, G. Tsoumakas, and I. Vlahavas, "Tracking recurring contexts using ensemble classifiers: An application to email filtering," Knowl. Inf. Syst., vol. 22, no. 3, pp. 371-391, 2010.
45. P. Zhang, X. Zhu, J. Tan, and L. Guo, "Classifier and cluster ensembles for mining concept drifting data streams," in Proc. IEEE 10th Int. Conf. Data Mining, Dec. 2010, pp. 1175-1180.
46. G. Krempl and V. Hofer, "Classification in presence of drift and latency," in Proc. IEEE Int. Conf. Data Mining Workshops, Dec. 2011, pp. 596-603.
47. R. Alaiz-Rodriguez, A. Guerrero-Curieses, and J. Cid-Sueiro, "Class and subclass probability re-estimation to adapt a classifier in the presence of concept drift," Neurocomputing, vol. 74, no. 16, pp. 2614-2623, Sept.2011.
48. V. Hofer and G. Krempl, "Drift mining in data: A framework for addressing drift in classification," Comput. Stat. Data Anal., vol. 57, no. 1, pp. 377-391, Jan. 2013.
49. G. Krempl, "The algorithm APT to classify in concurrence of latency and drift," in Advances in Intelligent Data Analysis (Lecture Notes in Computer Science), vol. 7014, J. Gama, E. Bradley, and J. Hollmen, Eds. Berlin Heidelberg, Germany: Springer-Verlag, 2011, pp. 222-233.
50. K. Dyer and R. Polikar, "Semi-supervised learning in initially labeled non-stationary environments with gradual drift," in Proc. IJCNN, 2012, pp. 1-9.
51. M. Teichmann and M. Capps, "Surface reconstruction with anisotropic density-scaled alpha shapes," in Proc. IEEE Visualizat., Oct. 1998, pp. 67-72.
52. C. Barber, D. Dobkin, and H. Huhdanpaa, "The QuickHull algorithm for convex hulls," ACM Trans. Math. Softw., vol. 22, no. 4, pp. 469-483, Dec. 1996. (Pubitemid 126417394)
53. D. Pedoe, Circles: A Mathematical View, 2nd ed. New York, NY, USA: Dover, 1979.
54. H. Edelsbrunner and E. Mucke, "Three-dimensional alpha shapes," ACM Trans. Graph., vol. 13, no. 1, pp. 43-72, 1994.
55. R. Elwell and R. Polikar, "Incremental learning in nonstationary environments with controlled forgetting," in Proc. IJCNN, 2009, pp. 771-778.