Solving nonstationary classification problems with coupled upport vector machines

IEEE Transactions on Neural Networks

Volumn 22, Issue 1, 2011, Pages 37-51

  Grinblat, Guillermo L ^a   Uzal, Lucas C ^a   Ceccatto, H Alejandro ^a   Granitto, Pablo M ^a  

^a UNIVERSIDAD NACIONAL DE ROSARIO   (Argentina)

Author keywords

Adaptive methods; drifting concepts; support vector machine

Indexed keywords

ADAPTIVE CLASSIFIERS; ADAPTIVE METHODS; ADAPTIVE SUPPORT; BASIC IDEA; DATA SETS; DRIFTING CONCEPTS; ELECTRICITY PRICING; INPUT-OUTPUT FUNCTION; LEARNING METHODS; LEARNING PROBLEM; NONSTATIONARY; REAL-WORLD; REAL-WORLD APPLICATION; TIME WINDOWS; VECTOR MACHINES;

CLASSIFIERS; SUPPORT VECTOR MACHINES; VECTORS;

COUPLINGS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER PROGRAM; COMPUTER SIMULATION; PROBLEM SOLVING; SOLUTION AND SOLUBILITY; STANDARD;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; PROBLEM SOLVING; SOFTWARE; SOLUTIONS;

EID: 78651305965     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2083684     Document Type: Article

References (55)

1. J. C. Schlimmer and R.H. Granger, "Beyond incremental processing: Tracking concept drift," in Proc. 5th Nat. Conf. Artif. Intell., Irvine, CA, 1986, pp. 502-507.
2. R. Klinkenberg and T. Joachims, "Detecting concept drift with support vector machines," in Proc. 17th Int. Conf. Mach. Learn., San Mateo, CA, 2000, pp. 487-494.
3. R. Vicente, O. Kinouchi, and N. Caticha, "Statistical mechanics of online learning of drifting concepts: A variational approach," Mach. Learn., vol. 32, no. 2, pp. 179-201, Aug. 1998.
4. P. L. Bartlett, S. Ben-Dabid, and S. R. Kulkarni, "Learning changing concepts by exploiting the structure of change," Mach. Learn., vol. 41, no. 2, pp. 153-174, Nov. 2000.
5. K. Stanley, "Learning concept drift with a committee of decision trees," Dept. Comput. Sci., Univ. Texas, Austin, Tech. Rep. UTAI-TR-03-302, 2003.
6. A. Tsymbal, "The problem of concept drift: Definitions and related work," Dept. Comput. Sci., Trinity College Dublin, Dublin, Ireland, Tech. Rep. TCD-CS-2004-15, Apr. 2004.
7. T. Mitchell, R. Caruana, D. Freitag, J. McDermott, and D. Zabowski, "Experience with a learning personal assistant," Commun. ACM, vol. 37, no. 7, pp. 81-91, 1994.
8. G. Widmer and M. Kubat, "Learning in the presence of concept drift and hidden contexts," Mach. Learn., vol. 23, no. 1, pp. 69-101, Apr. 1996.
9. M. Salganicoff, "Tolerating concept and sampling shift in lazy learning using prediction error context switching," Artif. Intell. Rev., vol. 11, nos. 1-5, pp. 133-155, Feb. 1997.
10. W. N. Street and Y. Kim, "A streaming ensemble algorithm (SEA) for large-scale classification," in Proc. 7th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining. San Francisco, CA, 2001, pp. 377-382.
11. S. H. Bach and M. A. Maloof, "Paired learners for concept drift," in Proc. IEEE Int. Conf. Data Mining, Los Alamitos, CA, 2008, pp. 23-32.
12. C. Alippi and M. Roveri, "Just-in-time adaptive classifiers-part I: Detecting nonstationary changes," IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1145-1153, Jul. 2008.
13. C. Alippi and M. Roveri, "Just-in-time adaptive classifiers-part II: Designing the classifier," IEEE Trans. Neural Netw., vol. 19, no. 12, pp. 2053-2064, Dec. 2008.
14. D. P. Helmbold and P. M. Long, "Tracking drifting concepts by minimizing disagreements," Mach. Learn., vol. 14, no. 1, pp. 27-45, Jan. 1994.
15. P. L. Bartlett, "Learning with a slowly changing distribution," in Proc. 5th Annu. Workshop Comput. Learn. Theory, Pittsburgh, PA, 1992, pp. 243-252.
16. R. D. Barve and P. M. Long, "On the complexity of learning from drifting distributions," in Proc. 9th Annu. Workshop Comput. Learn. Theory, San Mateo, CA, 1996, pp. 170-193.
17. Y. Freund and Y. Mansour, "Learning under persistent drift," in Proc. 3rd Eur. Conf. Comput. Learn. Theory, London, U.K., 1997, pp. 109-118.
18. C. Alippi, G. Boracchi, and M. Roveri, "Just in time classifiers: Managing the slow drift case," in Proc. Int. Joint Conf. Neural Netw., Atlanta, GA, 2009, pp. 114-120.
19. R. Klinkenberg and I. Renz, "Adaptive information filtering: Learning in the presence of concept drifts," in Workshop Notes of the ICML/AAAI Workshop Learning for Text Categorization. Menlo Park, CA: AAAI Press, 1998, pp. 33-40.
20. C. Lanquillon, "Enhancing test classification to improve information filtering," Ph.D. thesis, Faculty Comp. Sci., Univ. Magdeburg, Magdeburg, Germany, 2001.
21. K. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf, "An introduction to kernel-based learning algorithms," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 181-201, Mar. 2001.
22. V. Vapnik, "An overview of statistical learning theory," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 988-999, Sep. 1999.
23. G. L. Grinblat, P. M. Granitto, and H. A. Ceccatto, "Time-adaptive support vector machines," Inteligencia Artif., vol. 12, no. 40, pp. 39-50, 2008.
24. M. Karnick, M. Ahiskali, M. D. Muhlbaier, and R. Polikar, "Learning concept drift in nonstationary environments using an ensemble of classifiers based approach," in Proc. IEEE Int. Joint Conf. Neural Netw., Hong Kong, China, Jun. 2008, pp. 3455-3462.
25. J. Z. Kolter and M. A. Maloof, "Dynamic weighted majority: An ensemble method for drifting concepts," J. Mach. Learn. Res., vol. 8, pp. 2755-2790, Dec. 2007.
26. J. C. Schlimmer and R.H. Granger, Jr., "Incremental learning from noisy data," Mach. Learn., vol. 1, no. 3, pp. 317-354, 1986.
27. G. Castillo, J. Gama, and P. Medas, "Adaptation to drifting concepts," in Proc. Progress Artif. Intell., 11th Portuguese Conf. Artif1. Intell. (EPIA), LNCS 2902. Beja, Portugal, 2003, pp. 279-293.
28. R. Klinkenberg, "Learning drifting concepts: Example selection versus example weighting," Intell. Data Anal., vol. 8, no. 3, pp. 281-300, Aug. 2004.
29. T. Joachims, "Estimating the generalization performance of a SVM efficiently," in Proc. 17th Int. Conf. Mach. Learn., San Francisco, CA, 2000, pp. 431-438.
30. I. Koychev and R. Lothian, "Tracking drifting concepts by time window optimization," in Proc. 25th SGAI Int. Conf. Innov. Tech. Appl. Artif. Intell., New York, 2005, pp. 46-59.
31. C. Alippi and M. Roveri, "Just-in-time adaptive classifiers in nonstationary conditions," in Proc. Int. Joint Conf. Neural Netw., Orlando, FL, 2007, pp. 1014-1019.
32. I. Koychev, "Tracking changing user interests through prior-learning of context," in Adaptive Hypermedia, LNCS 2347. New York: Springer- Verlag, 2002, pp. 223-232.
33. M. Maloof and R. Michalski, "Selecting examples for partial memory learning," Mach. Learn., vol. 41, no. 1, pp. 27-52, Oct. 2000.
34. I. Koychev, "Gradual forgetting for adaptation to concept drift," in Proc. ECAI Workshop Current Issues Spatio-Temporal Reason., Berlin, Germany, 2000, pp. 101-106.
35. H. Wang, W. Fan, P. S. Yu, and J. Han, "Mining concept-drifting data streams using ensemble classifiers," in Proc. 9th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Washington D.C., 2003, pp. 226- 235.
36. J. Gao, B. Ding, J. Han, W. Fan, and P. Yu, "Classifying data streams with skewed class distributions and concept drifts," IEEE Internet Comput., vol. 12, no. 6, pp. 37-49, Nov.-Dec. 2008.
37. S. Hashemi, Y. Yang, Z. Mirzamomen, and M. Kangavari, "Adapted one-versus-all decision trees for data stream classification," IEEE Trans. Knowl. Data Eng., vol. 21, no. 5, pp. 624-637, May 2009.
38. R. Elwell and R. Polikar, "Incremental learning in nonstationary environments with controlled forgetting," in Proc. Int. Joint Conf. Neural Netw., Atlanta, GA, 2009, pp. 771-778.
39. M. D. Muhlbaier, A. Topalis, and R. Polikar, "Learn ++.NC: Combining ensemble of classifiers with dynamically weighted consult-and-vote for efficient incremental learning of new classes," IEEE Trans. Neural Netw., vol. 20, no. 1, pp. 152-168, Jan. 2009.
40. Z. Kolter and M. Maloof, "Dynamic weighted majority: A new ensemble method for tracking concept drift," in Proc. 3rd IEEE Int. Conf. Data Mining, Nov. 2003, pp. 123-130.
41. N. Littlestone and M. K. Warmuth, "The weighted majority algorithm," Inform. Comput., vol. 108, no. 2, pp. 212-261, Feb. 1994.
42. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge, U.K.: Cambridge Univ. Press, 2000.
43. J. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods-Support Vector Learning. Cambridge, MA: MIT Press, 2000, pp. 185-208.
44. N. Littlestone, "Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm," Mach. Learn., vol. 2, no. 4, pp. 285- 318, Apr. 1987.
45. S. Ferrari and M. Jensenius, "A constrained optimization approach to preserving prior knowledge during incremental training," IEEE Trans. Neural Netw., vol. 19, no. 6, pp. 996-1009, Jun. 2008.
46. K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer, "Online passive-agressive algorithms," J. Mach. Learn. Res., vol. 7, pp. 551-585, Dec. 2006.
47. J. Kivinen and M. Warmuth, "Exponentiated gradient versus gradient descent for linear predictors," Inform. Comput., vol. 132, no. 1, pp. 1- 63, Jan. 1997.
48. J. Kivinen, A. J. Smola, and R. C. Williamson, "Online learning with kernels," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2165-2176, Aug. 2004.
49. M. Herbster and M. K. Warmuth, "Tracking the best linear predictor," J. Mach. Learn. Res., vol. 1, pp. 281-309, Sep. 2001.
50. R. Caruana, "Multi-task learning," Mach. Learn., vol. 28, no. 1, pp. 41-75, Jul. 1997.
51. S. Thrun and L. Pratt, Learning to Learn. Norwell, MA: Kluwer, 1997.
52. T. Evgeniou, C. M. Micchelli, and M. Pontil, "Learning multiple tasks with kernel methods," J. Mach. Learn. Res., vol. 6, pp. 615-637, Dec. 2005.
53. W. Fan, "Systematic data selection to mine concept-drifting data streams," in Proc. 10th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Seattle, WA, 2004, pp. 128-137.
54. A. Tsymbal, M. Pechenizkiy, P. Cunningham, and S. Puuronen, "Dynamic integration of classifiers for handling concept drift," Inform. Fusion, vol. 9, no. 1, pp. 56-68, Jan. 2008.
55. M. Harries, "Splice-2 comparative evaluation: Electricity pricing," School Comput. Sci. & Eng., Univ. New South Wales, Sydney, Australia, Tech. Rep. NSW-CSE-TR-9905, 1999.