Hybrid decision tree architecture utilizing local SVMs for efficient multi-label learning

International Journal of Pattern Recognition and Artificial Intelligence

Volumn 27, Issue 7, 2013, Pages

  Gjorgjevikj, Dejan ^a   Madjarov, Gjorgji ^a   Džeroski, Sašo ^b  

^a SS CYRIL AND METHODIUS UNIVERSITY   (Macedonia)

^b JO EF STEFAN INSTITUTE   (Slovenia)

Author keywords

Architecture; Decision tree; Hybrid; Learning; Multi label; SVM

Indexed keywords

BINARY CLASSIFICATION PROBLEMS; HYBRID; LEARNING; MACHINE LEARNING APPROACHES; MULTI-LABEL; STATE-OF-THE-ART METHODS; SUPPORT VECTOR MACHINE (SVMS); SVM;

ARCHITECTURE; DATA MINING; DATA PROCESSING; DECISION TREES; FORECASTING; LEARNING ALGORITHMS; SEMANTICS; TEXT PROCESSING;

SUPPORT VECTOR MACHINES;

EID: 84888123727     PISSN: 02180014     EISSN: None     Source Type: Journal    
DOI: 10.1142/S021800141351004X     Document Type: Article

References (62)

1. A. Argyriou, T. Evgeniou and M. Pontil, Convex multi-task feature learning, Mach. Learn. 73(3) (2008) 243-272.
2. G. H. Bakir, T. Hofmann, B. Schölkopf, A. J. Smola, B. Taskar and S. V. N. Vishwanathan, Predicting Structured Data (Neural Information Processing) (The MIT Press, 2007).
3. K. P. Bennett and J. A. Blue, A support vector machine approach to decision trees, in Proc. Int. Joint Conf. Neural Networks, Anchorage, Alaska (1997), pp. 2396-2401.
4. H. Blockeel, L. D. Raedt and J. Ramon, Top-down induction of clustering trees, in Proc. 15th Int. Conf. Machine Learning (1998), pp. 55-63.
5. L. Bottou and C.-J. Lin, Support vector machine solvers, in Large Scale Kernel Machines, eds. (L. Bottou, O. Chapelle, D. DeCoste and J. Weston) (MIT Press, 2007), pp. 301-320.
6. M. R. Boutell, J. Luo, X. Shen and C. M. Brown, Learning multi-label scene classification, Pattern Recogn. 37(9) (2004) 1757-1771. (Pubitemid 38804465)
7. K. Brinker, J. Fürnkranz and E. Hüllermeier, A unified model formultilabel classification and ranking, in Proc. 17th European Conf. Artificial Intelligence (2006), pp. 489-493.
8. C. J. C. Burges, A tutorial on support vector machines for pattern recognition, Data Mining Knowl. Discov. 2 (1998) 121-167. (Pubitemid 128695475)
9. R. Caruana, Multitask learning, Mach. Learn. 28(1) (1997) 41-75. (Pubitemid 127507169)
10. C.-C. Chang and C.-J. Lin, LIBSVM: A library for supportvector machines (2001), Available at http://www.csie.ntu.edu.tw/cjlin/libsvm.
11. A. Clare and R. D. King, Knowledge discovery in multi-label phenotype data, in Proc. 5th European Conf. PKDD (2001), pp. 42-53. (Pubitemid 33331849)
12. K. Crammer and Y. Singer, A family of additive online algorithms for category ranking, J. Mach. Learn. Res. 3 (2003) 1025-1058.
13. F. De Comité, R. Gilleron and M. Tommasi, Learning multi-label alternating decision trees from texts and data, in Proc. 3rd Int. Conf. Machine Learning and Data Mining in Pattern Recognition (2003), pp. 35-49. (Pubitemid 36904942)
14. G. M. Dong and J. Chen, Study on support vector machine based decision tree and application, in Proc. 5th Int. Conf. Fuzzy Systems and Knowledge Discovery (2008), pp. 318-322.
15. P. Duygulu, K. Barnard, J. de Freitas and D. Forsyth, Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary, in Proc. 7th European Conf. Computer Vision (2002), pp. 349-354.
16. A. Elissee® and J. Weston, A kernel method for multi-labelled classification, in Proc. Annual ACM Conf. Research and Development in Information Retrieval (2005), pp. 274-281.
17. T. Evgeniou and M. Pontil, Regularized multi-task learning, in Proc. Tenth ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (2004), pp. 109-117. (Pubitemid 40114921)
18. C.-S. Ferng and H.-T. Lin, Multi-label classification with error-correcting codes, J. Mach. Learn. Res. 20 (2011) 281-295.
19. J. Fürnkranz, Round robin classification, J. Mach. Learn. Res. 2 (2002) 721-747. (Pubitemid 135712579)
20. J. Fürnkranz, E. Hullermeier, E. L. Mencia and K. Brinker, Multi-label classification via calibrated label ranking, Mach. Learn. 73(2) (2008) 133-153.
21. J. Gama, Functional trees, Machi. Learn. 55 (2004) 219-250.
22. P. Geurts, L. Wehenkel and F. d'Alché Buc, Kernelizing the output of tree-based methods, in Proc. 23rd Int Conf. Machine Learning (2006), pp. 345-352.
23. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann and I. H. Witten, The weka data mining software: An update, SIGKDD Explorations 11 (2009) 10-18.
24. T. Joachims, T. Hofmann, Y. Yue and C.-N. Yu, Predicting structured objects withsupport vector machines, Commun. ACM, Research Highlight 52 (2009) 97-104.
25. I. Katakis, G. Tsoumakas and I. Vlahavas, Multilabel text classification for automated tag suggestion, in Proc. ECML/PKDD Discovery Challenge (Antwerp, Belgium, 2008), pp. 75-83.
26. B. Klimt and Y. Yang, The Enron Corpus: A new dataset for email classification research, in Proc. 15th European Conf. Machine Learning (2004), pp. 217-226 (Pubitemid 41050094)
27. D. Kocev, Ensembles for predicting structured outputs, Ph.D. thesis, IPS Joáef Stefan, Ljubljana, Slovenia (2011).
28. X. Kong and P. S. Yu, An ensemble-based approach to fast classification of multilabel data streams, in Proc. 7th Int. Conf. Collaborative Computing: Networking, Applications and Worksharing (Orlando, FL, USA, 2011), pp. 95-104.
29. A. M. Kumar and M. Gopal, A hybrid svm based decision tree, Pattern Recogn. 43 (2010) 3977-3987.
30. P. Li, H. Li and M. Wu, Multi-label ensemble based on variable pairwise constraint projection, Inform. Sci. 222 (2013) 269-281.
31. G. Madjarov and D. Gjorgjevikj, Hybrid decision tree architecture utilizing local svms for multi-label classification, in Proc. 7th Int. Conf. Hybrid Artificial Intelligent Systems - Volume Part II, HAIS'12 (2012), pp. 1-12.
32. G. Madjarov, D. Gjorgjevikj and S. Dáeroski, Two stage architecture for multi-label learning, Pattern Recogn. 45(3) (2012) 1019-1034.
33. E. L. Mencía, S.-H. Park and J. Fürnkranz, Eücient voting prediction for pairwise multilabel classification, Neurocomputing 73 (2010) 1164-1176.
34. G. Nasierding, G. Tsoumakas and A. Kouzani, Clustering based multi-label classification for image annotation and retrieval, in Proc. IEEE Int. Conf. Systems, Man and Cybernetics (2009), pp. 4514-4519.
35. S.-H. Park and J. Fürnkranz, E±cient pairwise classification, in Proc. 18th European Conf. Machine Learning (2007), pp. 658-665.
36. J. R. Quinlan, C4.5: Programs for Machine Learning (Morgan Kaufmann Series in Machine Learning), 1st edn. (Morgan Kaufmann, 1992).
37. J. Read, A pruned problem transformation method for multi-label classification, in Proc. New Zealand Computer Science Research Student Conf. (2008), pp. 143-150.
38. J. Read, B. Pfahringer and G. Holmes, Multi-label classifiscation using ensembles of pruned sets, in Proc. 8th IEEE Int. Conf. Data Mining (2008), pp. 995-1000.
39. J. Read, B. Pfahringer, G. Holmes and E. Frank, Classifier chains for multi-label classification, Mach. Learn. 85(3) (2011) 333-359.
40. R. E. Schapire and Y. Singer, Boostexter: A boosting-based system for text categorization, Mach. Learn. 39 (2000) 135-168. (Pubitemid 30594821)
41. C. N. Silla Jr. and A. A. Freitas, A survey of hierarchical classification across di®erent application domains, Data Min. Knowl. Discov. 22(1-2) (2011) 31-72.
42. C. G. M. Snoek, M. Worring, J. C. Van Gemert, J.-M. Geusebroek and A. W. M. Smeulders, The challenge problem for automated detection of 101 semantic conceptsin multimedia, in Proc. 14th Annual ACM Int. Conf. Multimedia 2006, pp. 421-430. (Pubitemid 47113558)
43. E. Spyromitros, G. Tsoumakas and I. Vlahavas, An empirical study of lazy multilabel classification algorithms in Proc. 5th Hellenic Conf. Artificial Intelligence: Theories, Models and Applications (2008), pp. 401-406.
44. A. Srivastava and B. Zane-Ulman, Discovering recurring anomalies in text reports regarding complex space systems, in Proc. IEEE Aerospace Conf. (2005), pp. 55-63.
45. F. Tai and H.-T. Lin, Multi-label classification with principle label space transformation, in Proc. 2nd Int. Workshop on Learning from Multi-Label Data (2010), pp. 45-52.
46. F. A. Thabtah, P. Cowling and Y. Peng, MMAC: A new multi-class, multi-label associative classification approach, in Proc. 4th IEEE Int. Conf. Data Mining (2004), pp. 217-224. (Pubitemid 40731034)
47. K. Ting and L. Zhu, Boosting support vector machines successfully, in Multiple Classifier Systems (Springer, 2009), pp. 509-518.
48. K. Trohidis, G. Tsoumakas, G. Kalliris and I. Vlahavas, Multilabel classification of music into emotions, in Proc. 9th Int. Conf. Music Information Retrieval (2008), pp. 320-330.
49. I. Tsochantaridis, T. Hofmann, T. Joachims and Y. Altun, Support vector machine learning for interdependent and structured output spaces, Int. Conf. Machine Learning (ICML) (2004), pp. 104-112.
50. G. Tsoumakas and I. Katakis, Multi label classification: An overview, Int. J. Data Warehouse and Mining 3(3) (2007) 1-13.
51. G. Tsoumakas, I. Katakis and I. Vlahavas, E®ective and e±cient multilabel classification in domains with large number of labels, in Proc. ECML/PKDD Workshop on Mining Multidimensional Data (2008), pp. 30-44.
52. G. Tsoumakas, I. Katakis and I. Vlahavas, Mining multi-label data, in Data Mining and Knowledge Discovery Handbook (Springer Berlin/Heidelberg, 2010), pp. 667-685.
53. G. Tsoumakas, I. Katakis and I. Vlahavas, Random k-labelsets for multilabel classification, IEEE Trans. Knowl. Data Eng. 23(7) (2011) 1079-1089.
54. G. Tsoumakas, E. Spyromitros-Xioufis, J. Vilcek and I. Vlahavas, Mulan: A javalibrary for multi-label learning, J. Mach. Learn. Res. 12 (2011) 2411-2414.
55. J. Wang, Y. Zhao, X. Wu and X.-S. Hua, A transductive multi-label learning approach for video concept detection, Pattern Recogn. 44(10-11) (2011) 2274-2286.
56. A. Wieczorkowska, P. Synak and Z. Ras, Multi-label classification of emotions in music, in Intelligent Information Processing and Web Mining (Springer Berlin/Heidelberg, 2006), pp. 307-315.
57. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 1 st edn. (Morgan Kaufmann, 2005).
58. T.-F. Wu, C.-J. Lin and R. C. Weng, Probability estimates for multi-class classification by pairwise coupling, J. Mach. Learn. Res. 5 (2004) 975-1005.
59. J. Xu, An extended one-versus-rest support vector machine for multi-label classification, Neurocomputing 74(17) (2011) 3114-3124.
60. X. Zhang, Q. Yuan, S. Zhao, W. Fan, W. Zheng and Z. Wang, Multi-label classification without the multi-label cost, in Proc. 10th SIAM Int. Conf. Data Mining (Columbus, Ohio, USA, 2010), pp. 778-789.
61. M. L. Zhang and Z. H. Zhou, Multi-label neural networks with applications tofunctional genomics and text categorization, IEEE Trans. Knowl. Data Eng. 18(10) (2006) 1338-1351.
62. M. L. Zhang and Z. H. Zhou, Ml-knn: A lazy learning approach to multi-label learning, Pattern Recogn. 40(7) (2007) 2038-2048.