Hierarchical linear support vector machine

Pattern Recognition

Volumn 45, Issue 12, 2012, Pages 4414-4427

  Rodriguez Lujan, Irene ^a   Santa Cruz, Carlos ^a   Huerta, Ramon ^b  

^a UNIVERSIDAD AUTÓNOMA DE MADRID   (Spain)

^b UNIVERSITY OF CALIFORNIA SAN DIEGO   (United States)

Author keywords

Decision tree; Large scale learning; Pegasos algorithm; Real time prediction; Support vector machine

Indexed keywords

CLASSIFICATION METHODS; CLASSIFICATION RESULTS; CREDIT CARD FRAUD DETECTIONS; DATA SPACE; LARGE DATASETS; LARGE-SCALE DATASETS; LARGE-SCALE LEARNING; LINEAR SUPPORT VECTOR MACHINES; OBLIQUE DECISION TREE; PARALLELIZATIONS; REAL-TIME PREDICTION; REAL-WORLD DATASETS; TRAINING COSTS; TRAINING PROCESS; TREE STRUCTURES;

ALGORITHMS; DECISION TREES; FORECASTING; TREES (MATHEMATICS);

SUPPORT VECTOR MACHINES;

EID: 84864282165     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2012.06.002     Document Type: Article

References (56)

1. I. Steinwart Sparseness of support vector machines - some asymptotically sharp bounds Neural Information Processing Systems 16 2004 1069 1076
2. S.S. Keerthi, O. Chapelle, D. DeCoste Building support vector machines with reduced classifier complexity Journal of Machine Learning Research 7 2006 1493 1515
3. C.J.C. Burges, B. Schölkopf, Improving the accuracy and speed of support vector learning machines, in: Advances in Neural Information Processing Systems, vol. 9, MIT Press, Cambridge, MA, 1997, pp. 375-381.
4. T. Downs, K.E. Gates, A. Masters Exact simplification of support vector solutions Journal of Machine Learning Research 2 2002 293 297
5. E.E. Osuna, F. Girosi, Reducing the run-time complexity in support vector machines, in: Advances in Kernel Methods, MIT Press, Cambridge, MA, USA, 1999, pp. 271-283.
6. M. Wu, B. Schölkopf, G.H. Bakir, Building sparse large margin classifiers, in: ICML '05: Proceedings of the 22nd International Conference on Machine Learning, vol. 119, 2005, pp. 996-1003.
7. R.E. Schapire, Y. Singer Improved boosting algorithms using confidence-rated predictions Machine Learning 37 3 1999 297 336
8. L. Breiman Bagging predictors Machine Learning 24 2 1996 123 140
9. R. Collobert, S. Bengio, Y. Bengio A parallel mixture of SVMs for very large scale problems Neural Computation 14 5 2002 1105 1114
10. H.P. Graf, E. Cosatto, L. Bottou, I. Durdanovic, V. Vapnik, Parallel support vector machines: the cascade SVM, in: Advances in Neural Information Processing Systems, MIT Press, 2005, pp. 521-528.
11. C.G.X.L.F. Chang, C. Lu Tree decomposition for large-scale SVM problems Journal of Machine Learning Research 11 2010 2935 2972
12. L. Breiman, J.H. Friedman, R.A. Olshen, C.J. Stone Classification and Regression Trees 1984 Chapman & Hall New York, NY
13. Y. Singer, N. Srebro, Pegasos: primal estimated sub-gradient solver for SVM, in: ICML, 2007, pp. 807-814.
14. A. Bordes, L. Bottou, P. Gallinari Sgd-qn: careful quasi-Newton stochastic gradient descent Journal of Machine Learning Research 10 2009 1737 1754
15. X. Zhou, N. Cui, Z. Li, F. Liang, T.S. Huang, Hierarchical gaussianization for image classification, in: ICCV, IEEE, 2009, pp. 1971-1977.
16. K. Yu, T. Zhang, Y. Gong, Non-linear learning using local coordinate coding, in: Advances in Neural Information Processing Systems, vol. 22, 2009, pp. 2223-2231.
17. L. Ladicky, P. Torr, Locally linear support vector machines, in: ICML '11: Proceedings of the 28th International Conference on Machine Learning, 2011, pp. 985-992.
18. Z. Fu, A. Robles-Kelly, J. Zhou Mixing linear SVMs for nonlinear classification IEEE Transactions on Neural Networks 21 12 2010 1963 1975
19. K.P. Bennett, N. Cristianini, J. Shawe-Taylor, D. Wu Enlarging the margins in perceptron decision trees Machine Learning 41 2000 295 313
20. M. Arun Kumar, M. Gopal A hybrid SVM based decision tree Pattern Recognition 43 2010 3977 3987
21. K.Z. Arreola, J. Fehr, H. Burkhardt, Fast support vector machine classification using linear SVMs, in: ICPR (3), IEEE Computer Society, 2006, pp. 366-369.
22. M. Anthony Generalization error bounds for threshold decision lists Journal of Machine Learning Research 5 2004 189 217
23. J.R. Quinlan, R.L. Rivest Inferring decision trees using the minimum description length principle Information and Computation 80 3 1989 227 248
24. J. Fehr, K.Z. Arreola, H. Burkhardt, Fast support vector machine classification of very large datasets, in: GfKl, Studies in Classification, Data Analysis, Knowledge Organization, Springer, 2007, pp. 11-18.
25. J. Su, G. Wang, Q. Hu, S. Li, A novel SVM decision tree and its application to face detection, in: SNPD (1), IEEE Computer Society, 2007, pp. 385-389.
26. V. Menkovski, I. Christou, S. Efremidis, Oblique decision trees using embedded support vector machines in classifier ensembles, in: 7th IEEE International Conference on Cybernetic Intelligent Systems, 2008, CIS 2008., 2008, pp. 1-6.
27. J.R. Quinlan C4.5: Programs for Machine Learning 1993 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA
28. J.C. Platt, Fast training of support vector machines using sequential minimal optimization, in: Advances in Kernel Methods: Support Vector Learning, MIT Press, Cambridge, MA, 1998, pp. 185-208.
29. K.P. Bennett, J. Blue, A support vector machine approach to decision trees, in: The 1998 IEEE International Joint Conference on Neural Networks Proceedings, 1998, IEEE World Congress on Computational Intelligence, 1998.
30. Y. Ben-Haim, E. Tom-Tov A streaming parallel decision tree algorithm Journal of Machine Learning Research 11 2010 849 872
31. S. Viaene, R.A. Derrig, G. Dedene Cost-sensitive learning and decision making for Massachusetts pip claim fraud data International Journal of Intelligent Systems 19 12 2004 1197 1215
32. S.-B. Park, S. Hwang, B.-T. Zhang, Mining the risk types of human papillomavirus (hpv) by adacost, in: DEXA, Lecture Notes in Computer Science, vol. 2736, Springer, 2003, pp. 403-412.
33. Y. Huang, S. Du, Weighted Support vector machine for classification with uneven training class sizes, in: Proceedings of 2005 International Conference on Machine Learning and Cybernetics, vol. 7, 2005, pp. 4365-4369.
34. J. Quinlan Simplifying decision trees International Journal of Man-Machine Studies 27 1987 221 234
35. W.W. Cohen, Efficient pruning methods for separate-and-conquer rule learning systems, in: IJCAI, 1993, pp. 988-994.
36. R. Fan, K. Chang, C. Hsieh, X. Wang, C.-J. Lin LIBLINEAR: a library for large linear classification Journal of Machine Learning Research 9 2008 1871 1874 Software available at 〈 http://www.csie.ntu.edu.tw/∼cjlin/ liblinear/ 〉
37. C. Hsieh, K. Chang, C. Lin, S.S. Keerthi, S. Sundararajan, A dual coordinate descent method for large-scale linear SVM, in: ICML '08: Proceedings of the 25th International Conference on Machine Learning, vol. 307, 2008, pp. 408-415.
38. A.K. Menon, Large-scale support vector machines: algorithms and theory, Technical Report, University of California, San Diego, 2009.
39. S. Shalev-Shwartz, N. Srebro, SVM optimization: inverse dependence on training set size, in: Proceedings of the 25th International Conference on Machine Learning (ICML'08), vol. 307, 2008, pp. 928-935.
40. M.G. Peter, I. Llew Mason, Generalization in decision trees and dnf: Does size matter?, in: Advances in Neural Information Processing Systems, The MIT Press, 1997, pp. 259-265.
41. Y. Mansour, D.A. McAllester, Generalization bounds for decision trees, in: COLT, 2000, pp. 69-74.
42. M. Shah, Sample compression bounds for decision trees, in: ICML, vol. 227, 2007, pp. 799-806.
43. M. Marchand, M. Sokolova Learning with decision lists of data-dependent features Journal of Machine Learning Research 6 2005 427 451
44. F. Laviolette, M. Marchand, M. Shah, S. Shanian Learning the set covering machine by bound minimization and margin-sparsity trade-off Machine Learning 78 1-2 2010 175 201
45. M. Shah, M. Marchand, J. Corbeil Feature selection with conjunctions of decision stumps and learning from microarray data IEEE Transactions on Pattern Analysis and Machine Intelligence 34 1 2012 174 186
46. R.C. Holte Very simple classification rules perform well on most commonly used datasets Machine Learning 11 1993 63 90
47. U.M. Fayyad, K.B. Irani, What should be minimized in a decision tree?, in: Association for the Advancement of Artificial Intelligence (AAAI), 1990, pp. 749-754.
48. A. Ehrenfeucht, D. Haussler, M. Kearns Learning decision trees from random examples needed for learning Information and Computation 82 1989 231 246
49. V.N. Vapnik Statistical Learning Theory 1998 Wiley-Interscience
50. Y. Freund, R.E. Schapire, Experiments with a new boosting algorithm, in: International Conference on Machine Learning (ICML), 1996, pp. 148-156.
51. C.-C. Chang, C.-J. Lin, LIBSVM: a library for support vector machines, software available at 〈 http://www.csie.ntu.edu.tw/∼cjlin/libsvm 〉, 2001.
52. J. Fehr, K.Z. Arreola, H. Burkhardt, Fast support vector machine classification of very large datasets, Technical Report, University of Freiburg, Department of Computer, 2007.
53. I.W. Tsang, J.T. Kwok, P.-M. Cheung Core vector machines: fast SVM training on very large data sets Journal of Machine Learning Research 6 2005 363 392
54. Y. Freund, R.E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, in: Proceedings of the Second European Conference on Computational Learning Theory, 1995, pp. 23-37.
55. R.E. Schapire, Y. Freund, P. Bartlett, W.S. Lee Boosting the margin: a new explanation for the effectiveness of voting methods The Annals of Statistics 26 5 1998 1651 1686
56. P. Viola, M. Jones Robust real-time face detection International Journal of Computer Vision 57 2 2004 137 154