A new approximate maximal margin classification algorithm

Journal of Machine Learning Research

Volumn 2, Issue 4, 2001, Pages 213-242

  Gentile, Claudio ^a  

^a UNIVERSITY OF MILAN   (Italy)

Author keywords

Binary classification; Large margin; On line learning; Support vector machines

Indexed keywords

ACCURACY LEVEL; BINARY CLASSIFICATION; CLASSIFICATION ALGORITHM; INCREMENTAL ALGORITHM; INCREMENTAL LEARNING; LARGE MARGIN; LINEARLY SEPARABLE; MAXIMAL MARGIN; ON-LINE ALGORITHMS; ONLINE LEARNING; PERCEPTRON ALGORITHMS; TARGET VECTORS; TRAINING ALGORITHMS;

LEARNING ALGORITHMS;

SUPPORT VECTOR MACHINES;

EID: 84868111801     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (48)

1. M. A. Aizerman, E. M. Braverman and L. I. Rozonoer. Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control, 25:821-837, 1964.
2. E. L. Allwein, R. E. Schapire and Y. Singer. Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research, 1:113-141, 2000. (Pubitemid 33738778)
3. D. Angluin. Queries and concept learning. Machine Learning, 2(4):319-342, 1988.
4. M. Anthony and P. Bartlett. Neural Network Learning: Theoretical Foundations. CMU, 1999.
5. P. Auer, N. Cesa Bianchi and C. Gentile. Adaptive and self-confident on-line learning algorithms. Journal of Computer and System Sciences, forthcoming. Preliminary version in Proceedings of 13th Annu. Conf. on Comput. Learning Theory, pages 107-117, Palo Alto, CA, 2000.
6. C. Blake, E. Keogh and C. Merz. Uci repository of machine learning databases. Technical report, Dept. of Information and Computer Sciences, University of California, Irvine, 1998. http://www.ics.uci.edu/-mlearn/ MLRepository.html.
7. H. D. Block. The perceptron: A model for brain functioning. Reviews of Modern Physics, 34:123-135, 1962.
8. C. Cortes and V. Vapnik. Support-vector networks. Machine Learning, 20(3):273-297, 1995.
9. N. Cristianini and J. Shawe-Taylor. An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, 2000.
10. I. Dagan, Y. Karov and D. Roth. Mistake-driven learning in text categorization. In Proceedings of 2nd Conference on Empirical Methods in Natural Language Processing, pages 55-63. Association for Computational Linguistics, Somerset, New Jersey, 1997.
11. T. G. Dietterich and G. Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2:263-286, 1995.
12. R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. Wiley, 1973.
13. Y. Freund and R. E. Schapire. Large margin classification using the perceptron algorithm. Journal of Machine Learning, 37(3):277-296, 1999. (Pubitemid 32210619)
14. T.-T. Friess, N. Cristianini and C. Campbell. The kernel adatron algorithm: A fast and simple learning procedure for support vector machines. In Proceedings of 15th International Conference in Machine Learning, pages 188-196. Morgan Kaufmann, San Mateo, CA, 1998.
15. C. Gentile and N. Littlestone. The robustness of the p-norm algorithms. In Proc. 12th Annu. Conf. on Comput. Learning Theory, pages 1-11. ACM, 1999.
16. C. Gentile and M. K. Warmuth. Linear hinge loss and average margin. Unpublished. Preliminary version in Proc. Advances in Neural Information Processing Systems 11, pages 225-231, MIT Press, Cambridge, MA, 1999, 2001.
17. C. Gentile. A new approximate maximal margin classification algorithm. In T. K. Leen, T. G. Dietterich, and V. Tresp editors, Advances in Neural Information Processing Systems 13, pages 500-506. MIT Press, Cambridge, MA, 2001.
18. A. R. Golding and D. Roth. Applying winnow to context-sensitive spelling correction. In Proceedings of 13th International Conference in Machine Learning, pages 182-190. Morgan Kaufmann, San Mateo, CA, 1996.
19. A. J. Grove, N. Littlestone and D. Schuurmans. General convergence results for linear discriminant updates. Journal of Machine Learning, 43(3):173-210, 2001. (Pubitemid 32471832)
20. D. P. Helmbold and M. K. Warmuth. On weak learning. Journal of Computer and System Sciences, 50(3):551-573, 1995.
21. T. Joachims. Making large-scale support vector machines learning practical. In B. Scholkopf, C Purges and A. Smola (eds.): Advances in kernel methods: support vector machines. MIT Press, Cambridge, MA, 2000.
22. S. S. Keerthi, S. K. Shevade, C. Bhattacharyya and K.R.K. Murthy. A fast iterative nearest point algorithm for support vector machine classifier design. Technical report, Indian Institute of Science, ISL-99-03, 1999.
23. J. Kivinen and M. K. Warmuth. Additive versus exponentiated gradient updates for linear prediction. Information and Computation, 132(1):l-64, 1997.
24. J. Kivinen and M. K. Warmuth. Relative loss bounds for multidimensional regression problems. Journal of Machine Learning, forthcoming. Preliminary version in Proc. Advances in Neural Information Processing Systems 10, pages 287-293, MIT Press, Cambridge, MA, 1998.
25. J. Kivinen, M. K. Warmuth and P. Auer. The perceptron algorithm vs. winnow: linear vs. logarithmic mistake bounds when few input variables are relevant. Artificial Intelligence, 97:325-343, 1997. (Pubitemid 127401108)
26. A. Kowalczyk. Maximal margin perceptron. MIT Press, Cambridge, MA, 1999.
27. Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard and L. J. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1:541-551, 1989.
28. Y. Le Cun, L. J. Jackel, L. Bottou, A. Brunot, C. Cortes, J. S. Denker, H. Drucker, I. Guyon, U. Muller, S. Sackinger, P. Simard and V. Vapnik. Comparison of learning algorithms for handwritten digit recognition. In Proceedings of ICANN 1995, pages 53-60, 1995.
29. Y. Li. From support vector machines to large margin classifiers. PhD thesis, School of Computing, National University of Singapore, 2000.
30. Y. Li and P. Long. The relaxed online maximum margin algorithm. Journal of machine learning, forthcoming. Preliminary version in S. A. Solla, T. K. Leen and K. R. Müller editors, Advances in Neural Information Processing Systems 12, pages 498-504, MIT Press, Cambridge, MA, 2000.
31. N. Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2:285-318, 1988.
32. N. Littlestone and M. K. Warmuth. The weighted majority algorithm. Information and Computation, 108(2):212-261, 1994.
33. O. Mangasarian. Multi-surface method of pattern separation. IEEE Trans, on Information Theory, 14:801-807, 1968.
34. O. Mangasarian. Mathematical programming in data mining. Data Mining and Knowledge Discovery, 42(1):183-201, 1997. (Pubitemid 127721240)
35. P. Nachbar, J. A. Nossek and J. Strobl. The generalized adatron algorithm. In Proceedings of 1993 IEEE ISCAS, pages 2152-2155, 1993.
36. A. B. J. Novikov. On convergence proofs on perceptrons. In Proc. of the Symposium on the Mathematical Theory of Automata, vol. XII, pages 615-622, 1962.
37. E. Osuna, R. Freund and F. Girosi. An improved training algorithm for support vector machines. In Proceedings of IEEE NNSP'97, 1997.
38. J. C. Piatt. Fast training of support vector machines using sequential minimal optimization. MIT Press, Cambridge, MA, 1998.
39. J. C. Piatt, N. Cristianini and J. Shawe-Taylor. Large margin dags for multiclass classification. In S. A. Solla, T. K. Leen and K. R. Müller editors, Advances in Neural Information Processing Systems 12, pages 547-553. MIT Press, Cambridge, MA, 1999.
40. F. Rosenblatt. Principles of neurodynamics: Perceptrons and the theory of brain mechanisms. Spartan Books, Washington, D.C, 1962.
41. R. E. Schapire, P. Bartlett, Y. Freund and W. S. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5):1651-1686, 1998. (Pubitemid 128376902)
42. B. Scholkopf, S. Mika, C.J.C. Burges, P. Knirsch, K. Muller, G. Ratsch and A. Smola. Input space vs. feature space in kernel-based methods. IEEE Trans, on Neural Network, 10(5): 1000-1017, 1999.
43. B. Scholkopf, K. Sung, C.J.C. Burges, F. Girosi, P. Niyogi, T. Poggio and V. Vapnik. Comparing support vector machines with gaussian kernels to radial basis function classifiers. IEEE Trans, on Signal Processing, 45:2758-2765, 1997. (Pubitemid 127766345)
44. H. Schwenk and Y. Bengio. Boosting neural networks. Neural Computation, 12(8):18691887, 2000.
45. R. A. Servedio. On pac learning using winnow, perceptron, and a perceptron-like algorithm. In Proc. 12th Annu. Conf. on Comput. Learning Theory, pages 296-307. ACM, 1999.
46. J. Shawe-Taylor, P. Bartlett, R. Williamson and M. Anthony. Structural risk minimization over data-dependent hierarchies. IEEE Trans, on Information Theory, 44(5): 1926-1940, 1998. (Pubitemid 128738561)
47. P. Simard, Y. LeCun and J. Denker. Efficient pattern recognition using a new transformation distance. In S. Hanson, J. Cowan, and L. Giles, editors, Advances in Neural Information Processing Systems, volume 5. Morgan Kaufmann, 1993.
48. V. Vapnik. Statistical learning theory. J. Wiley & Sons, New York, 1998.