Margin maximization with feed-forward neural networks: A comparative study with SVM and AdaBoost

Neurocomputing

Volumn 57, Issue 1-4, 2004, Pages 313-344

  Romero, Enrique ^a   Màrquez, Lluís ^a   Carreras, Xavier ^a  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

AdaBoost; Feed forward Neural Networks; Margin maximization; NLP classification problems; Support Vector Machines

Indexed keywords

ALGORITHMS; ERROR ANALYSIS; LEARNING SYSTEMS; NONLINEAR SYSTEMS;

SUPPORT VECTOR MACHINES (SVM);

FEEDFORWARD NEURAL NETWORKS;

ALGORITHM; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; CLASSIFICATION; COMPUTER PROGRAM; CONTROLLED STUDY; EXPERIMENTATION; LEARNING; LINEAR SYSTEM; MACHINE; MATHEMATICAL MODEL; NONLINEAR SYSTEM; PERFORMANCE; POSITIVE FEEDBACK; PREDICTION; PRIORITY JOURNAL; THEORETICAL STUDY;

EID: 1542680908     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2003.10.011     Document Type: Article

References (42)

1. Bishop C.M. Neural Networks for Pattern Recognition 1995 Oxford University Press Inc New York
2. Bruce R.F. Wiebe J.M. Decomposable Modeling in Natural Language Processing Comput. Linguistics 25 2 1999 195-207
3. C. Cardie, R. Mooney (Guest Eds.) Introduction: machine learning and natural language, Machine Learning (Special Issue on Natural Language Learning) 34 (1-3) (1999) 5-9.
4. C.C. Chang, C.J. Lin, LIBSVM: A Library for Support Vector Machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm
5. Cohen J. A coefficient of agreement for nominal scales J. Educational Psychol. Measure. 20 1960 37-46
6. Cristianini N. Shawe-Taylor J. An Introduction to Support Vector Machines 2000 Cambridge University Press UK
7. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization Mach. Learning 40 2 2000 139-157
8. G. Escudero, L. Màrquez, G. Rigau, Boosting Applied to Word Sense Disambiguation in: R. López de Mántaras, E. Plaza (Eds.), Proceedings of the 11th European Conference on Machine Learning, ECML-2000, Springer LNAI 1810, Barcelona, Spain, 2000, pp. 129-141.
9. G. Escudero, L. Màrquez, G. Rigau, Using LazyBoosting for Word Sense Disambiguation, in: Proceedings of the Second Senseval Workshop on the Evaluation of WSD Systems, Tolouse, France, 2001, pp. 71-74.
10. C. Fellbaum (Ed.), WordNet. An Electronic Lexical Database, MIT Press, 1998.
11. Freund Y. An adaptive version of the boost by majority algorithm Mach. Learning 43 3 2001 293-318
12. Freund Y. Schapire R.E. A decision-theoretic generalization of on-line learning and an application to boosting J. Comput. System Sci. 55 1 1997 119-139
13. Girosi F. An equivalence between sparse approximation and support vector machines Neural Comput. 10 6 1998 1455-1480
14. Ide N. Véronis J. Introduction to the special issue on word sense disambiguation: the state of the art Comput. Linguistics 24 1 1998 1-40
15. T. Joachims, Text Categorization with Support Vector Machines: Learning with Many Relevant Features, in: C. Nédellec, C. Rouveirol, Proceedings of the 10th European Conference on Machine Learning, ECML-1998, LNAI-1398, Chemnitz, Germany, 1998, pp. 137-142.
16. Joachims T. Making large-scale SVM learning practical, in: Schölkopf B. Burges C. Smola A. (Eds.), Advances in Kernel Methods-Support Vector Learning 1999 169-184 The MIT Press Cambridge MA
17. A. Kilgarriff, J. Rosenzweig, English SENSEVAL: report and results, in: Proceedings of the Second International Conference on Language Resources and Evaluation, LREC-2000, Athens, Greece, 2000, pp. 1239-1243.
18. Y.K. Lee, H.T. Ng, An empirical evaluation of knowledge sources and learning algorithms for word sense disambiguation, in: Proceedings of the Seventh Conference on Empirical Methods in Natural Language Processing, EMNLP-2002, Philadelphia, PA, 2002, pp. 41-48.
19. Leshno M. Lin V.Y. Pinkus A. Schocken S. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function Neural Networks 6 6 1993 861-867
20. López de Mántaras R. A distance-based attribute selection measure for decision tree induction Mach. Learning 6 1 1991 81-92
21. B. Magnini, G. Cavaglia, Integrating subject field codes into wordnet, in: Proceedings of the Second International Conference on Language Resources and Evaluation, LREC-2000, Athens, Greece, 2000, pp. 1413-1418.
22. H.T. Ng, Exemplar-based word sense disambiguation: some recent improvements, in: Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, EMNLP-1997, Providence, RI, 1997, pp. 208-213.
23. H.T. Ng, C.Y. Lim, S.K. Foo, A case study on inter-annotator agreement for word sense disambiguation, in: Proceedings of the ACL SIGLEX Workshop: Standardizing Lexical Resources, SIGLEX-1999, College Park, MD, 1999, pp. 9-13.
24. Park J. Sandberg I.W. Approximation and radial-basis-function networks Neural Comput. 5 2 1993 305-316
25. Perrone M.P. Cooper L.N. When networks disagree: ensemble methods for hybrid neural networks, in: Mammone R.J. (Ed.), Artificial Neural Networks for Speech and Vision 1993 126-142 Chapman & Hall London
26. Rätsch G. Mika S. Schölkopf B. Müller K.-R. Constructing boosting algorithms from SVMs: an application to one-class classification IEEE Trans. Pattern Anal. Mach. Intell. 24 9 2002 1184-1199
27. Raudys S. Evolution and generalization of a single neurone: I. Single-layer perceptron as seven statistical classifiers Neural Networks 11 2 1998 283-296
28. Romero E. Carreras X. Màrquez L. Exploiting diversity of margin-based classifiers research report, LSI-03-49-R, LSI Department 2003 Universitat Politècnica de Catalunya Barcelona
29. D.E. Rumelhart, G.E. Hinton, R.J. Williams, Parallel Distributed Processing, Vol. 1, The MIT Press, Cambridge, MA, 1986.
30. Schapire R.E. Singer Y. Improved boosting algorithms using confidence-rated predictions Mach. Learning 37 3 1999 297-336
31. Schapire R.E. Singer Y. BOOSTEXTER: a boosting-based system for text categorization Mach. Learning 39 2-3 2000 135-168
32. Sebastiani F. Machine learning in automated text categorization ACM Comput. Surveys 34 1 2002 1-47
33. Smola A.J. Schölkopf B. Müller K.R. The connection between regularization operators and support vector kernels Neural Networks 11 4 1998 637-650
34. J.M. Sopena, E. Romero, R. Alquézar, Neural networks with periodic and monotonic activation functions: a comparative study in classification problems, in: Proceedings of the Ninth International Conference on Artificial Neural Networks, ICANN-1999, Edinburgh, Scotland, pp. 323-328.
35. Suykens J.A.K. Vandewalle J. Training multilayer perceptron classifiers based on a modified support vector method IEEE Trans. Neural Networks 10 4 1999 907-911
36. Towell G. Voorhees E.M. Disambiguating highly ambiguous words Comput. Linguistics 24 1 1998 125-146
37. Vapnik V. The Nature of Statistical Learning Theory 1995 Springer New York
38. Vapnik V. The support vector method of function estimation, in: Bishop C. Neural (Ed.), Networks and Machine Learning 1998 239-268 Springer Berlin
39. J. Véronis, A study of polysemy judgements and inter-annotator agreement, in: Programme and Advanced Papers of the First Senseval Workshop, Herstmonceux Castle, Sussex, England, 1998, pp. 2-4.
40. P. Vincent, Y. Bengio, A neural support vector architecture with adaptive kernels, in: Proceedings of the IEEE International Joint Conference on Neural Networks, IJCNN-2000, Como, Italy, 2000, pp. 187-192.
41. Y. Yang, X. Liu, A Re-examination of Text Categorization Methods, in: Proceedings of the 22nd Annual ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR-1999, Berkeley, CA, 1999, pp. 42-49.
42. Yarowsky D. Hierarchical decision lists for word sense disambiguation Comput. Humanities 34 2 2000 179-186