The Perceptron algorithm vs. winnow: Linear vs. Logarithmic mistake bounds when few input variables are relevant

Proceedings of the 8th Annual Conference on Computational Learning Theory, COLT 1995

Volumn 1995-January, Issue , 1995, Pages 289-296

  Kivinen, Jyrki ^a   Warmuth, Manfred K ^b  

^a UNIVERSITY OF HELSINKI   (Finland)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INPUT VARIABLES; LINEAR THRESHOLD FUNCTIONS; MISTAKE BOUNDS; MULTIPLICATIVE UPDATES; PERCEPTRON ALGORITHMS; RANDOM DATA; WEIGHT VECTOR;

COMPUTATION THEORY;

EID: 84969342575     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth. Learnability and the Vapnik-Chervonenkis dimension. J. ACM, 36(4):929-965, 1989.
2. R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. Wiley, 1973.
3. E. Dichterman and R. Khardon. A tight bound for the VC dimension of κ-term DNF. Private communication.
4. L. G. Khachiyan. A polynomial algorithm in linear programming (in Russian). Doklady Akademn Nauk SSSR, 244:1093-1096, 1979.
5. (English translation: Soviet Mathematics Doklady 20:191-194, 1979.)
6. J. Kivinen and M. K. Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Report UCSC-CRL-94-16, University of California, Santa Cruz, June 1994.
7. N. Littlestone. Learning quickly when irrelevant attributes abound: A new linearthreshold algorithm. Mach. Learning, 2:285-318, 1988.
8. N. Littlestone. Mistake Bounds and Logarithmic Linear-threshold Learning Algorithms. PhD thesis, Report UCSC-CRL-89-11, University of California Santa Cruz, 1989.
9. N. Littlestone. Redundant noisy attributes, attribute errors, and linear threshold learning using Winnow. In Proc. 4th Workshop on Comput. Learning Theory, pages 147-156. Morgan Kaufmann, 1991.
10. N. Littlestone. Comparing several linearthreshold learning algorithms on tasks involving superfluous attributes. In Proc. 12th International Machine Learning Conference (ML-95), July 1995.
11. N. Littlestone, P. M. Long, and M. K. Warmuth. On-line learning of linear functions. In Proc. 23rd ACM Symposium on Theory of Computing, pages 465-475, 1991.
12. W. Maass and G. Turan. How fast can a threshold gate learn. In Computational Learning Theory and Natural Learning Systems, Volume I, pages 381-414. MIT Press, 1994.
13. F. Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psych. Rev., 65:386-407, 1958. (Reprinted in Neurocomputing (MIT Press, 1988).).
14. II. Sompolinsky, H. S. Seung, and N. Tishby. Learning curves in large neural networks. In Proc. 4th. Workshop on Cornput. Learning Theory, pages 112-127. Morgan Kaufmann, 1991.
15. P M. Vaidya. A new algorithm for minimizing convex functions over convex sets. In Proc. 30th Symposium on Foundations of Computer Science, pages 338-343. IEEE Computer Society, 1989.
16. L. G. Valiant. A theory of the learnable. Commun. ACM, 27(11):1134-1142, 1984.
17. V N. Vapnik and A. Y. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probab. and its Applications, 16(2):264-280, 1971.
18. T. L. H. Watkin, A. Rau, and M. Biehl. The statistical mechanics of learning a rule. Rev. Mod. Phys., 65:499-556, 1993.