Worst-case analysis of the Perceptron and Exponentiated Update algorithms

Artificial Intelligence

Volumn 106, Issue 2, 1998, Pages

  Bylander, Tom ^a  

^a University of Texas at San Antonio   (United States)

Author keywords

Absolute loss bounds; Learning algorithms; Mistake bounds; Randomized classification algorithms

Indexed keywords

FUNCTIONS; LEARNING ALGORITHMS; NEURAL NETWORKS; RANDOM PROCESSES; VECTORS;

EXPONENTIATED UPDATE LEARNING ALGORITHM; RANDOMIZED CLASSIFICATION ALGORITHM; WEIGHTED MAJORITY ALGORITHM;

ARTIFICIAL INTELLIGENCE;

EID: 0032305711     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/s0004-3702(98)00098-8     Document Type: Article

References (26)

1. P. Auer, M. Warmuth, Tracking the best disjunction, Machine Learning, to appear.
2. H.D. Block, The Perceptron: a model for brain functioning, Reviews of Modern Physics 34 (1) (1962) 123-135.
3. A. Blum, A. Frieze, R. Kannan, S. Vempala, A polynomial-time algorithm for learning noisy linear threshold functions, in: Proceedings 37th IEEE Annual Symposium on Foundations of Computer Science, 1996.
4. T. Bylander, Learning linear-threshold functions in the presence of classification noise, in: Proceedings 7th Annual ACM Conference on Computational Learning Theory, 1994, pp. 340-347.
5. T. Bylander, Learning linear threshold approximations using Perceptrons, Neural Computation 7 (1995) 370-379.
6. T. Bylander, Learning probabilistically consistent linear threshold functions, in: Proceedings 10th Annual Conference on Computational Learning Theory, 1997, pp. 485-490.
7. T. Bylander, Worst-case absolute loss bounds for linear learning algorithms, in: Proceedings AAAI-97, Providence, RI, 1997, pp. 485-490.
8. N. Cesa-Bianchi, Analysis of two gradient-based algorithms for on-line regression, in: Proceedings 10th Annual Conference on Computational Learning Theory, 1997, pp. 163-170.
9. N. Cesa-Bianchi, P.M. Long, M.K. Warmuth, Worst-case quadratic loss bounds for a generalization of the Widrow-Hoff rule, IEEE Trans. Neural Networks 7 (1996) 604-619.
10. E. Cohen, Learning noisy Perceptrons by a Perceptron in polynomial time, in: Proceedings 38th IEEE Annual Symposium on Foundations of Computer Science, 1997.
11. R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, Wiley, New York, 1973.
12. S.I. Gallant, Perceptron-based learning algorithms, IEEE Trans. Neural Networks 1 (1990) 179-191.
13. M. Herbster, M. Warmuth, Tracking the best expert, Machine Learning, to appear.
14. M. Herbster, M. Warmuth, Tracking the best regressor, in: Proceedings 11th Annual Conference on Computational Learning Theory, 1998.
15. W. Hoeffding, Probability inequalities for sums of bounded variables, J. Amer. Statist. Assoc. 58 (1963) 13-30.
16. R.L. Kashyap, Algorithms for pattern classification, in: J.M. Mendel, K.S. Fu (Eds.), Adaptive, Learning and Pattern Recognition Systems: Theory and Applications, Academic Press, New York, 1970, pp. 81-113.
17. J. Kivinen, M.K. Warmuth, Exponentiated gradient versus gradient descent for linear predictors, Information and Computation 132 (1997) 1-63.
18. N. Littlestone, Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm, Machine Learning 2 (1988) 285-318.
19. N. Littlestone, Mistake bounds and logarithmic linear-threshold learning algorithms, Ph.D. Thesis, University of California, Santa Cruz, CA, 1989.
20. N. Littlestone, Redundant noisy attributes, attribute errors, and linear-threshold learning using Winnow, in: Proceedings 4th Annual Workshop on Computational Learning Theory, 1991, pp. 147-156.
21. N. Littlestone, M.K. Warmuth, The weighted majority algorithm. Information and Computation 108 (1994) 212-261.
22. M.L. Minsky, S.A. Papert, Perceptrons, MIT Press, Cambridge. MA, 1969.
23. A.B.J. Novikoff, On convergence proofs for Perceptrons, in: Proceedings Symposium on the Mathematical Theory of the Automata, Vol. XII, 1962, pp. 615-622.
24. S. Papert, Some mathematical models of learning, in: Proceedings 4th London Symposium on Information Theory, 1961.
25. F. Rosenblatt, Principles of Neurodynamics, Spartan Books, New York, 1962.
26. J. Shavlik, R.J. Mooney, G. Towell, Symbolic and neural learning programs: an experimental comparison, Machine Learning 6 (1991) 111-143.