Worst-case quadratic loss bounds for prediction using linear functions and gradient descent

IEEE Transactions on Neural Networks

Volumn 7, Issue 3, 1996, Pages 604-619

  Cesa Bianchi, Nicolò ^a   Long, Philip M ^b   Warmuth, Manfred K ^c  

^a UNIVERSITY OF MILAN   (Italy)

^b Duke University   (United States)

^c University of California Santa Cruz   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ERRORS; LEARNING SYSTEMS; MATHEMATICAL MODELS; ONLINE SYSTEMS; PERFORMANCE; PROBLEM SOLVING;

GRADIENT DESCENT; LINEAR FUNCTIONS; QUADRATIC LOSS BOUNDS; SQUARED PREDICTION ERRORS;

NEURAL NETWORKS;

EID: 0030145382     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.501719     Document Type: Article

References (30)

1. D. Angluin, "Queries and concept learning," Machine Learning, vol. 2, pp. 319-342, 1988.
2. A. Barron, "Universal approximation bounds for superpositions of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, 1993.
3. N. Cesa-Bianchi, Y. Freund, D. P. Helmbold, D. Haussler, R. E. Schapire, and M. K. Warmuth, "How to use expert advice," in Proc. 25th ACM Symp. Theory Computa., 1993.
4. A. Dawid, "Statistical theory: The prequential approach," J. Roy. Statist. Soc. A, pp. 278-292, 1984.
5. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
6. V. Faber and J. Mycielski, "Applications of learning theorems," Fundamenta Informaticae, vol. 15, no. 2, pp. 145-167, 1991.
7. M. Feder, N. Merhav, and M. Gutman, "Universal prediction of individual sequences," IEEE Trans. Inform. Theory, vol. 38, pp. 1258-1270, 1992.
8. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: Johns Hopkins Univ. Press, 1990.
9. W. Hardle, Smoothing Techniques. New York: Springer-Verlag, 1991.
10. S. S. Haykin, Adaptive Filter Theory, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1991.
11. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1985.
12. S. Hui and S. H. Zak, "Robust stability analysis of adaptation algorithms for single perceptron," IEEE Trans. Neural Networks, vol. 2, pp. 325-328, 1991.
13. S. Kaczmarz, "Angenaherte Auflösung von systemen linearer gleichungen," Bull. Acad. Polon. Sci. Lett. A, vol. 35, pp. 355-357, 1937.
14. D. Kimber and P. M. Long, "The learning complexity of smooth functions of a single variable," in Proc. 1992 Wkshp. Computa. Learning Theory, pp. 153-159, 1992.
15. J. Kivinen and M. K. Warmuth, "Exponentiated gradient versus gradient descent for linear predictors," Univ. California, Santa Cruz, Rep. UCSC-CRL-94-16, June 1994.
16. N. Littlestone, "Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm," Machine Learning, vol. 2, no. 4, pp. 285-318, 1988.
17. _, Mistake Bounds and Logarithmic Linear-threshold Learning Algorithms, Ph.D. dissertation, Univ. California Santa Cruz, Tech. Rep. UCSC-CRL-89-11, 1989.
18. N. Littlestone, P. M. Long, and M. K. Warmuth, "On-line learning of linear functions," in Proc. 23rd ACM Symp. Theory Comp., pp. 465-475, 1991.
19. R. W. Lucky, "Techniques of adaptive equalization of digital communication systems," Bell Syst. Tech. J., vol. 45, pp. 255-286, 1966.
20. D. G. Luenberger, Linear and Nonlinear Programming. Reading, MA: Addison-Wesley, 1984.
21. N. Littlestone and M. K. Warmuth, "The weighted majority algorithm," Univ. California Santa Cruz, Tech. Rep. UCSC-CRL-91-28, Oct. 1991. A preliminary version appeared in the Proc. 30th Annu. IEEE Symp. Foundations Comput. Sci., October 89, pp. 256-261.
22. N. Merhav and M. Feder, "Universal sequential learning and decision from individual data sequences," in Proc. 1992 Wkshp. Computa. Learning Theory, pp. 413-427.
23. J. Mycielski and S. Swierczkowski, "General learning theorems," unpublished, 1991.
24. J. Mycielski, "A learning algorithm for linear operators," in Proc. Amer. Math. Soc., vol. 103, no. 2, pp. 547-550, 1988.
25. M. M. Sondhi, "An adaptive echo canceller," Bell Syst. Tech. J., vol. 46, pp. 497-511, 1967.
26. M. M. Sondhi and D. A. Berkley, "Silencing echoes in the telephone network," Proc. IEEE, vol. 68, pp. 948-963, 1980.
27. V. Vovk, "Aggregating strategies," in Proc. 3rd Wkshp. Computa. Learning Theory. San Mateo, CA: Morgan Kaufmann, pp. 371-383. 1990.
28. B. Widrow and M. E. Hoff, "Adaptive switching circuits," in Proc. 1960 IRE WESCON Conv. Rec., pp. 96-104.
29. B. Widrow and S. D. Stearns, Adaptive signal processing. Englewood Cliffs, NJ: Prentice-Hall, 1985.
30. N. Young, An introduction to Hilbert Space. Cambridge, U.K.: Cambridge Univ. Press, 1988.