Analysis of two gradient-based algorithms for on-line regression

Journal of Computer and System Sciences

Volumn 59, Issue 3, 1999, Pages 392-411

  Cesa Bianchi, Nicolò ^a  

^a UNIVERSITY OF MILAN   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; FORMAL LOGIC; PROBLEM SOLVING; TRANSFER FUNCTIONS;

GRADIENT-BASED ALGORITHMS; ONLINE REGRESSION;

REGRESSION ANALYSIS;

EID: 0033285751     PISSN: 00220000     EISSN: None     Source Type: Journal    
DOI: 10.1006/jcss.1999.1635     Document Type: Article

References (23)

1. Bylander T. Worst-case absolute loss bounds for linear learning algorithms. Proceedings of the 14th National conference on Artificial Intelligence. 1997;MIT Press, Cambridge. p. 485-490.
2. Cesa-Bianchi N., Freund Y., Helmbold D. P., Haussler D., Shapire R., Warmuth M. K. How to use expert advice. J. Assoc. Compute Mach. 44:1997;427-485.
3. Cesa-Bianchi N., Long P. M., Warmuth M. K. Worst-case quadratic loss bounds for prediction using linear functions and gradient descent. IEEE Trans. Neural Networks. 7:1996;604-619.
4. Cover T. M., Thomas J. A. Elements of Information Theory. 1991;Wiley, New York.
5. Devroye L., Gyorfi L., Lugosi G. A Probabilistic Theory of Pattern Recognition. 1996;Springer-Verlag, New York/Berlin.
6. Feder M., Merhav N., Gutman M. Universal prediction of individual sequences. IEEE Trans. Inform. Theory. 38:1992;1258-1270.
7. Freund Y. Predicting a binary sequence almost as well as the optimal biased coin. Proceedings of the 9th Annual Conference on Computational Learning Theory. 1996;ACM Press, New York. p. 89-98.
8. Grove A. J., Littlestone N., Schuurmans D. General convergence results for linear discriminant updates. Proceedings of the 10th Annual Conference on Computational Learning Theory. 1997;ACM Press. p. 171-183.
9. Haussler D. Decision theoretic generalizations of the PAC model for neural net and other learning applications. Inform. and Comput. 100:1992;78-150.
10. Helmbold D. P., Kivinen J., Warmuth M. K. Worst-case loss bounds for sigmoided neurons. Advances in Neural Information Processing Systems 8. 1997;MIT Press. p. 309-315.
11. Hoeffding W. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58:1963;13-30.
12. Kivinen J., Warmuth M. K. Exponentiated gradient versus gradient descent for linear predictors. Inform. and Comput. 132:1997;1-63.
13. Kivinen J., Warmuth M. K. Relative loss bounds for multidimensional regression problems. Advances in Neural Information Processing Systems 9. 1998;MIT Press. p. 287-293.
14. Littlestone N. Learning quickly when irrelevant attributes abound: A new linear threshold algorithm. Mach. Learning. 2:1988;285-318.
15. Littlestone N. From on-line to batch learning. Proceedings of the 2nd Annual Workshop on Computational Learning Theory. 1989;Morgan Kaufmann, San Mateo. p. 269-284.
16. Littlestone N., Long P. M., Warmuth M. K. On-line learning of linear functions. Comput. Complexity. 5:1995;1-23.
17. Littlestone N., Warmuth M. K. The weighted majority algorithm. Inform. and Comput. 108:1994;212-261.
18. Long P. M. Technical Report. 1996.
19. Pollard D. Convergence of Stochastic Processes. 1984;Springer-Verlag, New York/Berlin.
20. Vovk V. G. A game of prediction with expert advice. Journal of Computer and System Sciences. 56:1998;153-173.
21. Vovk V. G. Competitive on-line linear regression. Advances in Neural Information Processing Systems 10. 1998;MIT Press, Cambridge. p. 364-370.
22. Warmuth M. K., Jagota A. K. Continuous versus discrete-time nonlinear gradient descent: Relative loss bounds and convergence. Proceedings of the 5th International Symposium on Artificial Intelligence and Mathematics. 1997.
23. Yamanishi K. A decision-theoretic extension of stochastic complexity and its application to learning. IEEE Trans. Inform. Theory. 44:1998;1424-1440.