Learning rates of gradient descent algorithm for classification

Journal of Computational and Applied Mathematics

Volumn 224, Issue 1, 2009, Pages 182-192

  Dong, Xue Mei ^a   Chen, Di Rong ^a  

^a BEIHANG UNIVERSITY   (China)

Author keywords

Classification algorithm; computational complexity; Learning rates; Reproducing kernel Hilbert space; Stochastic gradient descent

Indexed keywords

ALGORITHMS; BANACH SPACES; COMMUNICATION CHANNELS (INFORMATION THEORY); COMPUTATIONAL COMPLEXITY; HILBERT SPACES; MATHEMATICAL TECHNIQUES; TIME VARYING NETWORKS;

BINARY CLASSIFICATION PROBLEMS; CLASSIFICATION ALGORITHM; GRADIENT DESCENT ALGORITHMS; HYPOTHESIS SPACES; LEARNING RATES; LOSS FUNCTIONS; REPRODUCING KERNEL HILBERT SPACE; SAMPLE SIZES; STOCHASTIC GRADIENT DESCENT; STOCHASTIC GRADIENT DESCENT ALGORITHMS; UNDERLYING DISTRIBUTIONS;

LEARNING ALGORITHMS;

EID: 56449126442     PISSN: 03770427     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cam.2008.04.022     Document Type: Article

References (22)

1. Aronszajn N. Theory of reproducing kernels. Trans. Amer. Math. Soc. 68 (1950) 337-404
2. Azoury K.S., and Warmuth M.K. Relative loss bounds for on-line density estimation with exponential family of distributions. Mach. Learn. 43 (2001) 211-246
3. Bartlett P., Jordan M.I., and McAuliffe J.D. Convexity, classification and risk bounds. J. Amer. Statist. Assoc. 101 (2006) 138-156
4. Bartlett P.L., and Mendelson S. Rademacher and Gaussian complexities: Risk bounds and structural results. J. Mach. Learn. Res. 3 (2002) 463-482
5. Bousquet A., and Elisseeff F. Stability and generalization. J. Mach. Learn. Res. 2 (2002) 499-526
6. Cesa-Bianchi N., Long P., and Warmuth M.K. Worst-case quadratic loss bounds for prediction using linear functions and gradient descent. IEEE Trans. Neural Netw. 7 (1996) 604-619
7. Chen D.R., Wu Q., Ying Y., and Zhou D.X. Support vector machine soft margin classifiers: Error analysis. J. Mach. Learn. Res. 5 (2004) 1143-1175
8. Cucker F., and Smale S. On the mathematical foundations of learning. Bull. Amer. Math. Soc. 39 (2001) 1-49
9. Dong X., and Zhou D.-X. Learning gradients by a gradient descent algorithm. J. Math. Anal. Appl. 2 314 (2008) 1018-1027
10. Dudley R.M. Uniform central limit theorems, covariances via gradients. Combridge Studies in Advanced Mathematics vol. 63 (1999), Combridge university Press
11. Kivinen J., Smola A.J., and Williamson R.C. Online learning with kernels. IEEE Trans. Signal Process. 52 (2004) 2165-2176
12. Saitoh S. Theory of Reproducing Kernels and its Applications (1998), Longman Scientifical & Technical
13. Scovel C., and Steinwart I. Fast rates for support vector machines using Gaussian kernels. Ann. Statist. 35 (2007) 575-607 279-294
14. Steinwart I. Support vector machines are uniformly consistent. J. Complexity 18 (2002) 768-791
15. Smale S., and Yao Y. Online learning algorithms. Found. Comp. Math. 6 (2006) 145-170
16. Smale S., and Zhou D.X. Learning theory estimates via integral operators and their approximations. Constr. Approx. 26 (2007) 153-172
17. S. Smale, D.X. Zhou, Online learning with Markov sampling, 2007 (submitted for publication)
18. Van de Vaart A.W., and Wellner J.A. Weak Convergence and Empirical Processes (1996), Springer-Verlag
19. Yao Y., Rosasco L., and Caponnetto A. On early stopping in gradient descent learning. Constr. Approx. 26 (2007) 289-315
20. Ye G.B., and Zhou D.X. Fully online classification by regularization. Appl. Comput. Harmon. Anal. 23 (2007) 198-214
21. Ying Y., and Zhou D.X. Online regularized classification algorithms. IEEE Trans. Inform. Theory 52 (2006) 4775-4788
22. Zhang T. Statistical behavior and consistency of classification methods based on convex risk minimization. Ann. Statist. 32 (2004) 56-85