On the complexity of linear prediction: Risk bounds, margin bounds, and regularization

Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

Volumn , Issue , 2009, Pages 793-800

  Kakade, Sham M ^a   Sridharan, Karthik ^a   Tewari, Ambuj ^a  

^a TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; ENTROPY; RISK ASSESSMENT;

GAUSSIAN COMPLEXITY; GENERALIZATION ABILITY; INPUT VECTOR; LINEAR PREDICTION; MARGIN BOUNDS; PREDICTION RISKS; RADEMACHER COMPLEXITY; REGULARISATION; RISK BOUNDS; SHARP BOUNDS;

FORECASTING;

EID: 84855609314     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. P. L. Bartlett and S. Mendelson. Rademacher and Gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3:463-482, 2002.
2. P. L. Bartlett and J. Shawe-Taylor. Generalization performance of support vector machines and other pattern classifiers. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 43-54. MIT Press, 1999.
3. N. Cesa-Bianchi and G. Lugosi. Prediction, learning, and games. Cambridge University Press, 2006.
4. V. Koltchinskii and D. Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. Annals of Statistics, 30(1):1-50, 2002. (Pubitemid 37095367)
5. J. Langford and J. Shawe-Taylor. PAC-Bayes & margins. In Advances in Neural Information Processing Systems 15, pages 423-430, 2003.
6. J. Langford,M. Seeger, and NimrodMegiddo. An improved predictive accuracy bound for averaging classifiers. In Proceedings of the Eighteenth International Conference on Machine Learning, pages 290-297, 2001.
7. M. Ledoux and M. Talagrand. Probability in Banach spaces: Isoperimetry and processes, volume 23 of Ergebnisse der Mathematik und ihrer Grenzgebiete (3). Springer-Verlag, 1991.
8. David A. McAllester. Simplified PAC-Bayesian margin bounds. In Proceedings of the Sixteenth Annual Conference on Computational Learning Theory, pages 203-215, 2003.
9. David A. McAllester. PAC-Bayesian model averaging. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory, pages 164-170, 1999.
10. Ron Meir and Tong Zhang. Generalization error bounds for Bayesian mixture algorithms. Journal of Machine Learning Research, 4:839-860, 2003.
11. 2 regularization, and rotational invariance. In Proceedings of the Twenty-First International Conference on Machine Learning, 2004.
12. David Pollard. Convergence of Stochastic Processes. Springer-Verlag, 1984.
13. R.E. Schapire, Y. Freund, P. Bartlett, and W.S. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5):1651-1686, October 1998. (Pubitemid 128376902)
14. S. Shalev-Shwartz. Online Learning: Theory, Algorithms, and Applications. PhD thesis, The Hebrew University, 2007.
15. S. Shalev-Shwartz and Y. Singer. Convex repeated games and Fenchel duality. In Advances in Neural Information Processing Systems 20, 2006.
16. M. Warmuth and A. K. Jagota. Continuous versus discrete-time non-linear gradient descent: Relative loss bounds and convergence. In Fifth International Symposium on Artificial Intelligence and Mathematics, 1997.
17. T. Zhang. Covering number bounds of certain regularized linear function classes. Journal of Machine Learning Research, 2:527-550, 2002. (Pubitemid 135712571)