Tighter PAC-Bayes bounds through distribution-dependent priors

Theoretical Computer Science

Volumn 473, Issue , 2013, Pages 4-28

  Lever, Guy ^a   Laviolette, François ^b   Shawe Taylor, John ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITÉ LAVAL   (Canada)

Author keywords

Exponential weights algorithm; Localized bounds; PAC Bayes; Semi supervised learning; Statistical learning theory; SVM

Indexed keywords

LOCALIZED BOUNDS; PAC-BAYES; SEMI-SUPERVISED LEARNING; STATISTICAL LEARNING THEORY; SVM;

SUPERVISED LEARNING;

ALGORITHMS;

EID: 84873278768     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2012.10.013     Document Type: Conference Paper

References (31)

1. A. Ambroladze, E. Parrado-Hernández, and J. Shawe-Taylor Tighter PAC-Bayes bounds B. Schölkopf, J. C. Platt, T. Hoffman, Advances in Neural Information Processing Systems 19 (NIPS 2006) 2006 MIT Press pp. 9-16
2. J.-Y. Audibert, 2010. PAC-Bayesian aggregation and multi-armed bandits. Habilitation thesis, Université Paris-Est.
3. K. Azuma Weighted sums of certain dependent random variables Tohoku Mathematical Journal 68 1967 357 367
4. M. Balcan, and A. Blum A discriminative model for semi-supervised learning Journal of the ACM 57 3 2010
5. M. Belkin, I. Matveeva, and P. Niyogi Regularization and semi-supervised learning on large graphs J. Shawe-Taylor, Y. Singer, 17th Annual Conference on Learning Theory (COLT 2004) Lecture Notes in Computer Science, vol. 3120 2004 Springer pp. 624-638
6. M. Belkin, P. Niyogi, and V. Sindhwani Manifold regularization: a geometric framework for learning from labeled and unlabeled examples Journal of Machine Learning Research 7 2006 2399 2434
7. G. Blanchard, and F. Fleuret Occam's hammer N. H. Bshouty, C. Gentile, 20th Annual Conference on Learning Theory (COLT 2007) Lecture Notes in Computer Science, vol. 4539 2007 Springer pp. 112-126
8. O. Bousquet, O. Chapelle, and M. Hein Measure based regularization S. Thrun, L. K. Saul, B. Schölkopf, Advances in Neural Information Processing Systems 16 (NIPS 2003) 2003 MIT Press
9. O. Bousquet, and A. Elisseeff Stability and generalization Journal of Machine Learning Research 2 2002 499 526
10. O. Catoni, 2007. PAC-Bayesian supervised classification: the thermodynamics of statistical learning, Monograph Series of the Institute of Mathematical Statistics. Institute of Mathematical Statistics.
11. N. Cesa-Bianchi, Y. Freund, D. Haussler, D. P. Helmbold, R. E. Schapire, and M. K. Warmuth How to use expert advice Journal of the ACM 44 3 1997 427 485
12. O. Chapelle, B. Schölkopf, and A. Zien Semi-Supervised Learning (Adaptive Computation and Machine Learning) 2006 The MIT Press
13. G. DaPrato An Introduction to Infinite-Dimensional Analysis 2006 Springer
14. B. A. Frigyik, S. Srivastava, and M. R. Gupta Functional Bregman divergence and Bayesian estimation of distributions IEEE Transactions on Information Theory 54 2008 5130 5139
15. P. Germain, A. Lacasse, F. Laviolette, and M. Marchand PAC-Bayesian learning of linear classifiers A. P. Danyluk, L. Bottou, M. L. Littman, Proceedings of the 26th Annual International Conference on Machine Learning (ICML 2009) ACM International Conference Proceeding Series, Vol. 382 2009 ACM p. 45
16. M. Hein, J.-Y. Audibert, and U. von Luxburg Graph laplacians and their convergence on random neighborhood graphs Journal of Machine Learning Research 8 2007 1325 1370
17. O. Kallenberg, and R. Sztencel Some dimension-free features of vector-valued martingales Probability Theory and Related Fields 88 1991 215 247
18. M. Krein Hermitian-positive kernels on homogeneous spaces American Mathematical Society Translations series 2 34 1963 69 164
19. J. Langford Tutorial on practical prediction theory for classification Journal of Machine Learning Research 6 2005 273 306
20. J. Langford, and J. Shawe-Taylor PAC-Bayes & margins S. Becker, S. Thrun, K. Obermayer, Advances in Neural Information Processing Systems 15 (NIPS 2002) 2002 MIT Press pp. 423-430
21. N. Littlestone, M. K. Warmuth, The weighted majority algorithm. In: IEEE Symposium on Foundations of Computer Science, 1989, pp. 256-261.
22. D. A. McAllester PAC-Bayesian model averaging S. Ben-David, P. M. Long, Proceedings of the Twelfth Annual Conference on Computational Learning Theory (COLT 1999) 1999 ACM pp. 164-170
23. L. Ralaivola, M. Szafranski, and G. Stempfel Chromatic PAC-Bayes bounds for non-iid data: applications to ranking and stationary β-mixing processes Journal of Machine Learning Research 11 2010 1927 1956
24. M. Seeger PAC-Bayesian generalisation error bounds for gaussian process classification Journal of Machine Learning Research 3 2002 233 269
25. R. Serfling Approximation Theorems of Mathematical Statistics 1980 Wiley
26. J. Shawe-Taylor, P. L. Bartlett, R. C. Williamson, and M. Anthony Structural risk minimization over data-dependent hierarchies IEEE Transactions on Information Theory 44 5 1998 1926 1940
27. V. Sindhwani, P. Niyogi, and M. Belkin Beyond the point cloud: from transductive to semi-supervised learning L. D. Raedt, S. Wrobel, Proceedings of the Twenty-Second International Conference on Machine Learning (ICML 2005) ACM International Conference Proceeding Series, vol. 119 2005 ACM pp. 824-831
28. I. Steinwart, and A. Christmann Support Vector Machines 2008 Springer
29. V. G. Vovk Aggregating strategies M. A. Fulk, J. Case, Proceedings of the Third Annual Workshop on Computational Learning Theory (COLT 1990) 1990 Morgan Kaufmann pp. 371-386
30. G. Wahba Spline Models for Observational Data 1990 SIAM
31. X. Zhu, Z. Ghahramani, and J. D. Lafferty Semi-supervised learning using gaussian fields and harmonic functions T. Fawcett, N. Mishra, Proceedings of the Twentieth International Conference on Machine Learning, (ICML 2003) 2003 AAAI Press pp. 912-919