Stability bounds for stationary φ-mixing and β-mixing processes

Journal of Machine Learning Research

Volumn 11, Issue , 2010, Pages 789-814

  Mohri, Mehryar ^a   Rostamizadeh, Afshin ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

Algorithmic stability; Generalization bounds; Learning in non i.i.d. scenarios; Learning theory; Mixing distributions; Weakly dependent observations

Indexed keywords

ALGORITHMIC STABILITY; GENERAL CLASS; GENERALIZATION BOUND; INDEPENDENTLY AND IDENTICALLY DISTRIBUTED; LEARNING THEORY; MACHINE LEARNING APPLICATIONS; MIXING DISTRIBUTIONS; MIXING PROCESS; REGULARIZATION ALGORITHMS; RELATIVE ENTROPY; RIDGE REGRESSION; SPECIFIC LEARNING; STABILITY ANALYSIS; SUPPORT VECTOR REGRESSIONS; TEMPORAL DEPENDENCE; THEORETICAL BASIS;

MIXING; REGRESSION ANALYSIS; STABILITY;

LEARNING ALGORITHMS;

EID: 77949498582     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (25)

1. Sergei Natanovich Bernstein. Sur l'extension du théorème limite du calcul des probabilités aux sommes de quantités dépendantes. Mathematische Annalen, 97:1-59, 1927.
2. Olivier Bousquet and André Elisseeff. Algorithmic stability and generalization performance. In Advances in Neural Information Processing Systems, 2001.
3. Olivier Bousquet and André Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2:499-526, 2002.
4. Jean-René Chazottes, Pierre Collet, Christof Külske, and Frank Redig. Concentration inequalities for random fields via coupling. Probability Theory and Related Fields, 137(1):201-225, 2007.
5. Corinna Cortes and Vladimir N. Vapnik. Support-vector networks. Machine Learning, 20(3):273-297, 1995.
6. Luc Devroye and T. J. Wagner. Distribution-free performance bounds for potential function rules. In Information Theory, IEEE Transactions on, volume 25, pages 601-604, 1979.
7. Paul Doukhan. Mixing: Properties and Examples. Springer-Verlag, 1994.
8. Michael Kearns and Dana Ron. Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. In Computational Learning Theory, pages 152-162, 1997.
9. Leo Kontorovich. Measure Concentration of Strongly Mixing Processes with Applications. PhD thesis, Carnegie Mellon University, 2007.
10. Leo Kontorovich and Kavita Ramanan. Concentration inequalities for dependent random variables via the martingale method. Annals of Probability, 36(6):2126-2158, 2008.
11. Aurélie Lozano, Sanjeev Kulkarni, and Robert Schapire. Convergence and consistency of regularized boosting algorithms with stationary b-mixing observations. In Advances in Neural Information Processing Systems, 2006.
12. Katalin Marton. Measure concentration for a class of random processes. Probability Theory and Related Fields, 110(3):427-439, 1998.
13. Davide Mattera and Simon Haykin. Support vector machines for dynamic reconstruction of a chaotic system. In Advances in Kernel Methods: Support Vector Learning, pages 211-241. MIT Press, Cambridge, MA, USA, 1999. ISBN 0-262-19416-3.
14. Colin McDiarmid. On the method of bounded differences. In Surveys in Combinatorics, pages 148-188. Cambridge University Press, 1989.
15. Ron Meir. Nonparametric time series prediction through adaptive model selection. Machine Learning, 39(1):5-34, April 2000.
16. DharmendraModha and EliasMasry. On the consistency in nonparametric estimation under mixing assumptions. IEEE Transactions of Information Theory, 44:117-133, 1998.
17. Mehryar Mohri and Afshin Rostamizadeh. Stability bounds for non-iid processes. In Advances in Neural Information Processing Systems, 2007.
18. Mehryar Mohri and Afshin Rostamizadeh. Rademacher complexity bounds for non-i.i.d. processes. In Advances in Neural Information Processing Systems (NIPS 2008), pages 1097-1104, Vancouver, Canada, 2009. MIT Press.
19. Klaus-Robert Müller, Alex Smola, Gunnar Rätsch, Bernhard Schölkopf, Jens Kohlmorgen, and Vladimir Vapnik. Predicting time series with support vector machines. In Proceedings of the International Conference on Artificial Neural Networks, Lecture Notes in Computer Science, pages 999-1004. Springer, 1997.
20. Paul-Marie Samson. Concentration of measure inequalities for Markov chains and-mixing processes. Annals Probability, 28(1):416-461, 2000.
21. Craig Saunders, Alexander Gammerman, and Volodya Vovk. Ridge regression learning algorithm in dual variables. In Proceedings of the Fifteenth International Conference on Machine Learning, pages 515-521. Morgan Kaufmann Publishers Inc., 1998.
22. Vladimir N. Vapnik. Statistical Learning Theory. Wiley-Interscience, New York, 1998.
23. Mathukumalli Vidyasagar. Learning and Generalization: with Applications to Neural Networks. Springer, 2003.
24. Bin Yu. Rates of convergence for empirical processes of stationary mixing sequences. The Annals of Probability, 22(1):94-116, Jan. 1994.
25. Shuheng Zhou, John Lafferty, and Larry Wasserman. Time varying undirected graphs. In Proceedings of the 21st Annual Conference on Learning Theory, 2008.