Concentration inequalities and model selection

Lecture Notes in Mathematics

Volumn 1896, Issue , 2007, Pages 1-324

  Massart, Pascal ^a  

^a UNIVERSITÉ PARIS SACLAY   (France)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34247553430     PISSN: 00758434     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-3-540-48503-2     Document Type: Article

References (128)

1. ADLER, R.J. An introduction to continuity, extrema and related topics for general Gaussian processes. Institute of Mathematical Statistics Lecture Notes-Monograph Series, 12 (1990).
2. AKAIKE, H. Information theory and an extension of the maximum likelihood principle. In P.N. Petrov and F. Csaki, editors, Proceedings 2nd International Symposium on Information Theory, pages 267-281. Akademia Kiado, Budapest, 1973.
3. ALDOUS, D.J. Exchangeability and related topics. In Ecole d'Eté de Probabilités de St-Flour 1983, 1-198 Springer Verlag Lecture Notes in Mathematics 1117 (1985).
4. ALEXANDER, K.S. Rates of growth and sample moduli for weighted empirical processes indexed by sets. Probab. Theory Rel. Fields 75 no3, 379-423 (1987).
5. ANÉ, C. et al. Sur les inégalités de Sobolev logarithmiques. Panoramas et Synthèses, vol. 10. Soc. Math. de France (2000).
6. ASSOUAD, P. Densité et dimension. Ann. Inst. Fourier 33, no3, 233-282 (1983).
7. BAHADUR., R.,R., Examples of inconsistency of maximum likelihood estimates. Sankhya Ser.A 20, 207-210 (1958).
8. BARAUD, Y. Model selection for regression on a fixed design. Probability Theory and Related Fields 117, no 4 467-493 (2000).
9. BARAUD, Y., COMTE, F. and VIENNET, G. Model selection for (auto-)regression with dependent data. ESAIM: Probability and Statistics 5, 33-49 (2001) http://www.emath.fr/ps/.
10. BARRON, A.R. and COVER, T.M. Minimum complexity density estimation. IEEE Transactions on Information Theory 37, 1034-1054 (1991).
11. BARRON, A.R. and SHEU, C.H. Approximation of density functions by sequences of exponential families. Ann. Statist. 19, 1054-1347 (1991).
12. BARRON, A.R., BIRGÉ, L. and MASSART, P. Risk bounds for model selection via penalization. Probab. Th. Rel. Fields. 113, 301-415 (1999).
13. BARTLETT, P., BOUCHERON, S. and LUGOSI, G. Model selection and error estimation. Machine Learning 48, 85-113 (2001).
14. BARTLETT, P., BOUSQUET, O. and MENDELSON, S. Local Rademacher Complexities (submitted) (2005).
15. BENNETT, G. Probability inequalities for the sum of independent random variables. Journal of the American Statistical Association 57, 33-45 (1962).
16. BLRGÉ, L. Approximation dans les espaces métriques et théorie de l'estimation. Z. Wahrscheinlichkeitstheorie Venu. Geb. 65, 181-237 (1983).
17. BLRGÉ, L. A new lower bound for multiple hypothesis testing. IEEE Trans. Inform. Theory. 51, 1611-1615 (2005).
18. BIRGÉ, L. Model selection via testing: an alternative to (penalized) maximum likelihood estimators. Ann. Inst. Henri Poincaré 42, 273-325 (2006).
19. BIRGÉ, L. and MASSART, P. Rates of convergence for minimum contrast estimators. Probab. Th. Relat. Fields 97, 113-150 (1993).
20. BIRGÉ, L. and MASSAHT, P. From model selection to adaptive estimation. In Festschrift for Luden Lecam: Research Papers in Probability and Statistics (D. Pollard, E. Torgersen and G. Yang, eds.), 55-87 (1997) Springer-Verlag, NewYork.
21. BIRGÉ, L. and MASSART, P. Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli, 4 (3), 329-375 (1998).
22. BIRGÉ, L. and MASSART, P. An adaptive compression algorithm in Besov spaces. Constructive Approximation 16, 1-36 (2000).
23. BIRGÉ, L. and MASSART, P. Gaussian model selection. Journal of the European Mathematical Society, no 3, 203-268 (2001).
24. p criterion for Gaussian model selection. Prépublication, no 647, Universités de Paris 6 & Paris 7 (2001).
25. BIRGÉ, L. and MASSART, P. Minimal penalties for Gaussian model selection. Probab. Th. Relat. Fields (to appear).
26. BIRGÉ, L. and ROZENHOLC, Y. How many bins should be put in a regular histogram. ESAIM: Probability and Statistics 10, 24-45 (2006).
27. BOBKOV, S. On Gross' and Talagrand's inequalities on the discrete cube. Vestnik of Syktyvkar University Ser. 1,1 12-19 (1995) (in Russian).
28. BOBKOV, S. and GOETZE, F. Exponential integrability and transportation cost under logarithmic Sobolev inequalities. J. Fund. Anal. 163, no1 1-28 (1999).
29. BORELL, C. The Brunn-Minkowski inequality in Gauss space. Invent. Math. 30, 207-216 (1975).
30. BOUCHERON, S., BOUSQUET, O., LUGOSI, G. and MASSART, P. Moment inequalities for functions of independent random variables. Ann. of Probability 33, no2, 514-560 (2005).
31. BOUCHERON, S., LUGOSI, G. and MASSART, P. A sharp concentration inequality with applications. Random Structures and Algorithms 16, no3, 277-292 (2000).
32. BOUCHERON, S., LUGOSI, G. and MASSART, P. Concentration inequalities using the entropy method. Ann. Probab. 31, no3, 1583-1614 (2003).
33. BOUSQUET, O. A Bennett concentration inequality and its application to suprema of empirical processes. C.B. Math. Acad. Sci. Paris 334 no6, 495-500 (2002).
34. CASTELLAN, G. Modified Akaike's criterion for histogram density estimation. Technical report #99.61, (1999) Université de Paris-Sud.
35. CASTELLAN, G. Density estimation via exponential model selection. IEEE Trans. Inform. Theory 49 no8, 2052-2060 (2003).
36. CIREL'SON, B.S., IBRAGIMOV, LA. and SUDAKOV, V.N. Norm of Gaussian sample function. In Proceedings of the 3rd Japan-U.S.S.R. Symposium on Probability Theory, Lecture Notes in Mathematics 550, 20-41 (1976) Springer-Verlag, Berlin.
37. CIREL'SON, B.S. and SUDAKOV, V.N. Extremal properties of half spaces for spherically invariant measures. J. Soviet. Math. 9, 9-18 (1978);
38. translated from Zap. Nauch. Sem. L.O.M.I. 41, 14-24 (1974).
39. COHEN, A., DAUBECHIES, I. and VIAL, P. Wavelets and fast wavelet transform on an interval. Appl. Cornput. Harmon. Anal. 1, 54-81 (1993).
40. COVER, T.M. and THOMAS, J.A. Elements of Information Theory. Wiley series in telecommunications. Wiley (1991).
41. CSISZAR., I. and K̂RNER, J. Information Theory: Coding Theorems for discrete Memory-less Systems. Academic Press, New York (1981).
42. DANIEL, C. and WOOD, F.S. Fitting Equations to Data. Wiley, New York (1971).
43. DEMBO, A. Information inequalities and concentration of measure. Ann. Probab. 25, 927-939 (1997).
44. DEVORE, R.A., KYRIZIAKIS, G., LEVIATAN, D. and TIKHOMIROV, V.M. Wavelet compression and nonlinear n-widths. Adv. Computational Math. 1, 197-214 (1993).
45. DEVORE, R.A. and LORENTZ, G.G. Constructive Approximation. Springer-Verlag, Berlin (1993).
46. DEVROYE, L. and LUGOSI, G. Lower bounds in pattern recognition. Pattern recognition 28, 1011-1018 (1995).
47. DOBRUSHIN, R.L. Prescribing a system of random variables by conditional distributions. Theor. Prob. Appl. 15, 458-486 (1970).
48. DONOHO, D.L. and JOHNSTONE, I.M. Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 425-455 (1994).
49. q,-error. Probab. Theory Relat. Fields 99, 277-303 (1994).
50. DONOHO, D.L. and JOHNSTONE, I.M. Ideal denoising in an orthonormal basis chosen from a library of bases. C. R. Acad. Se. Paris Ser. I Math. 319, 1317-1322 (1994).
51. DONOHO, D.L. and JOHNSTONE, I.M. Adapting to unknown smoothness via wavelet shrinkage. JASA. 90, 1200-1224 (1995).
52. DONOHO, D.L. and JOHNSTONE, I.M. Neo-classical minimax problems, thres-holding and adaptive function estimation. Bernoulli 2, 39-62 (1996).
53. DONOHO, D.L. and JOHNSTONE, I.M. Minimax estimation via wavelet shrinkage. Ann. Statist. 26, 879-921 (1998).
54. DONOHO, D.L. and JOHNSTONE, I.M., K ERKYACHARIAN, G. and PICARD, D. Wavelet shrinkage: Asymptopia? J.R. Statist. Soc. B 57, 301-369 (1995).
55. DONOHO, D.L. and JOHNSTONE, I.M., K ERKYACHARIAN, G. and PICARD, D. Density estimation by wavelet thresholding. Ann. Statist. 24, 508-539 (1996).
56. DRAPER, N.R. and SMITH, H. Applied Regression Analysis, second edition. Wiley, New York (1981).
57. DUDLEY, R.M. The sizes of compact subsets of Hilbert space and continuity of Gaussian processes. J. Fund. Anal. 1, 290-330 (1967).
58. DUDLEY, R.M. Uniform Central Limit Theorems. Cambridge Studies in advanced mathematics 63, Cambridge University Press (1999).
59. EFROIMOVITCH, S.Yu. and PINSKER, M.S. Learning algorithm for nonparametric filtering. Automat. Remote Control 11, 1434-1440 (1984),
60. translated from Avtomatika, i Telemekhanika 11, 58-65.
61. EFRON, B. and STEIN, C. The Jacknife estimate of variance. Ann. Statist. 9, 586-596 (1981).
62. FEDERER, H. Geometric measure theory. Springer (1969).
63. FERNIQUE, X. Régularité des trajectoires des fonctions aléatoires gaussiennes. Ecole d'Eté de Probabilités de St Flour 1974. Lecture Notes in Mathematics (Springer), 480 (1975).
64. FERNIQUE, X. Fonctions aléatoires gaussiennes, vecteurs aléatoires gaussiens. Les publications CRM, Montréal (1997).
65. FIGIEL, T., LINDENSTRAUSS, J., MILMAN, V.D. The dimensions of almost spherical sections of convex bodies. Acta Math. 139, 52-94 (1977).
66. GROMOV, M. Paul Lévy's isoperimetric inequality. Preprint I.H.E.S. (1980).
67. GROSS, L. Logarithmic Sobolev inequalities. Amer. J. Math. 97, 1061-1083 (1975).
68. HAUSSLER, D. Sphere packing numbers for subsets of the boolean n-cube with bounded Vapnik-Chervonenkis dimension. Journal of Combinatorial Theory Series A, 69, 217-232 (1995).
69. HAUSSLER., D., LITTLESTONE, N. and WARMUTH, M. Predicting {0,1}-functions on randomly drawn points. Information and Computation, 115, 248-292 (1994).
70. HOEFFDING, W. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58, 13-30 (1963).
71. KARPINSKI, M. and MACINTYRE, A. Polynomial bounds for VC dimension of sigmoidal neural networks. Proceedings of the 27th annual ACM symposium on the theory of computing (STOC). Las Vegas. NV, USA, May 29 - June 1, 1995, NY: ACM, 200-208 (1995).
72. KOLTCHINSKII, V. Rademacher penalties and structural risk minimization. IEEE Transactions on Information Theory 47, 1902-1914 (2001).
73. KOLTCHINSKII, V. 2004 IMS Medallion Lecture: Local Rademacher complexities and oracle inequalities in risk minimization. Annals of Statistics (to appear).
74. KOROSTELEV, A.P. and TSYBAKOV, A.B. Minimax theory of image reconstruction. Lectures notes in Statistics 82, Springer Verlag, New York (1993).
75. LATALA, R. and OLESZKIEWICZ, C. Between Sobolev and Poincaré. In Geometric aspects of Functional Analysis, Israel Seminar (GAFA), 1996-2000, pages 147-168 Springer, 2000. Lecture Notes in Mathematics 1745.
76. LEBARBIER, E. Detecting multiple change points in the mean of Gaussian process by model selection. Signal Processing 85, no4, 717-736 (2005).
77. LE CAM, L. M. Convergence of estimates under dimensionality restrictions. Ann. Statist, 1, 38-53 (1973).
78. LEDOUX, M. Isoperimetry and Gaussian Analysis. In Lectures on Probability Theory and Statistics, Ecole d'Eté de Probabilités de St-Flour XXIV-1994 (P. Bernard, ed.), 165-294 (1996) Springer, Berlin.
79. LEDOUX, M. On Talagrand deviation inequalities for product measures. ESAIM: Probability and Statistics 1, 63-87 (1996) http://www.emath.fr/ps/.
80. LEDOUX, M. The concentration of measure phenomenon. Mathematical Surveys and Monographs 89, American Mathematical Society.
81. LEDOUX, M. and TALAGRAND, M. Probability in Banach spaces (Isoperimetry and processes). Ergebnisse der Mathematik und ihrer Grenzgebiete (1991) Springer-Verlag.
82. LEPSKII, O.V. On a problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 36, 454-466 (1990).
83. LEPSKII, O.V. Asymptotically minimax adaptive estimation I: Upper bounds. Optimally adaptive estimates. Theory Probab. Appl. 36, 682-697 (1991).
84. LÉVY, P. Problèmes concrets d'analyse fonctionnelle. Gauthier-Villars (1951).
85. LUGOSI, G. Pattern classification and learning theory. Principles of Nonparametric Learning (L. Györfi ed.) Springer, Wien, New York, 1-56 (2002).
86. MALLOWS, C.L. Some comments on Cp. Technometrics 15, 661-675 (1973).
87. MAMMEN, E. and TSYBAKOV, A.B. Smooth discrimination analysis. Ann. Statist. 27, No6, 1808-1829 (1999).
88. MARTON, K. A simple proof of the blowing up lemma. IEEE Trans. Inform. Theory IT-32, 445-446 (1986).
89. MARTON, K. Bounding d-distance by information divergence: a method to prove measure concentration. Ann. Probab. 24, 927-939 (1996).
90. MARTON, K. A measure concentration inequality for contracting Markov chains. Geom. Funct. Anal. 6, no3, 556-571 (1996).
91. MASON, D.M. and VAN ZWET, W.R. A refinement of the KMT inequality for the uniform empirical process. Ann. Probab. 15, 871-884 (1987).
92. MASSART, P. About the constants in Talagrand's concentration inequalities for empirical processes. Ann. Probab. 28, no2, 863-884 (2000).
93. MASSART, P. Some applications of concentration inequalities to Statistics. Probability Theory. Annales de la Faculté des Sciences de Toulouse (6) 9, no2, 245-303 (2000).
94. MASSART, P. and NÉDÉLEC, E. Risk bounds for statistical learning. Ann. Statist. 34, no5, 2326-2366.
95. MCDIARMLD, C. On the method of bounded differences. In Surveys in Combinatorics 1989, pages 148-188. Cambridge University Press, Cambridge (1989).
96. MEYER, Y. Ondelettes et Opérateurs I. Hermann, Paris (1990).
97. MILMAN, V.D. and SCHECHTMAN, G. Asymptotic theory of finite dimensional normed spaces. Lecture Notes in Mathematics (Springer), 1200 (1986).
98. MISITI, M., MISITI, Y., OPPENHEIM, G. and POGGI, J.M. Matlab Wavelet Toolbox. The Math Works Inc., Natick (1996).
99. PINELIS, I. Optimum bounds on moments of sums of independent random vectors. Siberian Advances in Mathematics, 5, 141-150 (1995).
100. PINSKER, M.S. Information and Information Stability of Random Variables and Processes. Holden-Day, San Francisco (1964).
101. PINSKER., M.S. Optimal filtration of square-integrable signals in Gaussian noise. Problems of Information Transmission 16, 120-133 (1980).
102. PISIER, G. Some applications of the metric entropy condition to harmonic analysis, in Banach spaces, Harmonic analysis and Probability, Univ. of Connecticut 1980-81. Springer, Berlin Heidelberg 1983, p. 123-159. Lecture Notes in Mathematics, 995.
103. RACHEV, S.T. Probability metrics and the stability of stochastic models. Wiley Series in Probability and Mathematical Statistics, Wiley Sons Ltd XIV 494 p. (1991).
104. REYNAUD-BOURET, P. Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities. Probab. Theory Relat. Fields 126, no1, 103-153 (2003).
105. RIO, E. Une inégalité de Bennett pour les maxima, de processus empiriques. Ann. I. H. Poincaréμ. 38, no6 1053-1057 (2002).
106. RUDEMO, M. Empirical choice of histograms and kernel density estimators. Scand. J. Statist. 9, 65-78 (1982).
107. SAMSON, P.M. Concentration of measure inequalities for Markov chains and Φ -mixing processes. Ann. Probab. 28, 416-461 (2000).
108. SCHMIDT, E. Die Brunn-Minkowskische Ungleichung und ihr Spiegelbild sowie die isoperimetrische Eigenschaft der Kugel in der euklidischen und nichteuklidischen Geometrie. Math. Nach, 1, 81-15 (1948).
109. SCHUMAKER, L.L. Spline Functions: Basic Theory. Wiley, New York (1981).
110. STRASSEN, V. The existence of probability measures with given marginals. Ann. Math. Statist. 36, 423-439 (1965).
111. TALAGRAND, M. Regularity of Gaussian processes. Acta Math. 159, 99-149 (1987).
112. TALAGRAND, M. An isoperimetric theorem on the cube and the Khintchine-Kahane inequalities in product spaces. Proc. Amer. Math. Soc. 104, 905-909 (1988).
113. TALAGRAND, M. Sharper bounds for empirical processes. Ann. Probab. 22, 28-76 (1994).
114. TALAGRAND, M. Concentration of measure and isoperimetric inequalities in product spaces. Publications Mathématiques de l'I.H.E.S. 81, 73-205 (1995).
115. TALAGRAND, M. New concentration inequalities in product spaces. Invent. Math. 126, 505-563 (1996).
116. TALAGRAND, M. Transportation cost for Gaussian and other product measures. Geometric and Functional Analysis 6, 587-600 (1996).
117. TSYBAKOV, A.B. Optimal Aggregation of Classifiers in Statistical Learning. Ann. Statist. 32, no1 (2004).
118. TSYBAKOV, A.B. and VAN DE GEER, S. Square root penalty: adaptation to the margin in classification and in edge estimation. Prépublication PMA-820, Laboratoire de Probabilités et Modèles Aléatoires, Université Paris VI, (2003).
119. USPENSKY, J.V. Introduction to Mathematical Probability. New York: McGraw-Hill (1937).
120. VAN DE GEER, S. The method of sieves and minimum contrast estimators. Math. Methods Statist. 4, 20-38 (1995).
121. VAN DER VAART, A. Asymptotic Statistics. Cambridge University Press (1998).
122. VAN DER VAART, A. and WELLNER J. Weak Convergence and Empirical Processes. Springer, New York (1996).
123. VAPNIK, V.N. Estimation of dependencies based on empirical data. Springer, New York (1982).
124. VAPNIK, V.N. Statistical learning theory. J. Wiley, New York (1990).
125. VAPNIK, V.N. and CHERVONENKIS A. Ya. Theory of pattern recognition. Nauka, Moscow, 1974. (in Russian); German translation; Theorie der Zeichenerkennung, Akademie Verlag, Berlin, 1979.
126. WAHBA, G. Spline Models for Observational Data. S.I.A.M., Philadelphia (1990).
127. WHITTAKER, E.T., WATSON, G.N. A Course of Modern Analysis. Cambridge University Press, London (1927).
128. YANG, Y. and BARRON, A.R. Information-theoretic determination of minimax rates of convergence. Ann. Statist. 27, no5, 1564-1599 (1999).