Minimal penalties for Gaussian model selection

Probability Theory and Related Fields

Volumn 138, Issue 1-2, 2007, Pages 33-73

  Birgé, Lucien ^a   Massart, Pascal ^b  

^a SORBONNE UNIVERSITÉ   (France)

^b UNIVERSITÉ PARIS SUD   (France)

Author keywords

Gaussian linear regression; Mallows' Cp; Model selection; Penalized least squares; Variable selection

Indexed keywords

EID: 33847613502     PISSN: 01788051     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00440-006-0011-8     Document Type: Article

References (41)

1. Abramovich, F., Benjamini, Y., Donoho, D.L., Johnstone, I.M.: Adapting to unknown sparsity by controlling the false discovery rate. Ann. Statist. 34, (2006)
2. Akaike, H.: Statistical predictor identification. Ann. Inst. Statist. Math. 22, 203-217 (1969)
3. Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Petrov, P.N., Csaki, F. (eds.) Proceedings 2nd International Symposium on Information Theory, Akademia Kiado, Budapest, pp. 267-281 (1973)
4. Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19, 716-723 (1974)
5. Akaike, H.: A Bayesian analysis of the minimum AIC procedure. Ann. Inst. Statist. Math. 30, Part A, 9-14 (1978)
6. Amemiya, T.: Advanced Econometrics. Basil Blackwell, Oxford (1985)
7. Barron, A.R., Birgé, L., Massart, P.: Risk bounds for model selection via penalization. Probab. Theory Relat. Fields 113, 301-415 (1999)
8. Barron, A.R., Cover, T.M.: Minimum complexity density estimation. IEEE Trans. Inf. Theory 37, 1034-1054 (1991)
9. Birgé, L.: An alternative point of view on Lepski's method. In: de Gunst, M.C.M., Klaassen, C.A.J., van der Vaart, A.W. (eds.) State of the Art in Probability and Statistics, Festschrift for Willem R. van Zwet, Institute of Mathematical Statistics, Lecture Notes-Monograph Series, Vol. 36. 113-133 (2001)
10. Birgé, L., Massart, P.: Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli 4, 329-375 (1998)
11. Birgé, L., Massart, P.: Gaussian model selection. J. Eur. Math. Soc. 3, 203-268 (2001)
12. p criterion for Gaussian model selection. Technical Report No 647. Laboratoire de Probabilités, Université Paris VI (2001) http://www.proba. jussieu.fr/mathdoc/preprints/index.html#2001
13. Daniel, C., Wood, F.S.: Fitting Equations to Data. Wiley, New York (1971)
14. Draper, N.R., Smith, H.: Applied Regression Analysis, 2nd edn. Wiley, New York (1981)
15. Efron, B., Hastie, R., Johnstone, I.M., Tibshirani, R.: Least angle regression. Ann. Statist. 32, 407-499 (2004)
16. Feller, W.: An Introduction to Probability Theory and its Applications, Vol. I (3rd edn.). Wiley, New York (1968)
17. George, E.I., Foster, D.P.: Calibration and empirical Bayes variable selection. Biometrika 87, 731-747 (2000)
18. Gey, S., Nédélec, E.: Model selection for CART regression trees. IEEE Trans. Inf. Theory 51, 658-670 (2005)
19. Guyon, X., Yao, J.F.: On the underfitting and overfitting sets of models chosen by order selection criteria. Jour. Multivar. Anal. 70, 221-249 (1999)
20. Hannan, E.J., Quinn, B.G.: The determination of the order of an autoregression. J.R.S.S., B 41, 190-195 (1979)
21. Hoeffding, W.: Probability inequalities for sums of bounded random variables. J.A.S.A. 58, 13-30 (1963)
22. Hurvich, K.L., Tsai, C.-L.: Regression and time series model selection in small samples. Biometrika 76, 297-307 (1989)
23. Johnstone, I.: Chi-square oracle inequalities. In: de Gunst, M.C.M., Klaassen, C.A.J. van der Vaart, A.W. (eds.) State of the Art in Probability and Statistics, Festschrift for Willem R. van Zwet, Institute of Mathematical Statistics, Lecture Notes-Monograph Series, Vol. 36. pp. 399-418 (2001)
24. Kneip, A.: Ordered linear smoothers. Ann. Statist. 22, 835-866 (1994)
25. Lavielle, M., Moulines, E.: Least Squares estimation of an unknown number of shifts in a time series. J. Time Series Anal. 21, 33-59 (2000)
26. Lebarbier, E.: Detecting multiple change-points in the mean of a Gaussian process by model selection. Signal Proces. 85, 717-736 (2005)
27. L, cross-validation, and generalized cross-validation: Discrete index set. Ann. Statist. 15, 958-975 (1987)
28. Loubes, J.-M., Massart, P.: Discussion of "Least angle regression" by Efron, B., Hastie, R., Johnstone, I., Tibshirani, R. Ann. Statist. 32, 460-465 (2004)
29. p. Technometrics 15, 661-675 (1973)
30. Massart, P.: The tight constant in the D.K.W. inequality. Ann. Probab. 18, 1269-1283 (1990)
31. McQuarrie, A.D.R., Tsai, C.-L.: Regression and Time Series Model Selection. World Scientific, Singapore (1998)
32. Mitchell, T.J., Beauchamp, J.J.: Bayesian variable selection in linear regression. J.A.S.A. 83, 1023-1032 (1988)
33. p-test for the orthogonal series estimation of regression. Theory Probab. Appl. 35, 293-306 (1990)
34. Rissanen, J.: Modeling by shortest data description. Automatica 14, 465-471 (1978)
35. Schwarz, G.: Estimating the dimension of a model. Ann. Statist. 6, 461-464 (1978)
36. Shen, X., Ye, J.: Adaptive model selection. J.A.S.A. 97, 210-221 (2002)
37. Shibata, R.: An optimal selection of regression variables. Biometrika 68, 45-54 (1981)
38. Wallace,D.L.: Bounds on normal approximations to Student's and the chi-square distributions. Ann. Math. Stat. 30, 1121-1130 (1959)
39. Whittaker, E.T., Watson, G.N.: A Course of Modern Analysis. Cambridge University Press, London (1927)
40. Yang, Y.: Can the strenghths of AIC and BIC be shared? A conflict between model identification and regression estimation. Biometrika 92, 937-950 (2005)
41. Yao, Y.C.: Estimating the number of change points via Schwarz criterion. Stat. Probab. Lett. 6, 181-189 (1988)