On Improved Loss Estimation for Shrinkage Estimators

Statistical Science

Volumn 27, Issue 1, 2012, Pages 61-81

  Fourdrinier, Dominique ^a,b   Wells, Martin T ^b  

^a UNIVERSITÉ DE ROUEN   (France)

^b Cornell University ^*  (United States)

Author keywords

Conditional inference, linear model; Loss estimation; Quadratic loss; Risk function; Robustness; Shrinkage estimation; Spherical symmetry; SURE; Unbiased estimator of loss; Uniform distribution on a sphere

Indexed keywords

EID: 84865063087     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/11-STS380     Document Type: Review

References (54)

1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory (Tsahkadsor, 1971) 267-281. Akadémiai Kiadó, Budapest. MR0483125
2. Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301-413. MR1679028
3. Bartlett, P., Boucheron, S. and Lugosi, G. (2002). Model selection and error estimation. Machine Learning 48 85-113.
4. Berger, J. (1985). The frequentist viewpoint and conditioning. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. I (Berkeley, Calif., 1983) (L. Le Cam and R. Olshen, eds.) 15-44.Wadsworth, Belmont, CA. MR0822033
5. Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York. MR0804611
6. Berger, J. O. (1985). In defense of the likelihood principle: Axiomatics and coherency. In Bayesian Statistics, 2 (Valencia, 1983) (D. V. Lindley, J. M. Bernardo, M. H. DeGroot and A. F. M. Smith, eds.) 33-66. North-Holland, Amsterdam. MR0862483
7. Blanchard, D. and Fourdrinier, D. (1999). Non-trivial solutions of non-linear partial differential inequations and order cut-off. Rend. Mat. Appl. (7) 19 137-154. MR1710109
8. Bock, M. E. (1988). Shrinkage estimators: Pseudo-Bayes rules for normal mean vectors. In Statistical Decision Theory and Related Topics, IV, Vol. 1 (West Lafayette, Ind., 1986) (S. S. Gupta and J. O. Berger, eds.) 281-297. Springer, New York. MR0927108
9. Brown, L. (1968). Inadmissiblity of the usual estimators of scale parameters in problems with unknown location and scale parameters. Ann. Math. Statist. 39 29-48. MR0222992
10. Brown, L. D. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855-903. MR0286209
11. Cellier, D. and Fourdrinier, D. (1995). Shrinkage estimators under spherical symmetry for the general linear model. J. Multivariate Anal. 52 338-351. MR1323338
12. Cellier, D., Fourdrinier, D. and ROBERT, C. (1989). Robust shrinkage estimators of the location parameter for elliptically symmetric distributions. J. Multivariate Anal. 29 39-52. MR0991055
13. Clevenson, M. L. and Zidek, J. V. (1975). Simultaneous estimation of the means of independent Poisson laws. J. Amer. Statist. Assoc. 70 698-705. MR0394962
14. Craven, P. and Wahba, G. (1978/79). Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. 31 377-403. MR0516581
15. Dey, D. K. and Srinivasan, C. (1985). Estimation of a covariance matrix under Stein's loss. Ann. Statist. 13 1581-1591. MR0811511
16. Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200-1224. MR1379464
17. Du Plessis, N. (1970). An Introduction to Potential Theory. University Mathematical Monographs 7. Hafner, Darien, CT. MR0435422
18. Efron, B. (2004). The estimation of prediction error: Covariance penalties and cross-validation. J. Amer. Statist. Assoc. 99 619-642. MR0845884
19. Efron, B., Hastie, T., Johnstone, I. and TIBSHIRANI, R. (2004). Least angle regression. Ann. Statist. 32 407- 499. MR2060166
20. Fang, K. T., Kotz, S. and Ng, K. W. (1990). Symmetric Multivariate and Related Distributions. Monographs on Statistics and Applied Probability 36. Chapman & Hall, London. MR1071174
21. Fourdrinier, D. and Lepelletier, P. (2008). Estimating a general function of a quadratic function. Ann. Inst. Statist. Math. 60 85-119. MR2400062
22. Rinier, D. and Strawderman, W. E. (2003). On Bayes and unbiased estimators of loss. Ann. Inst. Statist. Math. 55 803-816. MR2028618
23. Fourdrinier, D., Strawderman, W. E. and WELLS, M. T. (2003). Robust shrinkage estimation for elliptically symmetric distributions with unknown covariance matrix. J. Multivariate Anal. 85 24-39. MR1978175
24. Fourdrinier, D. and Wells, M. T. (1994). Comparaisons de procédures de sélection d'un modèle de régression: Une approche décisionnelle. C. R. Acad. Sci. Paris Sér. I Math. 319 865-870. MR1300958
25. Fourdrinier, D. and Wells, M. T. (1995). Estimation of a loss function for spherically symmetric distributions in the general linear model. Ann. Statist. 23 571-592. MR1332582
26. Fourdrinier, D. and Wells, M. T. (1995). Loss estimation for spherically symmetric distributions. J. Multivariate Anal. 53 311-331. MR1336060
27. Haff, L. R. (1979). An identity for the Wishart distribution with applications. J. Multivariate Anal. 9 531-544. MR0556910
28. Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Monographs on Statistics and Applied Probability 43. Chapman & Hall, London. MR1082147
29. Hoerl, A. E. and Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12 55-67.
30. Hsieh, F. S. and Hwang, J. T. G. (1993). Admissibility under the frequentist's validity constraint in estimating the loss of the least-squares estimator. J. Multivariate Anal. 44 279-285. MR1219208
31. Hudson, H. M. (1978). A natural identity for exponential families with applications in multiparameter estimation. Ann. Statist. 6 473-484. MR0467991
32. Johnstone, I. (1988). On inadmissibility of some unbiased estimates of loss. In Statistical Decision Theory and Related Topics, IV, Vol. 1 (West Lafayette, Ind., 1986) (S. S. Gupta and J. O. Berger, eds.) 361-379. Springer, New York. MR0927112
33. Kiefer, J. (1975). Conditional confidence approach in multi-decision problems. In Multivariate Analysis 4 (P. R. Krishnaiah, ed.). Academic Press, New York.
34. Kiefer, J. (1976). Admissibility of conditional confidence procedures. Ann. Statist. 4 836-865. MR0438543
35. Kiefer, J. (1977). Conditional confidence statements and confidence estimators. J. Amer. Statist. Assoc. 72 789-827. MR0518611
36. Kubokawa, T. and Srivastava, M. S. (1999). Robust improvement in estimation of a covariance matrix in an elliptically contoured distribution. Ann. Statist. 27 600-609. MR1714715
37. Lehmann, E. L. and Sheffé, H. (1950). Completeness, similar regions and unbiased estimates. Sankhyā 17 305-340.
38. Lele, C. (1992). Inadmissibility of loss estimators. Statist. Decisions 10 309-322. MR1200692
39. Lele, C. (1993). Admissibility results in loss estimation. Ann. Statist. 21 378-390. MR1212182
40. Lu, K. L. and Berger, J. O. (1989). Estimation of normal means: Frequentist estimation of loss. Ann. Statist. 17 890-906. MR0994274
41. Mallows, C. (1973). Some comments on Cp. Technometrics 15 661-675.
42. Rukhin, A. L. (1988). Estimated loss and admissible loss estimators. In Statistical Decision Theory and Related Topics, IV, Vol. 1 (West Lafayette, Ind., 1986) (S. S. Gupta and J. O. Berger, eds.) 409-418. Springer, New York. MR0927116
43. Sandved, E. (1968). Ancillary statistics and prediction of the loss in estimation problems. Ann. Math. Statist. 39 1756-1758. MR0231481
44. Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461-464. MR0468014
45. Stein, C. Estimating the covariance matrix. Unpublished manuscript.
46. Stein, C. (1964). Inadmissibility of the usual estimator for the variance of a normal distribution with unknown mean. Ann. Inst. Statist. Math. 16 155-160. MR0171344
47. Steǐn, Č. (1977). Lectures on multivariate estimation theory. Zap. NauČn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 74 4-65, 146, 148. MR0501548
48. Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151. MR0630098
49. Takemura, A. (1984). An orthogonally invariant minimax estimator of the covariance matrix of a multivariate normal population. Tsukuba J. Math. 8 367-376. MR0767967
50. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. MR1379242
51. Wan, A. T. K. and ZOU, G. (2004). On unbiased and improved loss estimation for the mean of a multivariate normal distribution with unknown variance. J. Statist. Plann. Inference 119 17-22. MR2018447
52. Ye, J. (1998). On measuring and correcting the effects of data mining and model selection. J. Amer. Statist. Assoc. 93 120-131. MR1614596
53. Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 301-320. MR2137327
54. Zou, H., Hastie, T. and Tibshirani, R. (2007). On the "degrees of freedom" of the lasso. Ann. Statist. 35 2173-2192. MR2363967