Local shrinkage rules, Lévy processes and regularized regression

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 74, Issue 2, 2012, Pages 287-311

  Polson, Nicholas G ^a   Scott, James G ^b  

^a UNIVERSITY OF CHICAGO   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

L vy processes; Normal scale mixtures; Partial least squares; Principal components regression; Shrinkage; Sparsity

Indexed keywords

EID: 84858279402     PISSN: 13697412     EISSN: 14679868     Source Type: Journal    
DOI: 10.1111/j.1467-9868.2011.01015.x     Document Type: Article

References (49)

1. Aït-Sahalia, Y. and Jacod, J. (2009) Estimating the degree of activity of jumps in high frequency data. Ann. Statist., 37, 2202-2244.
2. Armagan, A., Dunson, D. and Lee, J. (2010) Bayesian generalized double Pareto shrinkage. Technical Report. Department of Statistical Science, Duke University, Durham.
3. Bae, K. and Mallick, B. (2004) Gene selection using a two-level hierarchical Bayesian model. Bioinformatics, 20, 3423-3430.
4. Barndorff-Nielsen, O. (1997) Normal inverse Gaussian distributions and stochastic volatility modeling. Scand. J. Statist., 24, 1-13.
5. Barndorff-Nielsen, O., Kent, J. and Sorensen, M. (1982) Normal variance-mean mixtures and z distributions. Int. Statist. Rev., 50, 145-159.
6. Barndorff-Nielsen, O. E. and Shephard, N. (2001) Normal modified stable processes. Technical Report 2001-W6. Nuffield College, University of Oxford, Oxford.
7. Berger, J. O. (1985) Statistical Decision Theory and Bayesian Analysis, 2nd edn. New York: Springer.
8. Berger, J. O. and Delampady, M. (1987) Testing precise hypotheses. Statist. Sci., 2, 317-352.
9. Bondesson, L. (1990) Generalized gamma convolutions and complete monotonicity. Probab. Theor. Reltd Flds, 85, 181-194.
10. Candes, E. and Tao, T. (2007) The Dantzig selector: statistical estimation when p is much larger than n. Ann. Statist., 35, 2313-2351.
11. Caron, F. and Doucet, A. (2008) Sparse Bayesian nonparametric regression. In Proc. 25th Int. Conf. Machine Learning, pp. 88-95. New York: Association for Computing Machinery Press.
12. Carvalho, C. M., Polson, N. G. and Scott, J. G. (2010) The horseshoe estimator for sparse signals. Biometrika, 97, 465-480.
13. Chun, H. and Keles, S. (2010) Sparse partial least squares regression for simultaneous dimension reduction and variable selection. J. R. Statist. Soc. B, 72, 3-25.
14. Clyde, M., Desimone, H. and Parmigiani, G. (1996) Prediction via orthogonalized model mixing. J. Am. Statist. Ass., 91, 1197-1208.
15. Clyde, M. and Wolpert, R. (2011) Discussion on 'Shrink globally, act locally: sparse Bayesian regularization and prediction' (by N. G. Polson and J. G. Scott). In Bayesian Statistics 9 (eds J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West). Oxford: Oxford University Press.
16. Cont, R. and Tankov, P. (2004) Financial Modelling with Jump Processes. Boca Raton: Chapman and Hall-CRC.
17. Denison, D. and George, E. (2000) Bayesian prediction using adaptive ridge estimators. Technical Report. Imperial College, London.
18. Efron, B. (2009) Empirical Bayes estimates for large-scale prediction problems. J. Am. Statist. Ass., 104, 1015-1028.
19. Fan, J. and Li, R. (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Ass., 96, 1348-1360.
20. Fearn, T. (1983) A misuse of ridge regression in the calibration of a near infrared reflectance instrument. Appl. Statist., 32, 73-79.
21. Figueiredo, M. (2003) Adaptive sparseness for supervised learning. IEEE Trans. Pattn Anal. Mach. Intell., 25, 1150-1159.
22. Fisher, R. A. (1935) The mathematical distributions used in the common tests of significance. Econometrica, 3, 353-365.
23. Frank, I. and Friedman, J. H. (1993) A statistical view of some chemometrics regression tools (with discussion). Technometrics, 35, 109-135.
24. George, E. I. and Foster, D. P. (2000) Calibration and empirical Bayes variable selection. Biometrika, 87, 731-747.
25. Griffin, J. and Brown, P. (2005) Alternative prior distributions for variable selection with very many more variables than observations. Technical Report. University of Warwick, Coventry.
26. Griffin, J. and Brown, P. (2010) Inference with normal-gamma prior distributions in regression problems. Baysn Anal., 5, 171-188.
27. Grigelionis, B. (1999) Processes of Meixner type. Lith. Math. J., 39, 33-41.
28. Grigelionis, B. (2001) Generalized z-distributions and related stochastic processes. Lith. Math. J., 41, 239-251.
29. Hans, C. M. (2009) Bayesian lasso regression. Biometrika, 96, 835-845.
30. Kingman, J. F. C. (1975) Random discrete distributions (with discussion). J. R. Statist. Soc. B, 37, 1-22.
31. Liang, F., Paulo, R., Molina, G., Clyde, M. and Berger, J. (2008) Mixtures of g-priors for Bayesian variable selection. J. Am. Statist. Ass., 103, 410-423.
32. Madan, D. and Yor, M. (2006) CGMY and Meixner subordinators are absolutely continuous with respect to one sided stable subordinators. Technical Report arXiv:math/0601173. University of Maryland, College Park.
33. Maruyama, Y. and George, E. I. (2010) gbf: A fully Bayes factor with a generalized g-prior. Technical Report arXiv:0801.4410v2. University of Tokyo, Tokyo.
34. Masreliez, C. (1975) Approximate non-Gaussian filtering with linear state and observation relations. IEEE Trans. Autom. Control, 20, 107-110.
35. Mitchell, A. F. S. (1994) A note on posterior moments for a normal mean with double-exponential prior. J. R. Statist. Soc. B, 56, 605-610; correction, 57 (1995) 471.
36. Park, T. and Casella, G. (2008) The Bayesian lasso. J. Am. Statist. Ass., 103, 681-686.
37. Pericchi, L. R. and Smith, A. F. M. (1992) Exact and approximate posterior moments for a normal location parameter. J. R. Statist. Soc. B, 54, 793-804.
38. Polson, N. G. (1991) A representation of the posterior mean for a location model. Biometrika, 78, 426-430.
39. Polson, N. G. and Scott, J. G. (2010) Large-scale simultaneous testing with hypergeometric inverted-beta priors. Technical Report. University of Texas at Austin, Austin. (Available from).
40. Polson, N. G. and Scott, J. G. (2011) Shrink globally, act locally: sparse Bayesian regularization and prediction. In Bayesian Statistics 9 (eds J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West). Oxford: Oxford University Press.
41. R Development Core Team (2009) R: a Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.
42. Scott, J. G. and Berger, J. O. (2010) Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Statist., 38, 2587-2619.
43. Strawderman, W. (1971) Proper Bayes minimax estimators of the multivariate normal mean. Ann. Statist., 42, 385-388.
44. West, M. (1987) On scale mixtures of normal distributions. Biometrika, 74, 646-648.
45. West, M. (2003) Bayesian factor regression models in the ''large p, small n'' paradigm. In Bayesian Statistics 7 (eds J. M. Bernardo, M. Bayarri, J. Berger, A. Dawid, D. Heckerman, A. Smith and M. West), pp. 723-732. Oxford: Oxford University Press.
46. Wolpert, R. L., Clyde, M. A. and Tu, C. (2010) Stochastic expansions using continuous dictionaries: Lévy Adaptive Regression Kernels. Technical Report 2006-08. Department of Statistical Science, Duke University, Durham.
47. Wolpert, R. L. and Taqqu, M. (2005) Fractional Ornstein-Uhlenbeck Lévy processes and the Telecom process: upstairs and downstairs. Signal Process., 85, 1523-1545.
48. Zellner, A. (1986) On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti (eds P. K. Goel and A. Zellner), pp. 233-243. Amsterdam: Elsevier.
49. Zou, H. and Li, R. (2008) One-step sparse estimates in nonconcave penalized likelihood models. Ann. Statist., 36, 1509-1533.