Variational approximation for heteroscedastic linear models and matching pursuit algorithms

Statistics and Computing

Volumn 22, Issue 2, 2012, Pages 497-512

  Nott, David J ^a   Tran, Minh Ngoc ^a   Leng, Chenlei ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Bayesian model selection; Heteroscedasticity; Matching pursuit; Variational approximation

Indexed keywords

EID: 81955160896     PISSN: 09603174     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11222-011-9243-2     Document Type: Article

References (36)

1. Aitchison, J.: The Statistical Analysis of Compositional Data. Chapman and Hall, London (1986).
2. Beal, M. J., Ghahramani, Z.: The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures. Bayesian Stat. 7, 453-464 (2003).
3. Beal, M. J., Ghahramani, Z.: Variational Bayesian learning of directed graphical models with hidden variables. Bayesian Anal. 1, 1-44 (2006).
4. Bishop, C. M.: Pattern Recognition and Machine Learning. Springer, New York (2006).
5. Brown, P. J., Fearn, T., Vannucci, M.: Bayesian wavelet regression on curves with application to a spectroscopic calibration problem. J. Am. Stat. Assoc. 96, 398-408 (2001).
6. Carroll, R. J., Ruppert, D.: Transformation and Weighting in Regression. Monographs on Statistics and Applied Probability. Chapman and Hall, London (1988).
7. Chan, D., Kohn, R., Nott, D. J., Kirby, C.: Adaptive nonparametric estimation of mean and variance functions. J. Comput. Graph. Stat. 15, 915-936 (2006).
8. Chen, J., Chen, Z.: Extended Bayesian information criteria for model selection with large model spaces. Biometrika 95, 759-771 (2008).
9. Chib, S., Jeliazkov, I.: Marginal likelihood from the Metropolis-Hastings output. J. Am. Stat. Assoc. 96, 270-281 (2001).
10. Cottet, R., Kohn, R., Nott, D. J.: Variable selection and model averaging in overdispersed generalized linear models. J. Am. Stat. Assoc. 103, 661-671 (2008).
11. Davidian, M., Carroll, R.: Variance function estimation. J. Am. Stat. Assoc. 82, 1079-1091 (1987).
12. Efron, B.: Double exponential families and their use in generalised linear regression. J. Am. Stat. Assoc. 81, 709-721 (1986).
13. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression (with discussion). Ann. Stat. 32, 407-451 (2004).
14. Friedman, J. H.: Fast sparse regression and classification (2008). URL http://www-stat. stanford. edu/jhf/ftp/GPSpaper. pdf.
15. George, E. I., McCulloch, R. E.: Variable selection via Gibbs sampling. J. Am. Stat. Assoc. 88, 881-889 (1993).
16. Geweke, J., Keane, M.: Smoothly mixing regressions. J. Econom. 138, 252-291 (2007).
17. Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., Saul, L. K.: An introduction to variational methods for graphical models. In: Jordan, M. I. (ed.) Learning in Graphical Models. MIT Press, Cambridge (1999).
18. Mallat, S. G., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41, 3397-3415 (1993).
19. Nelder, J., Pregibon, D.: An extended quasi-likelihood function. Biometrika 74, 221-232 (1987).
20. O'Hagan, A., Forster, J. J.: Bayesian Inference, 2nd edn. Kendall's Advanced Theory of Statistics, vol. 2B. Arnold, London (2004).
21. Ormerod, J. T., Wand, M. P.: Explaining variational approximation. Am. Stat. 64(2), 140-153 (2010).
22. Osborne, B. G., Fearn, T., Miller, A. R., Douglas, S.: Application of near infrared reflectance spectroscopy to the compositional analysis of biscuits and biscuit doughs. J. Sci. Food Agric. 35, 99-105 (1984).
23. Raftery, A. E., Madigan, D., Hoeting, J.: Bayesian model averaging for linear regression models. J. Am. Stat. Assoc. 92, 179-191 (1997).
24. Rigby, R. A., Stasinopoulos, D. M.: Generalized additive models for location, scale and shape (with discussion). Appl. Stat. 54, 507-554 (2005).
25. Ruppert, D., Wand, M. P., Carroll, R. J.: Semiparametric Regression. Cambridge University Press, Cambridge (2003).
26. Smith, M., Kohn, R.: Nonparametric regression using Bayesian variable selection. J. Econom. 75, 317-343 (1996).
27. Smyth, G.: Generalized linear models with varying dispersion. J. R. Stat. Soc. B 51, 47-60 (1989).
28. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58, 267-288 (1996).
29. Tropp, J. A.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50, 2231-2242 (2004).
30. Villani, M., Kohn, R., Giordani, P.: Regression density estimation using smooth adaptive Gaussian mixtures. J. Econom. 153, 155-173 (2009).
31. Weisberg, S.: Applied Linear Regression, 3rd edn. Wiley, Hoboken (2005).
32. Yau, P., Kohn, R.: Estimation and variable selection in nonparametric heteroscedastic regression. Stat. Comput. 13, 191-208 (2003).
33. Yee, T., Wild, C.: Vector generalized additive models. J. R. Stat. Soc. B 58, 481-493 (1996).
34. Zhang, T.: On the consistency of feature selection using greedy least squares regression. J. Mach. Learn. Res. 10, 555-568 (2009).
35. Zhao, P., Yu, B.: Stagewise Lasso. J. Mach. Learn. Res. 8, 2701-2726 (2007).
36. Zou, H.: The adaptive Lasso and its oracle properties. J. Am. Stat. Assoc. 106, 1418-1429 (2006).