Bayesian Compressed Regression

Journal of the American Statistical Association

Volumn 110, Issue 512, 2015, Pages 1500-1514

  Guhaniyogi, Rajarshi ^a   Dunson, David B ^b  

^a University of California Santa Cruz   (United States)

^b Duke University   (United States)

Author keywords

Compressed sensing; Data compression; Dimensionality reduction; Large p, small n; Random projection; Sparsity; Sufficient dimension reduction

Indexed keywords

EID: 84954443066     PISSN: 01621459     EISSN: 1537274X     Source Type: Journal    
DOI: 10.1080/01621459.2014.969425     Document Type: Article

References (49)

1. D.Achlioptas, (2003), Database-Friendly Random Projections: Johnson-Lindenstrauss With Binary Coins, Journal of Computer and System Sciences, 66, 671–687.
2. E.Amaldi,, and V.Kann, (1998), On The Approximation of Minimizing Non-Zero Variables or Unsatisfied Relations in Linear Systems, Theoretical Computer Science, 209, 237–260.
3. A.Armagan,, D.B.Dunson,, and J.Lee, (2013), Generalized Double Pareto Shrinkage, Statistica Sinica, 23, 119–143.
4. A.Bhattacharya,, D.Pati,, N.Pillai,, and D.B.Dunson, (2012), Bayesian Shrinkage, available at Arxiv Preprint arxiv:1212.6088.
5. E.A.Candes,, J.Romberg,, and T.Tao, (2006), Stable Signal Recovery From Incomplete and Inaccurate Measurements, Communications in Pure and Applied Mathematics, 59, 1207–1223.
6. E.A.Candes,, and T.Tao, (2005), Decoding by Linear Programming, IEEE Transactions Information Theory, 51, 4203–4215.
7. ——— (2007), The Dantzig Selector: Statistical Estimator When p is Much Larger than n, The Annals of Statistics, 35, 2313–2351.
8. C.M.Carvalho,, N.G.Polson,, and J.G.Scott, (2009), Handling Sparsity via The Horseshoe, Journal of Machine Learning Research, 5, 73–80.
9. ——— (2010), The Horseshoe Estimator for Sparse Signals, Biometrika, 97, 465–480.
10. R.D.Cook, (1998), Regression Graphics:Ideas for Studying Regression Through Graphics, New York: Wiley.
11. S.Dasgupta, (2013), Experiments With Random Projection, available at Arxiv Preprint arxiv:1301.3849.
12. S.Dasgupta,, and Y.Freund, (2008), “Random Projection Trees and Low Dimensional Manifolds,” in Proceedings of the 40th Annual ACM Symposium on Theory of Computing, pp. 537–546.
13. S.Dasgupta,, and A.Gupta, (2003), An Elementary Proof of The Theorem of Johnson and Lindenstrauss, Random Structures and Algorithms, 22, 60–65.
14. M.Davenport,, P.T.Boufounos,, M.Wakin,, and R.Baraniuk, (2010), Signal Processing With Compressive Measurements, Selected Topics in Signal Processing, IEEE Journal of, 4, 445–460.
15. M.Davenport,, M.Durate,, M.Wakin,, J.Laska,, D.Takhar,, K.Kelly,, and R.Baraniuk, (2007), “The Smashed Filter for Compressive Classification and Target Recognition,” in Proceedings of Computational Imaging V.
16. D.Donoho, (2006), Compressed Sensing, IEEE Transactions on Information Theory, 52, 1289–1306.
17. W.DuMouchel, (1999), Data Squashing: Constructing Summary Data Sets, available at http://www.cs.princeton.edu/courses/archive/spr04/cos598B/bib/DuMouchel.pdf.
18. D.B.Dunson,, M.Watson,, and J.A.Taylor, (2003), Bayesian Latent Variable Models for Median Regression on Multiple Regression, Biometrics, 59, 296–304.
19. C.Faes,, J.T.Ormerod,, and M.P.Wand, (2011), Variational Bayesian Inference for Parametric and Nonparametric Regression With Missing Data, Journal of the American Statistical Association, 106, 959–971.
20. M.M.Fard,, Y.Grinberg,, J.Pineau,, and D.Precup, (2012), “Compressed Least-Squares Regression on Sparse Spaces,” in Proceedings of the 26th AAAI Conference on Artificial Intelligence, pp. 1054–1060.
21. D.Fradkin,, and D.Madigan, (2003), “Experiments With Random Projections for Machine Learning,” in Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 517–522.
22. S.Ghosal,, J.K.Ghosh,, and A.W.Van Der Vaart, (2000), Convergence Rates of Posterior Distributions, The Annals of Statistics, 28, 500–531.
23. S.Ghosal,, and A.W.Van Der Vaart, (2001), Entropies and Rates of Convergence for Bayes and Maximum Likelihood Estimation for Mixture of Normal Densities, The Annals of Statistics, 29, 1233–1263.
24. ——— (2007), Convergence Rates of Posterior Distributions for Non iid Observations, The Annals of Statistics, 35, 192–223.
25. M.Girolami,, and S.Rogers, (2006), Variational Bayesian Multinomial Probit Regression With Gaussian Process Priors, Neural Computation, 18, 1790–1817.
26. R.B.Gramacy, (2010), Estimation for Multivariate Normal and Student-t Data With Monotone Missingness Monomvn Package Manual.” Available at http://cran.r-project.org/web/packages/monomvn/monomvn.pdf.
27. C.Hans, (2009), Bayesian Lasso Regression, Biometrika, 96, 835–845.
28. T.Hastie,, and B.Efron, (2012), Lars Package Manual.” Available at http://cran.r-project.org/web/packages/lars/lars.pdf.
29. W.Jiang, (2007), Bayesian Variable Selection for High Dimensional Generalized Linear Models: Convergence Rates of The Fitted Densities, The Annals of Statistics, 35, 1487–1511.
30. C.M.Joshi,, and S.K.Bissu, (1991), Some Inequalities of Bessel and Modified Bessel Functions, Journal of Australian Math Society, Series A, 50, 333–342.
31. S.Krishnan,, C.Bhattacharyya,, and R.Hariharan, (2007), A Randomized Algorithm for Large Scale Support Vector Learning, Advances in Neural Information Processing Systems (NIPS), 20, eds. J. C. Platt, D. Koller, Y. Singer, and S. Roweis, Cambridge, MA: MIT Press.
32. H.K.H.Lee,, M.Taddy,, and G.A.Gray, (2008), Selection of a Representative Sample, Journal of Classification, 27, 41–53.
33. P.Li,, A.Shrivastava,, J.Moore,, and A.C.Konig, (2011), Hashing Algorithms for Large-Scale Learning, Advances in Neural Information Processing Systems (NIPS), 24, 2672–2680.
34. D.Madigan,, N.Raghavan,, and W.Dumouchel, (2002), Likelihood-Based Data Squashing: A Modeling Approach to Instance Construction, Data Mining and Knowledge Discovery, 6, 173–190.
35. O.A.Maillard,, and R.Munos, (2009), Compressed-Least Squares Regression, CiteSeerX 10.1.1.153.8922.
36. P.L.Olive,, J.P.Banath,, and R.E.Durand, (1990), Heterogeneity in Radiation-Induced DNA Damage and Repair in Tumour and Normal Cells Measured Using The Comet Assay, Radiation Research, 112, 86–94.
37. J.T.Ormerod,, and M.P.Wand, (2012), Gaussian Variational Approximate Inference for Generalized Linear Mixed Models, Journal of Computational and Graphical Statistics, 21, 2–17.
38. A.Owen, (2003), Data Squashing by Empirical Likelihood, Data Mining and Knowledge Discovery, 7, 101–113.
39. T.Park,, and G.Casella, (2008), The Bayesian Lasso, Journal of the American Statistical Association, 103, 681–686.
40. A.E.Raftery,, D.Madigan,, and J.A.Hoeting, (1997), Bayesian Model Averaging for Linear Regression Models, Journal of the American Statistical Association, 92, 179–191.
41. B.Ripley, (2012), MASS Package Manual. Available at http://cran.r-project.org/web/packages/MASS/MASS.pdf.
42. Q.Shi,, J.Petterson,, G.Dror,, J.Langford,, A.Smola,, and S.V.N.Viswanathan, (2009), Hash Kernels for Structured Data, Journal of Machine Learning Research, 11, 2615–2637.
43. N.Strawn,, A.Armagan,, R.Saab,, L.Carin,, and D.B.Dunson, (2012), Finite Sample Posterior Concentration in High Dimensional Regression, Arxiv Preprint arxiv:1207.4854.
44. R.Tibshirani, (1996), Regression Selection and Shrinkage via The Lasso. Journal of the Royal Statistical Society, Series B, 58, 267–288.
45. M.K.Titsias,, and N.D.Lawrence, (2010), “Bayesian Gaussian Process Latent Variable Model,” in 13th International Conference on Artificial Intelligence and Statistics (AISTAT), pp. 844–851.
46. S.T.Tokdar,, Y.M.Zhu,, and J.K.Ghosh, (2010), Bayesian Density Regression With Logistic Gaussian Process and Subspace Projection, Bayesian Analysis, 5, 319–344.
47. H.Zhou,, L.Li,, and H.Zhu, (2012), Tensor Regression With Application in Neuroimaging Data Analysis. Available at Arxiv Preprint arxiv:1203.3209.
48. S.Zhou,, J.Lafferty,, and L.Wasserman, (2009), Compressed and Privacy-Sensitive Sparse Regression, IEEE Transactions on Information Theory, 55, 846–866.
49. H.Zou,, and T.Hastie, (2005), Regularization and Variable Selection via the Elastic Net, Journal of the Royal Statistical Society, Series B, 67, 301–320.