Variable selection in high-dimension with random designs and orthogonal matching pursuit

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 1771-1800

  Joseph, Antony ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Compressed sensing; Greedy algorithms; High dimensional regression; Lasso

Indexed keywords

COEFFICIENT VECTOR; GREEDY ALGORITHMS; HIGH DIMENSIONS; HIGH-DIMENSIONAL REGRESSIONS; LASSO; ORTHOGONAL MATCHING PURSUIT; SPARSITY CONSTRAINTS; VARIABLE SELECTION;

ARTIFICIAL INTELLIGENCE; COMPRESSED SENSING; SOFTWARE ENGINEERING;

ALGORITHMS;

EID: 84884228973     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (30)

1. R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin. The Johnson-Lindenstrauss lemma meets compressed sensing. Constructive Approximation, 2007.
2. A. R. Barron and A. Joseph. Sparse superposition codes: Fast and reliable at rates approaching capacity with gaussian noise. Technical report, Yale University, 2010.
3. A. R. Barron and A. Joseph. Least squares superposition codes of moderate dictionary size, reliable at rates up to channel capacity. IEEE Trans. Inform. Theory, 58:2541-2557, 2012.
4. A. R. Barron, A. Cohen, W. Dahmen, and R. A. DeVore. Approximation and learning by greedy algorithms. Ann. Statist., 36(1):64-94, 2008.
5. T. T. Cai and L. Wang. Orthogonal matching pursuit for sparse signal recovery with noise. IEEE Trans. Inform. Theory, 57(7):4680-4688, 2011.
6. E. Candès and T. Tao. The Dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist., 35(6):2313-2351, 2007.
7. E. J. Candès and Y. Plan. Near-ideal model selection by l1 minimization. Ann. Statist., 37 (5 A):2145-2177, 2009.
8. E. J. Candès and T. Tao. Decoding by linear programming. IEEE Trans. Inform. Theory, 51(12):4203-4215, 2005.
9. E. J. Candès and T. Tao. Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inform. Theory, 52(12):5406-5425, 2006.
10. D. L. Donoho. Compressed sensing. IEEE Trans. Inform. Theory, 52(4):1289-1306, 2006a.
11. D. L. Donoho. For most large underdetermined systems of equations, the minimal l1-norm nearsolution approximates the sparsest near-solution. Communications On Pure And Applied Mathematics, 59(7):907-934, 2006b.
12. D. L. Donoho, M. Elad, and V. N. Temlyakov. Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory, 52(1):6-18, 2006.
13. W. Feller. An Introduction To Probability Theory And Its Applications. Vol. i. 1950.
14. A. K. Fletcher and S. Rangan. Orthogonal matching pursuit: A Brownian motion analysis. IEEE Trans. Sig. Processing, (99):1-1, 2011.
15. C. Huang, G. H. L. Cheang, and A. R. Barron. Risk of penalized least squares, greedy selection and l1 penalization for flexible function libraries. Technical Report, 2008.
16. L. Jones. A simple lemma for optimization in a Hilbert space, with application to projection pursuit and neural net training. Ann. Statist., 20:608-613, 1992.
17. W. S. Lee, P. L. Bartlett, and R. C. Williamson. Efficient agnostic learning of neural networks with bounded fan-in. IEEE Trans. Inform. Theory, 42(6):2118-2132, 1996.
18. S. Mallat and S. M. Z. Zhang. Matching pursuit with time-frequency dictionaries. IEEE Trans. Signal Processing, 41:3397-3415, 1993.
19. N. Meinshausen and P. Buhlmann. High-dimensional graphs and variable selection with the lasso. Ann. Statist., 34(3):1436-1462, 2006.
20. Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad. Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In Conf. Rec. 27th Asilomar Conf. Sig., Sys. and Comput., pages 40-44. IEEE, 1993.
21. S. J. Szarek. Condition numbers of random matrices. Journal of Complexity, 7(2):131-149, 1991.
22. R. Tibshirani. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol., pages 267-288, 1996.
23. J. A. Tropp. Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inform. Theory, 50(10):2231-2242, 2004.
24. J. A. Tropp. Just relax: Convex programming methods for identifying sparse signals in noise. IEEE Trans. Inform. Theory, 52(3):1030-1051, 2006.
25. J. A. Tropp and A. C. Gilbert. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inform. Theory, 53(12):4655-4666, 2007.
26. 1constrained quadratic programming (lasso). IEEE Trans. Inform. Theory, 55(5):2183-2202, 2009.
27. C. H. Zhang and J. Huang. The sparsity and bias of the lasso selection in high-dimensional linear regression. Ann. Statist., 36(4):1567-1594, 2008.
28. T. Zhang. On the consistency of feature selection using greedy least squares regression. J. Mach. Learn. Res., 10:555-568, 2009a.
29. T. Zhang. Some sharp performance bounds for least squares regression with l1 regularization. Ann. Statist., 7(5A):2109-2144, 2009b.
30. P. Zhao and B. Yu. On model selection consistency of lasso. J. Mach. Learn. Res., 7:2541-2563, 2006.