Accelerated sparse linear regression via random projection

30th AAAI Conference on Artificial Intelligence, AAAI 2016

Volumn , Issue , 2016, Pages 2337-2343

  Zhang, Weizhong ^a   Zhang, Lijun ^b   Jin, Rong ^c   Cai, Deng ^a   He, Xiaofei ^a  

^a ZHEJIANG UNIVERSITY   (China)

^b NANJING UNIVERSITY   (China)

^c ALIBABA GROUP   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; GRADIENT METHODS; LINEAR REGRESSION; REGRESSION ANALYSIS;

APPROXIMATORS; CONVERGENCE RATES; GRADIENT-BASED METHOD; HIGH-DIMENSIONAL DATASET; NUMBER OF DATUM; RANDOM PROJECTIONS; SPARSE LINEAR REGRESSIONS; TIME COMPLEXITY;

NUMERICAL METHODS;

EID: 85007168398     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Becker, S.; Bobin, J.; and Candes, E. J. 2011. Nesta: A fast and accurate first-order method for sparse recovery. SIAM Journal on Imaging Sciences 4(1):1-39.
2. Bickel, P. J.; Ritov, Y.; and Tsybakov, A. B. 2009. Simultaneous analysis of lasso and dantzig selector. The Annals of Statistics 1705-1732.
3. Bioucas-Dias, J. M., and Figueiredo, M. A. 2007. A new twist: Two-step iterative shrinkage/thresholding algorithms for image restoration. Image Processing, IEEE Transactions on 16(12):2992-3004.
4. Candes, E. J., and Tao, T. 2006. Near-optimal signal recovery from random projections: Universal encoding strategies? Information Theory, IEEE Transactions on 52(12):5406- 5425.
5. Daubechies, I.; Defrise, M.; and De Mol, C. 2003. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. arXiv preprint math/0307152.
6. Drineas, P.; Frieze, A.; Kannan, R.; Vempala, S.; and Vinay, V. 2004. Clustering large graphs via the singular value decomposition. Machine learning 56(1-3):9-33.
7. Drineas, P.; Mahoney, M. W.; Muthukrishnan, S.; and Sarĺos, T. 2011. Faster least squares approximation. Numerische Mathematik 117(2):219-249.
8. Drineas, P.; Kannan, R.; and Mahoney, M. W. 2006. Fast monte carlo algorithms for matrices ii: Computing a lowrank approximation to a matrix. SIAM Journal on Computing 36(1):158-183.
9. Drineas, P.; Mahoney, M. W.; and Muthukrishnan, S. 2008. Relative-error cur matrix decompositions. SIAM Journal on Matrix Analysis and Applications 30(2):844-881.
10. Figueiredo, M. A.; Nowak, R. D.; and Wright, S. J. 2007. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. Selected Topics in Signal Processing, IEEE Journal of 1(4):586-597.
11. Frieze, A.; Kannan, R.; and Vempala, S. 2004. Fast montecarlo algorithms for finding low-rank approximations. Journal of the ACM (JACM) 51(6):1025-1041.
12. Hale, E. T.; Yin, W.; and Zhang, Y. 2008. Fixed-point continuation for l1-minimization methodology and convergence. SIAM Journal on Optimization 19(3):1107-1130.
13. Halko, N.; Martinsson, P.-G.; and Tropp, J. A. 2011. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review 53(2):217-288.
14. Huang, J.; Ma, S.; and Zhang, C.-H. 2008. Adaptive lasso for sparse high-dimensional regression models. Statistica Sinica 18(4):1603.
15. Kim, S.-J.; Koh, K.; Lustig, M.; Boyd, S.; and Gorinevsky, D. 2007. An interior-point method for large-scale l 1- regularized least squares. Selected Topics in Signal Processing, IEEE Journal of 1(4):606-617.
16. Koltchinskii, V. 2011. Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems: Ecole dEte de Probabilites de Saint-Flour XXXVIII-2008, volume 2033. Springer.
17. Lu, Y.; Dhillon, P.; Foster, D. P.; and Ungar, L. 2013. Faster ridge regression via the subsampled randomized hadamard transform. In Advances in Neural Information Processing Systems, 369-377.
18. Mahoney, M. W., and Drineas, P. 2009. Cur matrix decompositions for improved data analysis. Proceedings of the National Academy of Sciences 106(3):697-702.
19. Mahoney, M.W. 2011. Randomized algorithms for matrices and data. Foundations and Trends R ® in Machine Learning 3(2):123-224.
20. Nesterov, Y. 2004. Introductory lectures on convex optimization: A basic course, volume 87. Springer.
21. Nesterov, Y. 2013. Gradient methods for minimizing composite functions. Mathematical Programming 140(1):125- 161.
22. Sarlos, T. 2006. Improved approximation algorithms for large matrices via random projections. In Foundations of Computer Science, 2006. FOCS'06. 47th Annual IEEE Symposium on, 143-152. IEEE.
23. Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological) 267-288.
24. Wen, Z.; Yin, W.; Goldfarb, D.; and Zhang, Y. 2010. A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation. SIAM Journal on Scientific Computing 32(4):1832-1857.
25. Wright, S. J.; Nowak, R. D.; and Figueiredo, M. A. 2009. Sparse reconstruction by separable approximation. Signal Processing, IEEE Transactions on 57(7):2479-2493.
26. Xiao, L., and Zhang, T. 2013. A proximal-gradient homotopy method for the sparse least-squares problem. SIAM Journal on Optimization 23(2):1062-1091.
27. Zhang, L.; Mahdavi, M.; Jin, R.; Yang, T.; and Zhu, S. 2013. Recovering the optimal solution by dual random projection. In Proceedings of the 26th Annual Conference on Learning Theory, 135-157.
28. Zhang, L.; Mahdavi, M.; Jin, R.; Yang, T.; and Zhu, S. 2014. Random projections for classification: A recovery approach. Information Theory, IEEE Transactions on 60(11):7300- 7316.
29. Zhang, L.; Yang, T.; Jin, R.; and Zhou, Z.-H. 2015. A simple homotopy algorithm for compressive sensing. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, 1116-1124.