Constant nullspace strong convexity and fast convergence of proximal methods under high-dimensional settings

Advances in Neural Information Processing Systems

Volumn 2, Issue January, 2014, Pages 1008-1016

  Yen, Ian E H ^a   Hsieh, Cho Jui ^a   Ravikumar, Pradeep ^a   Dhillon, Inderjit ^a  

^a University of Texas at Austin   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONVEX OPTIMIZATION; INFORMATION SCIENCE; NEWTON-RAPHSON METHOD;

HIGH-DIMENSIONAL PROBLEMS; LOCAL QUADRATIC CONVERGENCE; NUMERICAL EXPERIMENTS; OPTIMIZATION METHOD; QUALITATIVE DIFFERENCES; REGULARIZATION FUNCTION; STATISTICAL ESTIMATION; STATISTICAL ESTIMATORS;

SAMPLING;

EID: 84937930992     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. J. D. Lee, Y. Sun, and M. A. Saunders. Proximal newton-type methods for minimizing composite functions. In NIPS, 2012.
2. C.-J. Hsieh, M. A. Sustik, I. S. Dhillon, and P. Ravikumar. Sparse inverse covariance estimation using quadratic approximation. In NIPS 2011.
3. S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge Univ. Press, Cambridge, U.K., 2003.
4. P.-W. Wang and C.-J. Lin. Iteration Complexity of Feasible Descent Methods for Convex Optimization. Technical report, Department of Computer Science, National Taiwan University, Taipei, Taiwan, 2013.
5. A. Agarwal, S. Negahban, and M. Wainwright. Fast Global Convergence Rates of Gradient Methods for High-Dimensional Statistical Recovery. In NIPS 2010.
6. K. Hou, Z. Zhou, A. M.-S. So, and Z.-Q. Luo, On the linear convergence of the proximal gradient method for trace norm regularization, in Neural Information Processing Systems (NIPS), 2013.
7. L. Xiao and T. Zhang, A proximal-gradient homotopy method for the l1-regularized leastsquares problem, in ICML, 2012.
8. P. Tseng and S. Yun, A coordinate gradient descent method for nonsmooth separable minimization, Math. Prog. B. 117 (2009).
9. G.-X. Yuan, C.-H. Ho, and C.-J. Lin, An improved GLMNET for l1-regularized logistic regression, Journal of Machine Learning Research, Vol. 13, pp. 1999-2030, 2012
10. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, LIBLINEAR: A library for large linear classification, Journal of Machine Learning Research, Vol. 9, pp. 1871-1874, 2008.
11. Alan J Hoffman. On approximate solutions of systems of linear inequalities. Journal of Research of the National Bureau of Standards, 1952.
12. Tewari, A, Ravikumar, P, and Dhillon, I S. Greedy Algorithms for Structurally Constrained High Dimensional Problems. In NIPS, 2011.
13. S. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu. A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. In NIPS, 2009.
14. A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1): 183-202, 2009.
15. S. Becker, J. Bobin, and E.J. Candes. Nesta: a fast and accurate first-order method for sparse recovery. SIAM Journal on Imaging Sciences, 2011.
16. Z. Q. Luo and P. Tseng. Error bounds and convergence analysis of feasible descent methods: a general approach. Annals of Operations Research, 46-47: 157-178, 1993.
17. Rahul Garg and Rohit Khandekar. Gradient Descent with Sparsification: an iterative algorithm for sparse recovery with restricted isometry property. In ICML 2009.
18. S. Ji and J. Ye. An accelerated gradient method for trace norm minimization. In ICML, 2009.
19. Y. E. Nesterov, Gradient Methods for Minimizing Composite Objective Function, CORE report, 2007.
20. Y. Nesterov. Introductory Lectures on Convex Optimization. Kluwer Academic Publishers, New York, 2004
21. K. Scheinberg, X. Tang. Practical Inexact Proximal Quasi-Newton Method with Global Complexity Analysis. COR@L Technical Report at Lehigh University. arXiv: 1311.6547, 2013.