An efficient method, for large-scale l1-regularized convex loss minimization

2007 Information Theory and Applications Workshop, Conference Proceedings, ITA

Volumn , Issue , 2007, Pages 223-230

  Koh, Kwangmoo ^a   Kim, Seung Jean ^a   Boyd, Stephen ^a  

^a Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CYBERNETICS;

EFFICIENT METHODS; LOSS MINIMIZATION;

INFORMATION THEORY;

EID: 48049121521     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ITA.2007.4357584     Document Type: Conference Paper

References (51)

1. D. Bertsekas. Nonlinear Programming. Athena Scientific, second edition, 1999.
2. J Borwein and A Lewis. Convex Analysis and Nonlinear Optimization. Springer, 2000.
3. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
4. E. Candès. Compressive sampling. In Proceedings of International Congress of Mathematics, 2006.
5. E. Candès and J. Romberg. Quantitative robust uncertainty principles and optimally sparse decompositions. Foundations of Computational Mathematics, 6(2):227-254, 2006.
6. E. Candès, J. Romberg, and T. Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 52(2):489-509, 2006.
7. E. Candès and T. Tao. Near optimal signal recovery from random projections: universal encoding strategies? IEEE Transactions on. Information Theory, 52(12):5406-5425, 2006.
8. A. Conn, N Gould, and Ph. Toint. LANCELOT: A Fortran package for large-scale nonlinear optimization (Release A), volume 17 of Springer Series in Computational Maihemaiics. Springer-Verlag, 1992.
9. R. Dembo and T. Steihaug. Truncated-Newton algorithms for large-scale unconstrained optimization. Math. Program., 26:190-212, 1983.
10. J. Demmel. Applied Numerical Linear. Algebra. Society for Industrial and Applied Mathematics, 1997.
11. D. Donoho. Compressed sensing, IEEE Transactions on Information Theory, 52(4): 1289-1306, 2006.
12. 1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematic, .59(6):797-829, 2006.
13. 1 minimization. Proc. Nat. Aca. Sci., 100(5):2197-2202, March 2003.
14. D. Donoho and X. Huo. Uncertainty principle and ideal atomic decomposition. IEEE Transactions on Information Theory, 47(7):2845-2862, 2001.
15. B. Efron, T. Hastie, I. Johnstone, and R, Tibshirani. Least angle regression. Annals of Statistics, 32(2):407-499, 2004.
16. M, Elad and A. Bruckstein. A generalized uncertainty principle and sparse representation in pairs of bases. IEEE Transactions on Information Theory, 48(9):2558-2567, 2002.
17. A. Feuer and A. Nemirovski. On sparse representation in pairs of bases. IEEE Transactions on Information Theory, 49(6):l579-1581, 2003.
18. P. Gill W. Murray, M. Saunders, and M. Wright. User's guide for NPSOL (Version 4.0): A FORTRAN package for nonlinear programming. Technical Report. SOL 86-2, Operations Research Dept., Stanford University. Stanford, California 94305, January 1986.
19. R. Gribonval and M. Nielsen. Sparse representation in unions of bases. IEEE Transactions on Information Theory, 48(12):3320-3325, 2003.
20. T. Hastie, R. Tibshirani, and J. Friedman. The elements of statistical learning. Springer Series in Statistics. Springer-Verlag, New York, 2001.
21. S. Keerthi and D. DeCoste. A modified finite Newton method for fast solution of large scale linear SVMs. Journal of Machine Learning Research, 6:341-361, 2005.
22. K. Knight and W. Fu. Asymptotics for Lasso-type estimators. Annals of Statistics, 28(5):1356-1378, 2000.
23. 1-regularized logistic regression, 2006. Manuscript. Available from www. Stanford.edu/∼boyd/11_logistic_reg.html.
24. K. Lang Newsweeder Learning to filter netnews. In Proceedings of the twenty-first international conference on Machine learning (ICML), pages 331-339, 1995.
25. 1-regularized logistic regression. In Proceedings of the 21st National. Conference on Artificial Intelligence (AAAI-06). 2006.
26. M. Lustig, D. Donoho, and J. Pauly. Rapid MR imaging with compressed sensing and randomly under-sampled 3DFT trajectories. In Proceedings of the 14th Annual Meeting of ISMRM, 2006.
27. M. Lustig, D. Donoho, and J. Pauly. Sparse MRI: The application of compressed sensing for rapid MR imaging, 2007. Manuscript. Available from www Stanford.edu/∼mlustig/.
28. M. Lustig, J. Lee, D. Donoho, and J. Pauly. k-t SPARSE: High frame rate dynamic mri exploiting spatio-temporal sparsity. In Proceedings of the 14th Annual Meeting of.ISMRM, 2006.
29. A. McCallum. Bow: A toolkit for statistical language modeling, test retrieval, classification and clustering. Available from www.cs .cmu. edu/∼mccallum/bow.1996.
30. N. Meinshausen and P. Büuhlmann. High-dimensional graphs and variable selection with the Lasso. Annals of Statistics, 34(3): 14361462, 2006.
31. MOSEK ApS. The MOSEK Optimization Tools Version 2.5 User's Manual and .Reference, 2002. Available from www.mosek.com.
32. Y. Nesterov and A. Nemirovsky. Interior-Point Polynomial Methods in Convex Programming, volume 13 of Studies in Applied Mathematics. SIAM, Philadelphia, PA, 1994.
33. J. Nocedal and S. Wright. Numerical Optimization. Springer Series in Operations Research. Springer, 1999.
34. 1 regularization-path algorithm for generalized linear models, 2006. Manuscript. Available from ww-stat. Stanford.edu/∼hastie/Papers/glmpath.pdf.
35. B. Polyak Introduction to Optimization. Optimization Software, 1987. Translated from Russian.
36. L. Portugal, M. Resende, G. Veiga, and J. Judice. A trancated primal-infeasible dual-feasible network interior point method, 1994.
37. A. Ruszczynski. Nonlinear Optimization. Princeton university press, 2006
38. N. Z. Shor. Minimization Methods for Non-differentiable Functions. Springer Series in Computational Mathematics. Springer, 1985.
39. R. Tibshirani. Regression shrinkage and selection via. the Lasso. Journal of the Royal Statistical Society Series B, 58(I), 267-288, 1996.
40. R. Tibshirani. The Lasso for variable selection in the Cox model. Statistics in Medicine, 16:385-395, 1997.
41. 1 minimization. IEEE Transactions on Information Theory, 51(4):1568-1570, 2005.
42. J. Tropp. Just relax: Convex programming methods for identifying sparse signals iti noise. IEEE Transactions on Information Theory, 53(3):1030-1051,2006.
43. Y. Tsaig and D. Donoho. Extensions of compressed sensing. Signal Processing, 86(3):549-571, 2006.
44. L. Vandenberghe and S. Boyd. A primal-dual potential reduction method for problems involving matrix inequalities. Math. Program., 69:205-236,1995.
45. R. Vanderbei. LOQO User's Manual. - Version 3.10, 1997. Available from www.crfe.princeton.edu/Ioqo.
46. M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar. K. Kelly, and P. Baraniuk. An architecture for compressive imaging. In Proceedings of International Conference on Image Processing (ICIP), 2006.
47. M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R, Baraniuk. Compressive imaging for video representation and coding. In Proceedings of Picture Coding Symposium (PCS). 2006.
48. S. Wright. Primal-dual interior-point methods. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1997.
49. Y Ye. Interior Point Algorithms: Theory and Analysis. John Wiley & Sons, 1997.
50. P. Zhao and B. Yu. On model selection consistency of Lasso. Journal of Machine Learning Research, 7:2541-2563, 2006.
51. H. Zou. The adaptive Lasso and its oracle properties. Journal of the American Statistical Association, 101 (476): 1418-1429, 2006.