Efficient accelerated coordinate descent methods and faster algorithms for solving linear systems

Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS

Volumn , Issue , 2013, Pages 147-156

  Lee, Yin Tat ^a   Sidford, Aaron ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b USA   (United States)

Author keywords

Convex optimization; Coordinate descent; Kaczmarz method; Symmetric diagonally dominant matrix

Indexed keywords

ASYMPTOTICALLY OPTIMAL; COORDINATE DESCENT; COORDINATE DESCENT METHODS; DIAGONALLY DOMINANT MATRIX; KACZMARZ METHOD; LINEAR SYSTEM SOLVER; OVERDETERMINED SYSTEMS; SYMMETRIC POSITIVE DEFINITE;

ALGORITHMS; COMPUTER SCIENCE; CONVEX OPTIMIZATION; IMAGE RECONSTRUCTION; LINEAR SYSTEMS;

ITERATIVE METHODS;

EID: 84893479942     PISSN: 02725428     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/FOCS.2013.24     Document Type: Conference Paper

References (28)

1. S. Kaczmarz, "Angenäherte auflösung von systemen linearer gleichungen, " Bull. Internat. Acad. Polon.Sci. Lettres A, p. 335-57, 1937.
2. W. Zangwill, Nonlinear Programming: A Unified Approach. Prentice-Hall, 1969.
3. H. H. Bauschke and J. M. Borwein, "On projection algorithms for solving convex feasibility problems, " SIAM Rev., vol. 38, no. 3, pp. 367-426, Sep. 1996.
4. F. Natterer, The mathematics of computerized tomography. Philadelphia, PA, USA: Society for Industrial and Applied Mathematics, 2001.
5. G. T. Herman, Fundamentals of computerized tomography: image reconstruction from projections. Springer, 2009.
6. T. Strohmer and R. Vershynin, "A Randomized Kaczmarz Algorithm with Exponential Convergence, " Journal of Fourier Analysis and Applications, vol. 15, no. 2, pp. 262-278, Apr. 2008.
7. Y. Nesterov, "Efficiency of coordinate descent methods on huge-scale optimization problems, " SIAM Journal on Optimization, vol. 22, no. 2, pp. 341-362, 2012.
8. J. A. Kelner, L. Orecchia, A. Sidford, and Z. A. Zhu, "A Simple, Combinatorial Algorithm for Solving SDD Systems in Nearly-Linear Time, " Jan. 2013.
9. J. A. Kelner and A. Madry, "Faster generation of random spanning trees, " in Proceedings of the 2009 50th Annual IEEE Symposium on Foundations of Computer Science, ser. FOCS '09. Washington, DC, USA: IEEE Computer Society, 2009, pp. 13-21.
10. P. Christiano, J. A. Kelner, A. Madry, D. A. Spielman, and S.-H. Teng, "Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs, " in Proceedings of the 43rd annual ACM symposium on Theory of computing. ACM, 2011, pp. 273-282.
11. D. A. Spielman, "Algorithms, graph theory, and the solution of laplacian linear equations, " in Proceedings of the 39th international colloquium conference on Automata, Languages, and Programming - Volume Part II, ser. ICALP'12. Berlin, Heidelberg: Springer-Verlag, 2012, pp. 24-26.
12. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, 2003, vol. I.
13. C. T. Kelley, Iterative methods for optimization. Society for Industrial and Applied Mathematics, 1987, vol. 18.
14. Y. Nesterov, "A method for solving a convex programming problem with convergence rate 1/k̂2, " Doklady AN SSSR, vol. 269, pp. 543-547, 1983.
15. A. Beck and M. Teboulle, "A Fast Iterative Shrinkage- Thresholding Algorithm for Linear Inverse Problems, " SIAM Journal on Imaging Sciences, vol. 2, no. 1, pp. 183-202, Jan. 2009.
16. S. Becker, J. Bobin, and E. Candès, "NESTA: A fast and accurate first-order method for sparse recovery, " SIAM Journal on Imaging Sciences, vol. 4, no. 1, pp. 1-39, 2011.
17. Y. T. Lee, S. Rao, and N. Srivastava, "A New Approach to Computing Maximum Flows using Electrical Flows, " Proceedings of the 45th symposium on Theory of Computing - STOC '13, 2013.
18. Y. Nesterov, "Rounding of convex sets and efficient gradient methods for linear programming problems, " Optimization Methods Software, vol. 23, no. 1, pp. 109-128, Feb. 2008.
19. D. Bienstock and G. Iyengar, "Solving fractional packing problems in oast(1/esp) iterations, " in Proceedings of the thirty-sixth annual ACM symposium on Theory of computing, ser. STOC '04. New York, NY, USA: ACM, 2004, pp. 146- 155.
20. Z. Q. Luo and P. Tseng, "On the convergence of the coordinate descent method for convex differentiable minimization, " Journal of Optimization Theory and Applications, vol. 72, no. 1, pp. 7-35, Jan. 1992.
21. P. Richt'arik and M. Tak'ač, "Iteration Complexity of Randomized Block-Coordinate Descent Methods for Minimizing a Composite Function, " p. 33, Jul. 2011.
22. D. A. Spielman and S.-H. Teng, "Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems, " in Proceedings of the thirty-sixth annual ACM symposium on Theory of computing. ACM, 2004, pp. 81-90.
23. J. A. Kelner, L. Orecchia, Y. T. Lee, and A. Sidford, "An almost-linear-time algorithm for approximate max flow in undirected graphs, and its multicommodity generalizations, " arXiv preprint arXiv:1304.2338, 2013.
24. I. Koutis, G. L. Miller, and R. Peng, "A nearly-m log n time solver for sdd linear systems, " in Foundations of Computer Science (FOCS), 2011 IEEE 52nd Annual Symposium on, oct. 2011, pp. 590 -598.
25. T. Goldstein, B. ODonoghue, and S. Setzer, "Fast alternating direction optimization methods, " CAM report, pp. 12-35, 2012.
26. S. Boyd and L. Vandenberghe, Convex Optimization. New York, NY, USA: Cambridge University Press, 2004.
27. D. A. Spielman and J. Woo, "A Note on Preconditioning by Low-Stretch Spanning Trees, " Mar. 2009.
28. I. Abraham and O. Neiman, "Using petal-decompositions to build a low stretch spanning tree, " in Proceedings of the 44th symposium on Theory of Computing - STOC '12. New York, New York, USA: ACM Press, May 2012, p. 395.