Randomized methods for linear constraints: Convergence rates and conditioning

Mathematics of Operations Research

Volumn 35, Issue 3, 2010, Pages 641-654

  Leventhal, D ^a   Lewis, A S ^a  

^a Cornell University   (United States)

Author keywords

Averaged projections; Condition number; Coordinate descent; Distance to ill posedness; Error bound; Iterated projections; Linear constraint; Metric regularity; Randomization

Indexed keywords

AVERAGED PROJECTIONS; CONDITION NUMBERS; COORDINATE DESCENT; DISTANCE TO ILL-POSEDNESS; ERROR BOUND; ITERATED PROJECTIONS; LINEAR CONSTRAINTS; METRIC REGULARITY; RANDOMIZATION;

ALGORITHMS; ECHO SUPPRESSION; LINEAR EQUATIONS; NUMBER THEORY; PROBABILITY DISTRIBUTIONS;

CONVERGENCE OF NUMERICAL METHODS;

EID: 77956625417     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.1100.0456     Document Type: Article

References (45)

1. Akaike, H. 1959. On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method. Ann. Inst. Statist. Math. 11 1-16.
2. Amaldi, E., P. Belotti, R. Hauser. 2005. Randomized relaxation methods for the maximum feasible subsystem problem. M. Jünger, V. Kaibel, eds. Integer Programming and Combinatorial Optimization, Lecture Notes in Computer Science, Vol.3509. Springer-Verlag, Berlin, 249-264.
3. Amemiya, I., T. Ando. 1965. Convergence of random products of contractions in Hilbert space. Acta. Sci. Math. 26 239-244.
4. Bauschke, H. H. 1995. A norm convergence result on random products of relaxed projections in Hilbert space. Trans. Amer. Math. Soc. 347 1365-1373.
5. Bauschke, H. H. 2001. Projection algorithms: Results and open problems. D. Butnariu, Y. Censor, S. Reich, eds. Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, Elsevier, Amsterdam, 11-22.
6. Bauschke, H. H., J. M. Borwein. 1993. On the convergence of von Neumann's alternating projection algorithm for two sets. Set-Valued Anal. 1 185-212.
7. Bauschke, H. H., J. M. Borwein. 1996. On projection algorithms for solving convex feasibility problems. SIAM Rev. 38 367-426.
8. Billingsley, P. 1986. Probability and Measure. John Wiley & Sons, New York.
9. Bruck, R. E. 1982. Random products of contractions in metric and Banach spaces. J. Math. Anal. Appl. 88 319-332.
10. Demmel, J. W. 1987. On condition numbers and the distance to the nearest ill-posed problem. Numerische Mathematik 51 251-289.
11. Demmel, J. W. 1988. The probability that a numerical analysis problem is difficult. Math. Comput. 50 449-480.
12. Deutsch, F. 2001. Best Approximation in Inner Product Spaces. Springer-Verlag, New York.
13. Dontchev, A. L., A. S. Lewis, R. T. Rockafellar. 2003. The radius of metric regularity. Trans. Amer. Math. Soc. 355 493-517.
14. Dye, J., M. A. Khamsi, S. Reich. 1991. Random products of contractions in Banach spaces. Trans. Amer. Math. Soc. 325 87-99.
15. Eckart, C., G. Young. 1936. The approximation of one matrix by another of low rank. Psychometrika 1 211-218.
16. Golub, G., C. van Loan. 1996. Matrix Computations. Johns Hopkins University Press, Baltimore.
17. Gubin, L. G., B. T. Polyak, E. V. Raik. 1967. The method of projections for finding the common point of convex sets. U.S.S.R. Comput. Math. Math. Phys. 7 1-24.
18. Güler, O., A. J. Hoffman, U. Rothblum. 1995. Approximations to solutions to systems of linear inequalities. SIAM J. Matrix Anal. Appl. 16 688-696.
19. Ho, J. C. K., L. Tunçel. 2002. Reconciliation of various complexity and condition measures for linear programming problems and a generalization of Tardos' theorem. Foundations Comput. Math.: Proc. Smalefest 2000, World Scientific, Singapore, 93-148.
20. Hoffman, A. J. 1952. On approximate solutions of systems of linear inequalities. J. Res. National Bureau Standards 49 263-265.
21. Horn, R., C. Johnson. 1999. Matrix Analysis. Cambridge University Press, Cambridge, UK.
22. Leventhal, D. 2009. Metric subregularity and the proximal point method. J. Math. Anal. Appl. 360 681-688.
23. Lewis, A. S., D. R. Luke, J. Malick. 2009. Local convergence for alternating and averaged nonconvex projections. Foundations Comput. Math. 9 485-513.
24. Li, W. 1993. The sharp Lipschitz constants for feasible and optimal solutions of a perturbed linear program. Linear Algebra Appl. 187 15-40.
25. Luo, Z.-Q. 1991. On the convergence of the LMS algorithm with adaptive learning rate for linear feedback networks. Neural Comput. 3 226-245.
26. Luo, Z.-Q., P. Tseng. 1992a. On the convergence of the coordinate descent method for convex differentiable minimization. J. Optim. Theory Appl. 72 7-35.
27. Luo, Z.-Q., P. Tseng. 1992b. On the linear convergence of descent methods for convex essentially smooth minimization. SIAM J. Control Optim. 30 408-425.
28. Luo, Z.-Q., P. Tseng. 1992c. Error bound and convergence analysis of matrix splitting algorithms for the affine variational inequality problem. SIAM J. Optim. 2 43-54.
29. Luo, Z.-Q., P. Tseng. 1993a. Error bound and reduced gradient projection algorithms for convex minimization over a polyhedral set. SIAM J. Optim. 3 43-59.
30. Luo, Z.-Q., P. Tseng. 1993b. On the convergence rate of dual ascent methods for strictly convex minimization. Math. Oper. Res. 18 846-867.
31. Luo, Z.-Q., P. Tseng. 1993c. Error bounds and convergence analysis of feasible descent methods: A general approach. Ann. Oper. Res. 46 157-178.
32. Luo, Z.-Q., P. Tseng. 1994. Analysis of an approximate gradient projection method with applications to the back propagation algorithm. Optim. Methods Software 4 85-101.
33. Nesterov, Y. 2010. Efficiency of coordinate descent methods on huge-scale optimization problems. CORE Discussion Paper 2010/2, Université Catholique de Louvain, Center for Operations Research and Economics, Louvain-la-Neuve, Belgium.
34. Ng, K. F., X. Y. Zheng. 2004. Hoffman's least error bounds for linear inequalities. J. Global Optim. 30 391-403.
35. Pang, J.-S. 1997. Error bounds in mathematical programming. Math. Programming 79 299-332.
36. Pierra, G. 1984. Decomposition through formalization in a product space. Math. Programming 28 96-115.
37. Polyak, B. T. 2001. Random algorithms for solving convex inequalities. D. Butnariu, Y. Censor, S. Reich, eds. Inherently Parallel Algorithms in Feasibility and Other Applications. Elsevier, Amsterdam.
38. Quarteroni, A., R. Sacco, F. Saleri. 2007. Numerical Mathematics. Springer-Verlag, New York.
39. Renegar, J. 1994. Perturbation theory for linear programming. Math. Programming 65 73-91.
40. Renegar, J. 1995a. Incorporating conditions measures into the complexity theory of linear programming. SIAM J. Optim. 5 506-524.
41. Renegar, J. 1995b. Linear programming, complexity theory and elementary functional analysis. Math. Programming 70 279-351.
42. Rockafellar, R. T., R. J.-B. Wets. 1998. Variational Analysis. Springer-Verlag, Berlin.
43. Strohmer, T., R. Vershynin. 2009. A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15 262-278.
44. Tseng, P. 1995. On linear convergence of iterative methods for the variational inequality problem. J. Comput. Appl. Math. 60 237-252.
45. Zhang, S. 2000. Global error bounds for convex conic problems. SIAM J. Optim. 10 836-851.