Stabilization of stochastic iterative methods for singular and nearly singular linear systems

Mathematics of Operations Research

Volumn 39, Issue 1, 2014, Pages 1-30

  Wang, Mengdi ^a   Bertsekas, Dimitri P ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Approximate dynamic programming; Projected equation; Regularization; Simulation; Singular system; Sstabilization; Stochastic algorithm

Indexed keywords

ALGORITHMS; LINEAR SYSTEMS; STABILIZATION; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

APPROXIMATE DYNAMIC PROGRAMMING; PROJECTED EQUATION; REGULARIZATION; SIMULATION; SINGULAR SYSTEM; SSTABILIZATION; STOCHASTIC ALGORITHMS;

ITERATIVE METHODS;

EID: 84899690779     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.2013.0596     Document Type: Article

References (51)

1. Benveniste A, Metivier M, Priouret P (1990) Adaptive Algorithms and Stochastic Approximations (Springer-Verlag, New York).
2. Bertsekas DP (1999) Nonlinear Programming, 2nd ed. (Athena Scientific, Belmont, MA).
3. Bertsekas DP (2007) Dynamic Programming and Optimal Control: Vol. I and II (Athena Scientific, Belmont, MA).
4. Bertsekas DP (2010) Approximate policy iteration: A survey and some new methods. J. Control Theory Appl. 9:310-335.
5. Bertsekas DP (2011) Temporal difference methods for general projected equations. IEEE Trans. Automatic Control 56:2128-2139.
6. Bertsekas DP, Gafni EM (1982) Projection methods for variational inequalities with application to the traffic assignment problem. Math. Programming Study 17:139-159.
7. Bertsekas DP, Tsitsiklis JN (1989) Parallel and Distributed Computation: Numerical Methods (Athena Scientific, Belmont, MA).
8. Bertsekas DP, Tsitsiklis JN (1996) Neuro-Dynamic Programming (Athena Scientific, Belmont, MA).
9. Bertsekas DP, Yu H (2009) Projected equation methods for approximate solution of large linear systems. J. Comput. Appl. Math. 227:27-50.
10. Borkar VS (2008) Stochastic Approximation: A Dynamical Systems Viewpoint (Cambridge University Press, New York).
11. Boyan JA (2002) Technical update: Least-squares temporal difference learning. Machine Learn. 49:233-246. (Pubitemid 34325688)
12. Bradtke SJ, Barto AG (1996) Linear least-squares algorithms for temporal difference learning. Machine Learn. 22:33-57. (Pubitemid 126724362)
13. Campbell SL, Meyer CD (1991) Generalized Inverses of Linear Transformations (Dover, New York).
14. Censor J, Herman GT, Jiang M (2009) A note on the behavior of the randomized Kaczmarz algorithm of Strohmer and Vershynin. J. Fourier Anal. Appl. 15:431-436.
15. Cottle RW, Pang JS, Stone RE (1992) The Linear Complementarity Problem (Academic Press, Boston).
16. Curtiss JH (1954) A theoretical comparison of the efficiencies of two classical methods and a Monte Carlo method for computing one component of the solution of a set of linear algebraic equations. Meyer HA, ed. Sympos. Monte Carlo Methods (John Wiley & Sons, New York), 191-233.
17. Curtiss JH (1957) A Monte Carlo methods for the iteration of linear operators. Uspekhi Mat. Nauk 12:149-174.
18. Cvetkovic-llic DS (2008) A note on the representation for the Drazin inverse of 2 times; 2 block matrices. Linear Algebra Its Appl. 429:242-248.
19. Dax A (1990) The convergence of linear stationary iterative processes for solving singular unstructured systems of linear equations. SIAM Rev. 32:611-635.
20. Drineas P, Kannan R, Mahoney MW (2006a) Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication. SIAM J. Comput. 35:132-157. (Pubitemid 46374021)
21. Drineas P, Kannan R, Mahoney MW (2006b) Fast Monte Calo algorithms for matrices II: Computing a low-rank approximation to a matrix. SIAM J. Comput. 36:158-183. (Pubitemid 46374022)
22. Drineas P, Mahoney MW, Muthukrishnan S (2006c) Sampling algorithms for L2 regression and applications. Proc. 17th Annual SODA, 1127-1136.
23. Drineas P, Mahoney MW, Muthukrishnan S (2008) Relative-error CUR matrix decompositions. SIAM J. Matrix Anal. Appl. 30:844-881.
24. Drineas P, Mahoney MW, Muthukrishnan S, Sarlos T (2011) Faster least squares approximation. Numerische Mathematik 117:219-249.
25. Eiermann M, Marek I, Niethammer W (1988) On the solution of singular linear systems of algebraic equations by semiiterative methods. Numerische Mathematik 53:265-283.
26. Facchinei F, Pang JS (2003) Finite-Dimensional Variational Inequalities and Complementarity Problems (Springer, New York).
27. Forsythe GE, Leibler RA (1950) Matrix inversion by a Monte Carlo method. Math. Tables Other Aids Comput. 4:127-129.
28. Hageman LA, Young DM (1981) Applied Iterative Methods (Academic Press, New York).
29. Halton JH (1970) A retrospective and prospective survey of the Monte Carlo method. SIAM Rev. 12:1-63.
30. Keller HB (1965) On the solution of singular and semidefinite linear systems by iteration. J. SIAM: Series B, Numer. Anal. 2:281-290.
31. Korpelevich GM (1976) An extragradient method for finding saddle points and for other problems. Matecon 12:747-756.
32. Krasnoselskii MA (1972) Approximate Solution of Operator Equations (Wolters-Noordhoff, Groningen, The Netherlands).
33. Kushner HJ, Yin G (2003) Stochastic Approximation and Recursive Algorithms and Applications (Springer, New York).
34. Lancaster P, Tismenetsky M (1985) The Theory of Matrices (Academic Press, New York).
35. Leventhal D, Lewis AS (2010) Randomized methods for linear constraints: Convergence rates and conditioning. Math. Oper. Res. 35:641-654.
36. Luo ZQ, Tseng P (1989) On the convergence of a matrix splitting algorithm for the symmetric monotone linear complementarity problem. SIAM J. Control Optim. 29:1037-1060. (Pubitemid 21723027)
37. Martinet B (1970) Regularisation d'inequation variationelles par approximations successives. Rev. Francaise Inf. Rech. Oper. 4:154-159.
38. Meyn SP (2007) Control Techniques for Complex Systems (Cambridge University Press, New York).
39. Moser SM (2007) Some expectations of a non-central chi-square distribution with an even number of degrees of freedom. TENCON, IEEE Region 10 Conf.
40. Nedić A, Bertsekas DP (2003) Least-squares policy evaluation algorithms with linear function approximation. J. Discrete Event Systems 13:79-110.
41. Ortega JM, Rheinboldt WC (1970) Iterative Solution of Nonlinear Equations in Several Variables (Academic Press, New York).
42. Puterman ML (1994) Markovian Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley & Sons, New York).
43. Rockafellar RT (1976) Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14:877-898.
44. Sibony M (1970) Methodes iteratives pour les equations et inequations aux derivees partielles non lineaires de type monotone. Calcolo 7:65-183.
45. Stewart GW (1973) Introduction to Matrix Computations (Academic Press, New York).
46. Strohmer T, Vershynin R (2009) A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15:262-178.
47. Sutton RS, Barto AG (1998) Reinforcement Learning (MIT Press, Cambridge, MA).
48. Tanabe K (1974) Characterization of linear stationary iterative processes for solving a singular system of linear equations. Numerische Mathematik 22:349-359.
49. Wang M, Bertsekas DP (2013) On the convergence of simulation-based iterative methods for singular linear systems. Stochastic Sys. 3:38-95.
50. Wasow WR (1952) A note on inversion of matrices by random walks. Math. Tables Other Aids Comput. 6:78-81.
51. Young DM (1972) On the consistency of linear stationary iterative methods. SIAM J. Numer. Anal. 9:89-96.