Convergence of a block coordinate descent method for nondifferentiable minimization

Journal of Optimization Theory and Applications

Volumn 109, Issue 3, 2001, Pages 475-494

  Tseng, P ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

Block coordinate descent; Convergence; Gauss seidel method; Nondifferentiable minimization; Pseudoconvex functions; Quasiconvex functions; Stationary point

Indexed keywords

BLOCK COORDINATE DESCENTS; CONVERGENCE; COORDINATE DESCENT METHODS; GAUSS SEIDEL METHOD.; MINIMISATION; NON-DIFFERENTIABLE; NONDIFFERENTIABLE MINIMIZATION; PSEUDOCONVEX FUNCTIONS; QUASICONVEX FUNCTIONS; STATIONARY POINTS;

BLIND SOURCE SEPARATION;

EID: 0035533631     PISSN: 00223239     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1017501703105     Document Type: Article

References (39)

1. 1. ORTEGA, J. M., and RHEINBOLDT, W. C., Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, NY, 1970.
2. 2. HILDRETH, C., A Quadratic Programming Procedure, Naval Research Logistics Quarterly, Vol. 4, pp. 79-85, 1957; see also Erratum, Naval Research Logistics Quarterly, Vol. 4, p. 361, 1957.
3. 2. HILDRETH, C., A Quadratic Programming Procedure, Naval Research Logistics Quarterly, Vol. 4, pp. 79-85, 1957; see also Erratum, Naval Research Logistics Quarterly, Vol. 4, p. 361, 1957.
4. 3. WARGA, J., Minimizing Certain Convex Functions, SIAM Journal on Applied Mathematics, Vol. 11, pp. 588-593, 1963.
5. 4. AUSLENDER, A., Optimisation Méthodes Numériques, Masson, Paris, France, 1976.
6. 5. BERTSEKAS, D. P., Nonlinear Programming, 2nd Edition, Athena Scientific, Belmont, Massachusetts, 1999.
7. 6. BERTSEKAS, D. P., and TSITSIKLIS, J. N., Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, Englewood Cliffs, New Jersey, 1989.
8. 7. CENSOR, Y., and ZENIOS, S. A., Parallel Optimization: Theory, Algorithms, and Applications, Oxford University Press, Oxford, United Kingdom, 1997.
9. 8. LUENBERGER, D. G., Linear and Nonlinear Programming, Addison-Wesley, Reading, Massachusetts, 1973.
10. 9. POLAK, E., Computational Methods in Optimization: A Unified Approach, Academic Press, New York, NY, 1971.
11. 10. ZANGWILL, W. I., Nonlinear Programming, Prentice-Hall, Englewood Cliffs, New Jersey, 1969.
12. 11. ARIMOTO, S., An Algorithm for Computing the Capacity of Arbitrary DMCs, IEEE Transactions on Information Theory, Vol. 18, pp. 14-20, 1972.
13. 12. BLAHUT, R., Computation of Channel Capacity and Rate Distortion Functions, IEEE Transactions on Information Theory, Vol. 18, pp. 460-473, 1972.
14. 13. HOWSON, H. R., and SANCHO, N. G. F., A New Algorithm for the Solution of Multistate Dynamic Programming Problems, Mathematical Programming, Vol. 8, pp. 104-116, 1975.
15. 14. KORSAK, A. J., and LARSON, R. E., A Dynamic Programming Successive Approximations Technique with Convergence Proofs, Automatica, Vol. 6, pp. 253-260, 1970.
16. 15. ZUO, Z. Q., and WU, C. P., Successive Approximation Technique for a Class of Large-Scale NLP Problems and Its Application to Dynamic Programming, Journal of Optimization Theory and Applications, Vol. 62, pp. 515-527, 1989.
17. 16. STERN, T. E., A Class of Decentralized Routing Algorithms Using Relaxation, IEEE Transactions on Communications, Vol. 25, pp. 1092-1102, 1977.
18. 17. BREGMAN, L. M., The Relaxation Method o7f Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming, USSR Computational Mathematics and Mathematical Physics, Vol. 7, pp. 200-217, 1967.
19. 18. HAN, S. P., A Successive Projection Method, Mathematical Programming, Vol. 40, pp. 1-14, 1988.
20. 19. KIWIEL, K. C., Free-Steering Relaxation Methods for Problems with Strictly Convex Costs and Linear Constraints, Mathematics of Operations Research, Vol. 22, pp. 326-349, 1997.
21. 20. LUO, Z. Q., and TSENG, P., On the Convergence Rate of Dual Ascent Methods for Strictly Convex Minimization, Mathematics of Operations Research, Vol. 18, pp. 846-867, 1993.
22. 21. TSENG, P., Dual Ascent Methods for Problems with Strictly Convex Costs and Linear Constraints: A Unified Approach, SIAM Journal on Control and Optimization, Vol. 28, pp. 214-242, 1990.
23. 22. TSENG, P., Dual Coordinate Ascent Methods for Nonstrictly Convex Minimization, Mathematical Programming, Vol. 59, pp. 231-247, 1993.
24. 23. MANGASARIAN, O. L., and DE LEONE, R., Parallel Successive Overrelaxation Methods for Symmetric Linear Complementarity Problems and Linear Programs, Journal of Optimization Theory and Applications, Vol. 54, pp. 437-446, 1987.
25. 24. CEA, J., and GLOWINSKI, R., Sur des Methodes d'Optimisation par Relaxation, Revue Française d'Automatique, Informatique et Recherche Opérationnelle, Vol. R3, pp. 5-32, 1973.
26. 25. SARGENT, R. W. H., and SEBASTIAN, D. J., On the Convergence of Sequential Minimization Algorithms, Journal of Optimization Theory and Applications, Vol. 12, pp. 567-575, 1973.
27. 26. ZADEH, N., A Note on the Cyclic Coordinate Ascent Method, Management Science, Vol. 16, pp. 642-644, 1970.
28. 27. GRIPPO, L., and SCIANDRONE, M., On the Convergence of the Block Nonlinear Gauss-Seidel Method under Convex Constraints, Operations Research Letters, Vol. 26, pp. 127-136, 2000.
29. 28. POWELL, M. J. D., On Search Directions for Minimization Algorithms, Mathematical Programming, Vol. 4, pp. 193-201, 1973.
30. 29. LUO, Z. Q., and TSENG, P., Error Bounds and Convergence Analysis of Feasible Descent Methods: A General Approach, Annals of Operations Research, Vol. 46, pp. 157-178, 1993.
31. 30. HAN, S. P., A Decomposition Method and Its Application to Convex Programming, Mathematics of Operations Research, Vol. 14, pp. 237-248, 1989.
32. 31. BOFILL, P., and ZIBULEVSKY, M., Sparse Undetermined ICA: Estimating the Mixing Matrix and the Sources Separately, Technical Report UPC-DAC-2000-7, Universitat Politècnica de Catalunya, Barcelona, Spain, 1999.
33. 32. SARDY, S., BRUCE, A., and TSENG, P., Block Coordinate Relaxation Methods for Nonparametric Wavelet Denoising, Journal of Computational and Graphical Statistics, Vol. 9, pp. 361-379, 2000.
34. 33. ZIBULEVSKY, M., and PEARLMUTTER, B., Blind Source Separation by Sparse Decomposition, Technical Report CS99-1, Computer Science Department, University of New Mexico, Albuquerque, New Mexico, 1999.
35. 34. MANGASARIAN, O. L., Nonlinear Programming, McGraw-Hill, New York, NY, 1969.
36. 35. ROCKAFELLAR, R. T., Convex Analysis, Princeton University Press, Princeton, New Jersey, 1970.
37. 36. MARTINET, B., Determination Approchée d'un Point Fixe d'une Application Pseudo-Contractante: Cas de l'Application Prox, Comptes Rendus des Séances de l'Académie des Sciences, Vol. 274A, pp. 163-165, 1972.
38. 37. ROCKAFELLAR, R. T., Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming, Mathematics of Operations Research, Vol. 1, pp. 97-116, 1976.
39. 38. BAUSCHKE, H. H., and LEWIS, A. S., Dykstra's Algorithm with Bregman Projections: A Convergence Proof, Optimization (to appear).