On projection algorithms for solving convex feasibility problems

SIAM Review

Volumn 38, Issue 3, 1996, Pages 367-426

  Bauschke, Heinz H ^a   Borwein, Jonathan M ^a  

^a SIMON FRASER UNIVERSITY   (Canada)

Author keywords

Angle between two subspaces; Averaged mapping; Cimmino's method; Computerized tomography; Convex feasibility problem; Convex function; Convex inequalities; Convex programming; Convex set; Fej r monotone sequence; Firmly nonexpansive mapping

Indexed keywords

APPROXIMATION THEORY; COMPUTERIZED TOMOGRAPHY; CONVERGENCE OF NUMERICAL METHODS; FUNCTIONS; IMAGE RECONSTRUCTION; ITERATIVE METHODS; MATHEMATICAL PROGRAMMING; PROBLEM SOLVING;

CLASSICAL MATHEMATICS; CONVEX FEASIBILITY PROBLEMS; CONVEX INEQUALITIES; CONVEX PROGRAMMING; HILBERT SPACES; LINEAR CONVERGENCE; PROJECTION ALGORITHMS; SUBGRADIENT ALGORITHM;

ALGORITHMS;

EID: 0030246542     PISSN: 00361445     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0036144593251710     Document Type: Article

References (116)

1. S. AGMON, The relaxation method for linear inequalities, Canad. J. Math., 6 (1954), pp. 382-392.
2. R. AHARONI, A. BERMAN, AND Y. CENSOR, An interior points algorithm for the convex feasibility problem, Adv. Appl. Math., 4 (1983), pp. 479-489.
3. R. AHARONI AND Y. CENSOR, Block-iterative projection methods for parallel computation of solutions to convex feasibility problems, Linear Algebra Appl., 120 (1989), pp. 165-175.
4. E. ALLEN, R. HELGASON, J. KENNINGTON, AND B. SHETTY, A generalization of Polyak's convergence result for subgradient optimization, Math. Programming, 37 (1987), pp. 309-317.
5. I. AMEMIYA AND T. ANDO, Convergence of random products of contractions in Hilbert space, Acta Sci. Math. (Szeged), 26 (1965), pp. 239-244.
6. R. ANSORGE, Connections between the Cimmino-method and the Kaczmarz-methodfor the solution of singular and regular systems of equations. Computing, 33 (1984), pp. 367-375.
7. J. B. BAILLON, Comportement Asymptotique des Contractions et Semi-Groupes de Contractions, Ph.D. thesis, University of Paris VI, 1978. (In French.)
8. J. B. BAILLON AND H. BREZIS, Une remarque sur le comportement asymptotique des semigroupes non lineaires, Houston J. Math., 2 (1976), pp. 5-7. (In French.)
9. J. B. BAILLON, R. E. BRUCK, AND S. REICH, On the asymptotic behaviour of nonexpansive mappings and semigroups in Banach spaces, Houston J. Math., 4 (1978), pp. 1-9.
10. M. BARONTI AND P. L. PAPINI, Convergence of sequences of sets, in Methods of Functional Analysis in Approximation Theory, C. Micchelli, D. Pai, and B. Limaye, eds., Birkhäuser, Basel, 1986, pp. 135-155. Internat. Ser. Numer. Math., Vol. 76. Proceedings of the International Conference held at the Indian Institute of Technology, Bombay, December 16-20, 1985.
11. M. BARONTI AND P. L. PAPINI, Convergence of sequences of sets, in Methods of Functional Analysis in Approximation Theory, C. Micchelli, D. Pai, and B. Limaye, eds., Birkhäuser, Basel, 1986, pp. 135-155. Internat. Ser. Numer. Math., Vol. 76. Proceedings of the International Conference held at the Indian Institute of Technology, Bombay, December 16-20, 1985.
12. H. H. BAUSCHKE, The approximation of fixed points of compositions of nonexpansive mappings in Hilbert space, J. Math. Anal. Appl., 1996, to appear.
13. _, A norm convergence result on random products of relaxed projections in Hilbert space, Trans. Amer. Math. Soc., 347 (1995), pp. 1365-1373.
14. H. H. BAUSCHKE AND J. M. BORWEIN, On the convergence of von Neumann's alternating projection algorithm for two sets, Set-Valued Anal., 1 (1993), pp. 185-212.
15. H. H. BAUSCHKE, J. M. BORWEIN, AND A. S. LEWIS, The method of cyclic projections for closed convex sets in Hilbert space, Proc. of the Special Session on Optimization and Nonlinear Analysis, Y. Censor and S. Reich, eds., Jerusalem, 1995. Contemp. Math., to appear.
16. J. M. BORWEIN, S. REICH, AND I. SHAFRIR, Krasnoselski-Mann iterations in normed spaces, Canad. Math. Bull., 35 (1992), pp. 21-28.
17. L. M. BREGMAN, The method of successive projection for finding a common point of convex sets, Soviet Math. Dokl., 6 (1965), pp. 688-692.
18. F. E. BROWDER, Convergence theorems for sequences of nonlinear operators in Banach spaces, Math. Z., 100 (1967), pp. 201-225.
19. R. E. BRUCK, Random products of contractions in metric and Banach spaces, J. Math. Anal. Appl., 88 (1982), pp. 319-332.
20. R. E. BRUCK AND S. REICH, Nonexpansive projections and resolvents of accretive operators in Banach spaces, Houston J. Math., 3 (1977), pp. 459-470.
21. D. BUTNARIU AND Y. CENSOR, On the behavior of a block-iterative projection method for solving convex feasibility problems, J. Comput. Math., 32 (1990), pp. 79-94.
22. Y. CENSOR, Row-action methods for huge and sparse systems and their applications, SIAM Rev., 23 (1981), pp. 444-466.
23. _, Iterative methods for the convex feasibility problem, in Convexity and Graph Theory, M. Rosenfeld and J. Zaks, eds., North-Holland, Amsterdam, 1984, pp. 83-91. Proceedings of the Conference on Convexity and Graph Theory, Israel, March 1981. Ann. Discrete Math., Vol. 20.
24. _, Iterative methods for the convex feasibility problem, in Convexity and Graph Theory, M. Rosenfeld and J. Zaks, eds., North-Holland, Amsterdam, 1984, pp. 83-91. Proceedings of the Conference on Convexity and Graph Theory, Israel, March 1981. Ann. Discrete Math., Vol. 20.
25. _, Parallel application of block-iterative methods in medical imaging and radiation therapy, Math. Programming, 42 (1988), pp. 307-325.
26. _, On variable block algebraic reconstruction techniques, in Mathematical Methods in Tomography, G. Herman, A. Louis, and F. Natterer, eds., Springer, New York, 1990, pp. 133-140. Proceedings of a Conference held in Oberwolfach, Germany, June 11-15. Lecture Notes in Math., Vol. 1497.
27. _, On variable block algebraic reconstruction techniques, in Mathematical Methods in Tomography, G. Herman, A. Louis, and F. Natterer, eds., Springer, New York, 1990, pp. 133-140. Proceedings of a Conference held in Oberwolfach, Germany, June 11-15. Lecture Notes in Math., Vol. 1497.
28. Y. CENSOR, M. D. ALTSCHULER, AND W. D. POWLIS, On the use of Cimmino's simultaneous projections method for computing a solution of the inverse problem in radiation therapy treatment planning, Inverse Problems, 4 (1988), pp. 607-623.
29. Y. CENSOR AND T. ELFVING, New methods for linear inequalities. Linear Algebra Appl., 42 (1982), pp. 199-211.
30. Y. CENSOR AND G. T. HERMAN, On some optimization techniques in image reconstruction from projections, Appl. Numer. Math., 3 (1987), pp. 365-391.
31. Y. CENSOR AND A. LENT, Cyclic subgradient projections, Math. Programming, 24 (1982), pp. 233-235.
32. G. CIMMINO, Calcolo approssimate per le soluzioni dei sistemi di equazioni lineari, La Ricerca scientifica ed il Progresso tecnico nell' Economia nazionale (Roma), 9 (1938), pp. 326-333. Consiglio Nazionale delle Ricerche. Ministero dell' Educazione nazionale.
33. P. L. COMBETTES, The foundations of set theoretic estimation, Proc. IEEE, 81 (1993), pp. 182-208.
34. _, Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections, IEEE Trans. on Image Processing, 1996, to appear.
35. _, Inconsistent signal feasibility problems: Least-squares solutions in a product space, IEEE Trans. Signal Processing, 42 (1994), pp. 2955-2966.
36. _, Construction d'un point fixe commun à une famille de contractions fermes, C. R. Acad. Sci. Paris Sér. I Math., 320 (1995), pp. 1385-1390.
37. _, The Convex Feasibility Problem in Image Recovery, Vol. 95 of Advances in Imaging and Electron Physics, Academic Press, New York, 1995.
38. _, Hilbertian convex feasibility problem: Convergence of projection methods, Appl. Math. Optim., 1996, to appear.
39. P. L. COMBETTES AND H. PUH, Extrapolated projection method for the euclidean convex feasibility problem, Tech. report, City University of New York, 1993.
40. _, Iterations of parallel convex projections in Hilbert space, Numer. Funct. Anal. Optim., 15 (1994), pp. 225-243.
41. G. CROMBEZ, Image recovery by convex combinations of projections, J. Math. Anal. Appl., 155 (1991), pp. 413-419.
42. A. R. DE PIERRO AND A. N. IUSEM, A simultaneous projections method for linear inequalities, Linear Algebra Appl., 64 (1985), pp. 243-253.
43. _, A finitely convergent "row-action" method for the convex feasibility problem, Appl. Math. Optim., 17 (1988), pp. 225-235.
44. _, On the asymptotic behaviour of some alternate smoothing series expansion iterative methods, Linear Algebra Appl., 130 (1990), pp. 3-24.
45. F. DEUTSCH, Rate of convergence of the method of alternating projections, in Parametric Optimization and Approximation, B. Brosowski and F. Deutsch, eds., Birkhäuser, Basel, 1983, pp. 96-107. Internat. Ser. Numer. Math., Vol. 76.
46. _, The angle between subspaces of a Hilbert space, in Approximation Theory, Wavelets and Applications, S. P. Singh, ed., Kluwer Academic, 1995, pp. 107-130.
47. _, The method of alternating orthogonal projections, in Approximation Theory, Spline Functions and Applications, S. Singh, ed., Kluwer Academic, 1992, pp. 105-121. Proceedings of a Conference held in the Hotel villa del Mare, Maratea, Italy between April 28, 1991 and May 9, 1991.
48. _, The method of alternating orthogonal projections, in Approximation Theory, Spline Functions and Applications, S. Singh, ed., Kluwer Academic, 1992, pp. 105-121. Proceedings of a Conference held in the Hotel villa del Mare, Maratea, Italy between April 28, 1991 and May 9, 1991.
49. F. DEUTSCH AND H. HUNDAL, The rate of convergence for the method of alternating projections II, J. Math. Anal. Appl., to appear.
50. B. DJAFARI ROUHANI, Asymptotic behaviour of almost nonexpansive sequences in a Hilbert space, J. Math. Anal. Appl., 151 (1990), pp. 226-235.
51. L. T. DOS SANTOS, A parallel subgradient method for the convex feasibility problem, J. Comput. Appl. Math., 18 (1987), pp. 307-320.
52. J. M. DYE AND S. REICH, Unrestricted iterations of nonexpansive mappings in Hilbert space, Nonlinear Anal., 18 (1992), pp. 199-207.
53. P. P. EGGERMONT, G. T. HERMAN, and A. LENT, Iterative algorithms for large partitioned linear systems, with applications to image reconstruction, Linear Algebra Appl., 40 (1981), pp. 37-67.
54. I. EKELAND AND R. TEMAM, Convex Analysis and Variational Problems, North-Holland, Amsterdam, 1976.
55. L. ELSNER, I. KOLTRACHT, AND M. NEUMANN, Convergence of sequential and asynchronous nonlinear paracontractions, Numer. Math., 62 (1992), pp. 305-319.
56. I. I. EREMIN, Fejér mappings and convex programming, Siberian Math. J., 10 (1969), pp. 762-772.
57. S. D. FLÅM AND J. ZOWE, Relaxed outer projections, weighted averages and convex feasibility, BIT, 30 (1990), pp. 289-300.
58. K. FRIEDRICHS, On certain inequalities and characteristic value problems for analytic functions and for functions of two variables, Trans. Amer. Math. Soc., 41 (1937), pp. 321-364.
59. U. GARCÍA-PALOMARES, Parallel projected aggregation methods for solving the convex feasibility problem, SIAM J. Optim., 3 (1993), pp. 882-900.
60. A. GENEL AND J. LINDENSTRAUSS, An example concerning fixed points, Israel J. Math., 22 (1975), pp. 81-86.
61. K. GOEBEL AND W. A. KIRK, Iteration processes for nonexpansive mappings, in Topological Methods in Nonlinear Functional Analysis, S. Singh, S. Thomeier, and B. Watson, eds., American Mathematical Society, 1983, pp. 115-123. Contemp. Math., Vol. 21.
62. _, Topics in Metric Fixed Point Theory, Cambridge University Press, 1990. Cambridge Stud. Adv. Math., Vol. 28.
63. R. GORDON, R. BENDER, AND G. T. HERMAN, Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography, J. Theoret. Biol., 29 (1970), pp. 471-481.
64. L. G. GUBIN, B. T. POLYAK, AND E. V. RAIK, The method of projections for finding the common point of convex sets, Comput. Math. Math. Phys., 7(6) (1967), pp. 1-24.
65. I. HALPERIN, The product of projection operators, Acta Sci. Math. (Szeged), 23 (1962), pp. 96-99.
66. B. HALPERN, Fixed points of nonexpanding maps, Bull. Amer. Math. Soc., 73 (1967), pp. 957-961.
67. G. T. HERMAN, Image Reconstruction from Projections, Academic Press, New York, 1980.
68. G. T. HERMAN, A. LENT, AND P. H. LUTZ, Relaxation methods for image reconstruction, Comm. ACM, 21 (1978), pp. 152-158.
69. G. T. HERMAN, H. LEVKOWITZ, H. K. TUY, AND S. MCCORMICK, Multilevel image reconstruction, in Multiresolution Image Processing and Analysis, A. Rosenfeld, ed., Springer, New York, 1984, pp. 121-135. Based on a workshop held in Leesburg, VA, July 19-21, 1982. Springer Ser. Inform. Sci., Vol. 12.
70. G. T. HERMAN, H. LEVKOWITZ, H. K. TUY, AND S. MCCORMICK, Multilevel image reconstruction, in Multiresolution Image Processing and Analysis, A. Rosenfeld, ed., Springer, New York, 1984, pp. 121-135. Based on a workshop held in Leesburg, VA, July 19-21, 1982. Springer Ser. Inform. Sci., Vol. 12.
71. A. J. HOFFMAN, On approximate solutions of systems of linear inequalities, J. Research of the National Bureau of Standards, 49 (1952), pp. 263-265.
72. R. B. HOLMES, Geometric Functional Analysis and its Applications, Springer-Verlag, New York, 1975.
73. A. D. IOFFE AND V. M. TIHOMIROV, Theory of Extremal Problems, North-Holland, Amsterdam, 1979. Stud. Math. Appl., Vol. 6.
74. V. I. ISTRATESCU, Fixed Point Theory, D. Reidel, Boston, MA, 1981.
75. G. J. JAMESON, Topology and Normed Spaces, Chapman and Hall, London, UK, 1974.
76. S. KACZMARZ, Angenäherte Auflösung von Systemen linearer Gleichungen, Bulletin internationel de l'Académie Polonaise des Sciences et des Lettres. Classe des Sciences mathématiques et naturelles. Séries A: Sciences mathématiques, (1937), pp. 355-357. Cracovie, Imprimerie de l'Université.
77. K. C. KIWIEL, Block-iterative surrogate projection methods for convex feasibility problems, Linear Algebra Appl., 215 (1995), pp. 225-260.
78. _, The efficiency of subgradient projection methods for convex optimization, part I: General level methods, SIAM J. Control Optim., 34 (1996), pp. 660-676.
79. _, The efficiency of subgradient projection methods for convex optimization, part II: Implementations and extensions, SIAM J. Control Optim., 34 (1996), pp. 677-697.
80. K. C. KIWIEL AND B. LOPUCH, Surrogate projection methods for finding fixed points of firmly nonexpansive mappings, SIAM J. Optim., to appear.
81. E. KLEIN AND A. C. THOMPSON, Theory of Correspondences, Wiley-Interscience, New York, 1984.
82. P.-L. LIONS, Approximation de points fixes de contractions, Comptes Rendus des Séances de l'Académie des Sciences, Séries A-B, Paris, 284 (1977), pp. 1357-1359.
83. J. MANDEL, Convergence of the cyclic relaxation method for linear inequalities, Math. Programming, 30 (1984), pp. 218-228.
84. S. MARUSTER, The solution by iteration of nonlinear equations in Hilbert spaces, Proc. Amer. Math. Soc., 63 (1977), pp. 69-73.
85. Y. MERZLYAKOV, On a relaxation method of solving systems of linear inequalities, Comput. Math. Math. Phys., 2 (1963), pp. 504-510.
86. J. J. MOREAU, Un cas des convergence des iterées d'une contraction d'un éspace hilbertien, Comptes Rendus des Séances de l'Académie des Sciences, Séries A-B, Paris, 286 (1978), pp. 143-144. (In French.)
87. U. MOSCO, Convergence of convex sets and of solutions of variational inequalities, Adv. Math., 3 (1969), pp. 510-585.
88. T. S. MOTZKIN AND I. J. SCHOENBERG, The relaxation method for linear inequalities, Canad. J. Math., 6 (1954), pp. 393-404.
89. Z. OPIAL, Weak convergence of the sequence of successive approximations for nonexpansive mappings, Bull. Amer. Math. Soc., 73 (1967), pp. 591-597.
90. G. PIERRA, Decomposition through formalization in a product space, Math. Programming, 28 (1984), pp. 96-115.
91. B. T. POLYAK, Minimization of unsmooth functional, Comput. Math. Math. Phys., 9 (1969), pp. 14-29.
92. H. RÅDSTRÖM, An embedding theorem for spaces of convex sets, Proc. Amer. Math. Soc., 3 (1952), pp. 165-169.
93. A. W. ROBERTS AND D. E. VARBERG, Convex Functions, Academic Press, New York, 1973.
94. D. SCHOTT, A general iterative scheme with applications to convex optimization and related fields, Optimizaton, 22 (1991), pp. 885-902.
95. H. F. SENTER AND W. G. DOTSON, JR., Approximating fixed points of nonexpansive mappings, Proc. Amer. Math. Soc., 44 (1974), pp. 375-380.
96. M. I. SEZAN, An overview of convex projections theory and its applications to image recovery problems, Ultramicroscopy, 40 (1992), pp. 55-67.
97. N. Z. SHOR, Minimization Methods for Non-Differentiable Functions, Springer-Verlag, Berlin, 1985. Springer Ser. Comput. Math., Vol. 3.
98. A. SIMONIČ, personal communication.
99. K. T. SMITH, D. C. SOLMON, AND S. L. WAGNER, Practical and mathematical aspects of the problem of reconstructing objects from radiographs, Bull. Amer. Math. Soc., 83 (1977), pp. 1227-1270.
100. H. STARK, ED., Image Recovery: Theory and Application, Academic Press, Orlando, FL, 1987.
101. K. TANABE, Projection method for solving a linear system of linear equations and its applications, Numer. Math., 17 (1971), pp. 203-214.
102. M. R. TRUMMER, Reconstructing pictures from projections: On the convergence of the ART algorithm with relaxation, Computing, 26 (1981), pp. 189-195.
103. _, SMART - an algorithm for reconstructing pictures from projections, Z. Angew. Math. Phys., 34 (1983), pp. 746-753.
104. _, A note on the ART of relaxation, Computing, 33 (1984), pp. 349-352.
105. P. TSENG, On the convergence of the products of firmly nonexpansive mappings, SIAM J. Optim., 2 (1992), pp. 425-434.
106. M. TSUKADA, Convergence of best approximations in a smooth Banach space, J. Approx. Theory, 40 (1984), pp. 301-309.
107. M. A. VIERGEVER, Introduction to discrete reconstruction methods in medical imaging, in Mathematics and Computer Science in Medical Imaging, M. Viergever and A. Todd-Pokropek, eds., Berlin, 1988, Springer-Verlag, pp. 43-65. NATO Advanced Science Institute Series F: Computer and Systems Sciences, Vol. 39. Proceedings, held in Il Ciocco, Italy, September 21-October 4, 1986.
108. M. A. VIERGEVER, Introduction to discrete reconstruction methods in medical imaging, in Mathematics and Computer Science in Medical Imaging, M. Viergever and A. Todd-Pokropek, eds., Berlin, 1988, Springer-Verlag, pp. 43-65. NATO Advanced Science Institute Series F: Computer and Systems Sciences, Vol. 39. Proceedings, held in Il Ciocco, Italy, September 21-October 4, 1986.
109. J. VON NEUMANN, Functional Operators, Vol. II. The Geometry of Orthogonal Spaces, Princeton University Press, Princeton, NJ, 1950. Ann. Math. Stud., Vol. 22. Reprint of mimeographed lecture notes first distributed in 1933.
110. J. VON NEUMANN, Functional Operators, Vol. II. The Geometry of Orthogonal Spaces, Princeton University Press, Princeton, NJ, 1950. Ann. Math. Stud., Vol. 22. Reprint of mimeographed lecture notes first distributed in 1933.
111. R. WITTMANN, Approximation of fixed points of nonexpansive mappings, Arch. Math. (Basel), 58 (1992), pp. 486-491.
112. K. YANG AND K. G. MURTY, New iterative methods for linear inequalities, J. Optim. Theory Appl., 72 (1992), pp. 163-185.
113. D. C. YOULA, Mathematical theory of image restoration by the method of convex projections, in Image Recovery: Theory and Applications, H. Stark, ed., Academic Press, Orlando, FL, 1987, pp. 29-77.
114. _, On deterministic convergence of iterations of relaxed projection operators, J. Vis. Commun. Image Rep., 1 (1990), pp. 12-20.
115. D. C. YOULA AND H. WEBB, Image reconstruction by the method of convex projections: Part 1 - theory, IEEE Trans. Medical Imaging, MI-1 (1982), pp. 81-94.
116. E. H. ZARANTONELLO, Projections on convex sets in Hilbert space and spectral theory, in Contributions to Nonlinear Functional Analysis, E. Zarantonello, ed., Academic Press, New York, 1971, pp. 237-424. University of Wisconsin. Mathematics Research Center. Publication No. 27.