Spectral projected gradient methods: Review and perspectives

Journal of Statistical Software

Volumn 60, Issue 3, 2014, Pages 1-21

  Birgin, Ernesto G ^a   Martínez, José Mario ^b   Raydan, Marcos ^c  

^a UNIVERSITY OF SÃO PAULO   (Brazil)

^b UNIVERSITY OF CAMPINAS   (Brazil)

^c UNIVERSIDAD SIMÓN BOLÍVAR   (Venezuela)

Author keywords

Convex constrained problems; Large scale problems; Nonmonotone line search; Spectral projected gradient methods

Indexed keywords

EID: 84907533558     PISSN: 15487660     EISSN: None     Source Type: Journal    
DOI: 10.18637/jss.v060.i03     Document Type: Article

References (107)

1. Andrade R, Birgin EG, Chambouleyron I, Martínez JM, Ventura SD (2008). "Estimation of the Thickness and the Optical Parameters of Several Superimposed Thin Films Using Optimization". Applied Optics, 47, 5208-5220.
2. Andreani R, Birgin EG, Martínez JM, Schuverdt ML (2007). "On Augmented Lagrangian Methods with General Lower-Level Constraints". SIAM Journal on Optimization, 18, 1286-1309.
3. Andreani R, Birgin EG, Martínez JM, Schuverdt ML (2008). "Augmented Lagrangian Methods under the Constant Positive Linear Dependence Constraint Qualification". Mathemat-ical Programming, 111, 5-32.
4. Andreani R, Birgin EG, Martínez JM, Schuverdt ML (2010). "Second-Order Negative-Curvature Methods for Box-Constrained and General Constrained Optimization". Compu-tational Optimization and Applications, 45, 209-236.
5. Andreani R, Birgin EG, Martínez JM, Yuan JY (2005). "Spectral Projected Gradient and Variable Metric Methods for Optimization with Linear Inequalities". IMA Journal of Nu-merical Analysis, 25, 221-252.
6. Andretta M, Birgin EG, Martínez JM (2005). "Practical Active-Set Euclidian Trust-Region Method with Spectral Projected Gradients for Bound-Constrained Minimization". Opti-mization, 54, 305-325.
7. Andretta M, Birgin EG, Martínez JM (2010). "Partial Spectral Projected Gradient Method with Active-Set Strategy for Linearly Constrained Optimization". Numerical Algorithms, 53(1), 23-52.
8. Azofeifa D, Clark N, Vargas W (2005). "Optical and Electrical Properties of Terbium Films as a Function of Hydrogen Concentration". Physica Status Solidi B, 242, 2005-2009.
9. Barzilai J, Borwein JM (1988). "Two Point Step Size Gradient Methods". IMA Journal of Numerical Analysis, 8, 141-148.
10. Bello L, Raydan M (2007). "Convex Constrained Optimization for the Seismic Reection Tomography Problem". Journal of Applied Geophysics, 62, 158-166.
11. Benvenuto F, Zanella R, Zanni L, Bertero M (2010). "Nonnegative Least-Squares Image Deblurring: Improved Gradient Projection Approaches". Inverse Problems, 26, 025004.
12. Berlinet AF, Roland C (2011). "Geometric Interpretation of Some Cauchy Related Methods". Numerische Mathematik, 119, 437-464.
13. Bernaards CA, Jennrich RI (2005). "Gradient Projection Algorithms and Software for Arbitrary Rotation Criteria in Factor Analysis". Educational and Psychological Measurement, 65, 676-696.
14. Bertsekas DP (1976). "On the Goldstein-Levitin-Polyak Gradient Projection Method". IEEE Transactions on Automatic Control, 21, 174-184.
15. Bertsekas DP (1999). Nonlinear Programming. Athena Scientific, Boston.
16. Birgin EG, Biloti R, Tygel M, Santos LT (1999a). "Restricted Optimization: a Clue to a Fast and Accurate Implementation of the Common Reection Surface Stack Method". Journal of Applied Geophysics, 42, 143-155.
17. Birgin EG, Chambouleyron I, Martínez JM (1999b). "Estimation of the Optical Constants and the Thickness of Thin Films Using Unconstrained Optimization". Journal of Computational Physics, 151, 862-880.
18. Birgin EG, Chambouleyron I, Martínez JM, Ventura SD (2003a). "Estimation of Optical Parameters of Very Thin Films". Applied Numerical Mathematics, 47, 109-119.
19. Birgin EG, Evtushenko YG (1998). "Automatic Differentiation and Spectral Projected Gradient Methods for Optimal Control Problems". Optimization Methods & Software, 10, 125-146.
20. Birgin EG, Gentil JM (2012). "Evaluating Bound-Constrained Minimization Software". Com-putational Optimization and Applications, 53, 347-373.
21. Birgin EG, Martínez JM (2001). "A Box-Constrained Optimization Algorithm with Negative Curvature Directions and Spectral Projected Gradients". In G Alefeld, X Chen (eds.), Topics in Numerical Analysis-With Special Emphasis on Nonlinear Problems, volume 15 of Computing Supplementa, pp. 49-60. Springer-Verlag.
22. Birgin EG, Martínez JM (2002). "Large-Scale Active-Set Box-Constrained Optimization Method with Spectral Projected Gradients". Computational Optimization and Applica-tions, 23, 101-125.
23. Birgin EG, Martínez JM (2008). "Structured Minimal-Memory Inexact Quasi-Newton Method and Secant Preconditioners for Augmented Lagrangian Optimization". Computational Op-timization and Applications, 39, 1-16.
24. Birgin EG, Martínez JM, Martínez L, Rocha GB (2013). "Sparse Projected-Gradient Method as a Linear-Scaling Low-Memory Alternative to Diagonalization in Self-Consistent Field Electronic Structure Calculations". Journal of Chemical Theory and Computation, 9, 1043-1051.
25. Birgin EG, Martínez JM, Raydan M (2000). "Nonmonotone Spectral Projected Gradient Methods on Convex Sets". SIAM Journal on Optimization, 10, 1196-1211.
26. Birgin EG, Martínez JM, Raydan M (2001). "Algorithm 813: SPG-Software for Convex-Constrained Optimization". ACM Transactions on Mathematical Software, 27, 340-349.
27. Birgin EG, Martínez JM, Raydan M (2003b). "Inexact Spectral Projected Gradient Methods on Convex Sets". IMA Journal of Numerical Analysis, 23, 539-559.
28. Bonettini S, Zanella R, Zanni L (2009)". A Scaled Gradient ProjectionMethod for Constrained Image Deblurring". Inverse Problems, 25, 015002.
29. Borsdorf R, Higham NJ, Raydan M (2010). "Computing a Nearest Correlation Matrix with Factor Structure". SIAM Journal on Matrix Analysis and Applications, 31, 2603-2622.
30. Bouhamidi A, Jbilou K, Raydan M (2011). "Convex Constrained Optimization for Large-Scale Generalized Sylvester Equations". Computational Optimization and Applications, 48, 233-253.
31. Boyd S, Xiao L (2005). "Least-Squares Covariance Matrix Adjustment". SIAM Journal on Matrix Analysis and Applications, 27, 532-546.
32. Brás CP, Hager WW, Júdice JJ (2012). "An Investigation of Feasible Descent Algorithms for Estimating the Condition Number of a Matrix". TOP-Journal of the Spanish Society of Statistics and Operations Research, 20, 791-809.
33. Cauchy A (1847). "Méthode Génfferale pour la Résolution des Systémes D'équations Simultan ées". Compte-Rendu de l'Académie des Sciences, 27, 536-538.
34. Chambouleyron I, Ventura SD, Birgin EG, Martínez JM (2009). "Optical Constants and Thickness Determination of Very Thin Amorphous Semiconductor Films". Journal of Ap-plied Physics, 92, 3093-3102.
35. Cores D, Escalante R, Gonzalez-Lima M, Jiménez O (2009). "On the Use of the Spectral Projected Gradient Method for Support Vector Machines". Computational and Applied Mathematics, 28, 327-364.
36. Cores D, Loreto M (2007). "A Generalized Two-Point Ellipsoidal Anisotropic Ray Tracing for Converted Waves". Optimization and Engineering, 8, 373-396.
37. Crema A, Loreto M, Raydan M (2007). "Spectral Projected Subgradient with a Momentum Term for the Lagrangean Dual Approach". Computers and Operations Research, 34, 3174-3186.
38. Curiel F, Vargas WE, Barrera RG (2002). "Visible Spectral Dependence of the Scattering and Absorption Coeficients of Pigmented Coatings from Inversion of Diúse Reectance Spectra". Applied Optics, 41, 5969-5978.
39. Dai YH, Fletcher R (2005). "On the Asymptotic Behaviour of Some New Gradient Methods". Mathematical Programming, 103, 541-559.
40. Dai YH, Fletcher R (2006). "New Algorithms for Singly Linearly Constrained Quadratic Programs Subject to Lower and Upper Bounds". Mathematical Programming, 106, 403-421.
41. Dai YH, Liao LZ (2002). "R-Linear Convergence of the Barzilai and Borwein Gradient Method". IMA Journal on Numerical Analysis, 22, 1-10.
42. Dai YH, Zhang HC (2001). "Adaptive Two-Point Stepsize Gradient Algorithm". Numerical Algorithms, 27, 377-385.
43. De Asmundis R, Di Serańo D, Riccio F, Toraldo G (2013). "On Spectral Properties of Steepest Descent Methods". IMA Journal of Numerical Analysis, 33(4), 1416-1435.
44. Deidda GP, Bonomi E, Manzi C (2003). "Inversion of Electrical Conductivity Data With Tikhonov Regularization Approach: Some Considerations". Annals of Geophysics, 46, 549-558.
45. Dennis JE, Moré JJ (1977). "Quasi-Newton Methods, Motivation and Theory". SIAM Review, 19, 46-89.
46. Dennis JE, Schnabel RB (1983). Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall.
47. Diniz-Ehrhardt MA, Gomes-Ruggiero MA, Martínez JM, Santos SA (2004). "Augmented Lagrangian Algorithms Based on the Spectral Projected Gradient Method for Solving Nonlinear Programming Problems". Journal of Optimization Theory and Applications, 123, 497-517.
48. Escalante R, Raydan M (1996). "Dykstra's Algorithm for a Constrained Least-Squares Matrix Problem". Numerical Linear Algebra and Applications, 3, 459-471.
49. Figueiredo MA, Nowak RD, Wright SJ (2007). "Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems". IEEE Journal of Selected Topics in Signal Processing, 1, 586-597.
50. Fletcher R (1985). "Semi-Definite Matrix Constraints in Optimization". SIAM Journal on Control and Optimization, 23, 493-513.
51. Fletcher R (1990). "Low Storage Methods for Unconstrained Optimization". Lectures in Applied Mathematics, 26, 165-179.
52. Fletcher R (2005). "On the Barzilai-Borwein Method". In L Qi, K Teo, X Yang (eds.), Optimization and Control with Applications, Series in Applied Optimization, Volume 96, pp. 235-256. Kluwer.
53. Francisco JB, Martínez JM, Martínez L (2006). "Density-Based Globally Convergent Trust-Region Methods for Self-Consistent Field Electronic Structure Calculations". Journal of Mathematical Chemistry, 40, 349-377.
54. Friedlander A, Martínez JM, Molina B, Raydan M (1999). "Gradient Method with Retards and Generalizations". SIAM Journal on Numerical Analysis, 36, 275-289.
55. Glunt W, Hayden TL, Raydan M (1993). "Molecular Conformations from Distance Matrices". Journal of Computational Chemistry, 14, 114-120.
56. Goldfarb D, Idnani A (1983). "A Numerically Stable Dual Method for Solving Strictly Convex Quadratic Programs". Mathematical Programming, 27, 1-33.
57. Goldstein AA (1964). "Convex Programming in Hilbert Space". Bulletin of the American Mathematical Society, 70, 709-710.
58. Gomes-Ruggiero MA, Martínez JM, Santos SA (2009). "Spectral Projected Gradient Method with Inexact Restoration for Minimization with Nonconvex Constraints". SIAM Journal on Scientific Computing, 31, 1628-1652.
59. Gould NIM, Orban D, Toint PL (2003). "CUTEr and SifDec: A Constrained and Unconstrained Testing Environment, Revisited". ACM Transactions on Mathematical Software, 29, 373-394.
60. Grippo L, Lampariello F, Lucidi S (1986). "A Nonmonotone Line Search Technique for Newton's Method". SIAM Journal on Numerical Analysis, 23, 707-716.
61. Grippo L, Lampariello F, Lucidi S (1991). "A Class of Nonmonotone Stabilization Methods in Unconstrained Optimization". Numerische Mathematik, 59, 779-805.
62. Grippo L, Sciandrone M (2002). "Nonmonotone Globalization Techniques for the Barzilai-Borwein Gradient Method". Computational Optimization and Application, 23, 143-169.
63. Grippo L, Sciandrone M (2007). "Nonmonotone Derivative-Free Methods for Nonlinear Equations". Computational Optimization and Application, 37, 297-328.
64. Guerrero J, Raydan M, Rojas M (2013). "A Hybrid-Optimization Method for Large-Scale Non-Negative Full Regularization in Image Restoration". Inverse Problems in Science & Engineering, 21(5), 741-766.
65. Hestenes MR, Stiefel E (1952). "Methods of Conjugate Gradients for Solving Linear Systems". Journal of Research of the National Bureau of Standards, 49, 409-436.
66. Higham NJ (1988). "Computing a Nearest Symmetric Positive Semidefinite Matrix". Linear Algebra and Its Applications, 103, 103-118.
67. Higham NJ (2002). "Computing the Nearest Correlation Matrix-A Problem from Finance". IMA Journal of Numerical Analysis, 22, 329-343.
68. Hu H, Olkin I (1991). "A Numerical Procedure for Finding the Positive Definite Matrix Closest to a Patterned Matrix". Statistics and Probability letters, 12, 511-515.
69. Jiang Z (2006). "Applications of Conditional Nonlinear Optimal Perturbation to the Study of the Stability and Sensitivity of the Jovian Atmosphere". Advances in Atmospheric Sciences, 23, 775-783.
70. Júdice JJ, Raydan M, Rosa SS, Santos SA (2008). "On the Solution of the Symmetric Eigenvalue Complementarity Problem by the Spectral Projected Gradient Algorithm". Numerical Algorithms, 47, 391-407.
71. Kawalec A, Magdziak M (2012). "Usability Assessment of Selected Methods of Optimization for Some Measurement Task in Coordinate Measurement Technique". Measurement, 45, 2330-2338.
72. Kolehmainen V, Vanne A, Siltanen SSJ, Kaipio JP, Lassas M, Kalke M (2006). "Parallelized Bayesian Inversion for Three-Dimensional Dental X-Ray Imaging". IEEE Transactions on Medical Imaging, 25, 218-228.
73. Kolehmainen V, Vanne A, Siltanen SSJ, Kaipio JP, Lassas M, Kalke M (2007). "Bayesian Inversion Method for 3D Dental X-Ray Imaging". e & i Elektrotechnik und Information-stechnik, 124, 248-253.
74. La Cruz W, Martínez JM, Raydan M (2006). "Spectral Residual Method without Gradient Information for Solving Large-Scale Nonlinear Systems of Equations". Mathematics of Computation, 75, 1429-1448.
75. La Cruz W, Raydan M (2003). "Nonmonotone Spectral Methods for Large-Scale Nonlinear Systems". Optimization Methods and Software, 18, 583-599.
76. Levitin ES, Polyak BT (1966). "Constrained Minimization Problems". USSR Computational Mathematics and Mathematical Physics, 6, 1-50.
77. Li X, Liu H, Zheng X (2012). "Non-Monotone Projection Gradient Method for Non-Negative Matrix Factorization". Computational Optimization and Applications, 51, 1163-1171.
78. Loris I, Bertero M, Mol CD, Zanella R, Zanni L (2009). "Accelerating Gradient Projection Methods for 1-Constrained Signal Recovery by Steplength Selection Rules". Applied and Computational Harmonic Analysis, 27, 247-254.
79. Maciel MC, Mendonfica MG, Verdiell AB (2013). "Monotone and Nonmonotone Trust-Region-Based Algorithms for Large Scale Unconstrained Optimization Problems". Computational Optimization and Applications, 54(1), 27-43.
80. Martínez JM, Pilotta EA, Raydan M (2005). "Spectral Gradient Methods for Linearly Constrained Optimization". Journal of Optimization Theory and Applications, 125, 629-651.
81. Molina B, Raydan M (1996). "Preconditioned Barzilai-Borwein Method for the Numerical Solution of Partial Differential Equations". Numerical Algorithms, 13, 45-60.
82. Mu M, Duan WS, Wang B (2003). "Conditional Nonlinear Optimal Perturbation and Its Applications". Nonlinear Processes in Geophysics, 10, 493-501.
83. Mulato M, Chambouleyron I, Birgin EG, Martínez JM (2000). "Determination of Thickness and Optical Constants of a-Si: H Films from Transmittance Data". Applied Physics Letters, 77, 2133-2135.
84. Murphy AB (2007). "Optical Properties of an Optically Rough Coating from Inversion of Diúse Reectance Measurements". Applied Optics, 46, 3133-3143.
85. Ramirez-Porras A, Vargas-Castro WE (2004). "Transmission of Visible Light through Oxidized Copper Films: Feasibility of Using a Spectral Projected Gradient Method". Applied Optics, 43, 1508-1514.
86. Raydan M (1993). "On the Barzilai and Borwein Choice of Steplength for the Gradient Method". IMA Journal of Numerical Analysis, 13, 321-326.
87. Raydan M (1997). "The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem". SIAM Journal on Optimization, 7, 26-33.
88. Raydan M, Svaiter BF (2002). "Relaxed Steepest Descent and Cauchy-Barzilai-Borwein Method". Computational Optimization and Applications, 21, 155-167.
89. R Core Team (2014). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www. R-project. org/.
90. Roland C, Varadhan R (2005). "New Iterative Schemes for Nonlinear Fixed Point Problems, with Applications to Problems with Bifurcations and Incomplete-Data Problems". Applied Numerical Mathematics, 55, 215-226.
91. Roland C, Varadhan R, Frangakis CE (2007). "Squared Polynomial Extrapolation Methods with Cycling: An Application to the Positron Emission Tomography Problem". Numerical Algorithms, 44, 159-172.
92. Serafini T, Zanghirati G, Zanni L (2005). "Gradient Projection Methods for Quadratic Programs and Applications in Training Support Vector Machines". Optimization Methods and Software, 20, 347-372.
93. Tavakoli R, Zhang H (2012). "A Nonmonotone Spectral Projected Gradient Method for Large-Scale Topology Optimization Problems". Numerical Algebra, Control and Optimization, 2, 395-412.
94. Toint PL (1996). "An Assessment of Nonmonotone Linesearch Techniques for Unconstrained Optimization". SIAM Journal on Scientific Computing, 17, 725-739.
95. van den Berg E, Friedlander MP (2008). "Probing the Pareto frontier for basis pursuit solutions". SIAM Journal on Scientific Computing, 31, 890-912.
96. van den Berg E, Friedlander MP (2011). "Sparse Optimization with Least-Squares Constraints". SIAM Journal on Optimization, 21, 1201-1229.
97. van den Doel K, Ascher U (2012). "The Chaotic Nature of Faster Gradient Descent Methods". Journal of Scientific Computing, 51, 560-581.
98. Varadhan R, Gilbert P (2009). "BB: An R Package for Solving a Large System of Nonlinear Equations and for Optimizing a High-Dimensional Nonlinear Objective Function". Journal of Statistical Software, 32(4), 1-26. URL http://www. jstatsoft. org/v32/i04/.
99. Varadhan R, Roland C (2008). "Simple, Stable, and General Methods for Accelerating Any EM Algorithm". Scandinavian Journal of Statistics, 35, 335-353.
100. Vargas WE (2002). "Inversion Methods from Kiabelka-Munk Analysis". Journal of Optics A, 4, 452-456.
101. Vargas WE, Azofeifa DE, Clark N (2003). "Retrieved Optical Properties of Thin Films on Absorbing Substrates from Transmittance Measurements by Application of a Spectral Projected Gradient Method". Thin Solid Films, 425, 1-8.
102. Ventura SD, Birgin EG, Martínez JM, Chambouleyron I (2005). "Optimization Techniques for the Estimation of the Thickness and the Optical Parameters of Thin Films Using Re-ectance Data". Journal of Applied Physics, 97, 043512.
103. Weingessel A (2013). "quadprog: Functions to Solve Quadratic Programming Problems". R package version 1. 5-5, URL http://CRAN. R-project. org/package=quadprog.
104. Yuan Q (2012). "Matrix Linear Variational Inequality Approach for Finite Element Model Updating". Mechanical Systems and Signal Processing, 28, 507-516.
105. Zeev N, Savasta O, Cores D (2006). "Non-Monotone Spectral Projected Gradient Method Applied to Full Waveform Inversion". Geophysical Prospecting, 54, 525-534.
106. Zhang H, Hager WW (2004). "A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization". SIAM Journal on Optimization, 14, 1043-1056.
107. Zhang L, Zhou W (2006). "Spectral Gradient Projection Method for Solving Nonlinear Monotone Equations". Journal of Computational and Applied Mathematics, 196, 478-484.