Unconstrained optimization of real functions in complex variables

SIAM Journal on Optimization

Volumn 22, Issue 3, 2012, Pages 879-898

  Sorber, Laurent ^a   Van Barel, Marc ^a   De Lathauwer, Lieven ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

BFGS; Functions of complex variables; Gauss Newton; L BFGS; Levenberg Marquardt; Nonlinear conjugate gradient; Nonlinear least squares; Quasi Newton; Unconstrained optimization; Wirtinger calculus

Indexed keywords

BFGS; COMPLEX VARIABLE; GAUSS-NEWTON; L-BFGS; LEVENBERG-MARQUARDT; NON-LINEAR LEAST SQUARES; NONLINEAR CONJUGATE GRADIENT; QUASI-NEWTON; UNCONSTRAINED OPTIMIZATION; WIRTINGER CALCULUS;

ALGORITHMS; SIGNAL PROCESSING; TAYLOR SERIES;

OPTIMIZATION;

EID: 84867297041     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/110832124     Document Type: Article

References (51)

1. T. J. Abatzoglou, J. M. Mendel, and G. A. Harada, The constrained total least squares technique and its applications to harmonic superresolution, IEEE Trans. Signal Process., 39 (1991), pp. 1070-1087.
2. Z. Bai, D. Day, J.W. Demmel, J. J. Dongarra, M. Gu, A. Ruhe, and H. A. van der Vorst, Templates for linear algebra problems, in Computer Science Today, J. van Leeuwen, ed., Lecture Notes in Comput. Sci. 1000, Springer, Berlin, Heidelberg, 1995, pp. 115-140.
3. D. H. Brandwood, A complex gradient operator and its application in adaptive array theory, Proc. IEE-H, 130 (1983), pp. 11-16.
4. R. Bro, Multi-way Analysis in the Food Industry: Models, Algorithms, and Applications, Ph.D. thesis, University of Amsterdam, Amsterdam, The Netherlands, 1998.
5. C. G. Broyden, The convergence of a class of double-rank minimization algorithms. II. The new algorithm, J. Inst. Math. Appl., 6 (1970), pp. 222-231.
6. R. H. Byrd, R. B. Schnabel, and G. A. Schultz, Approximate solution of the trust region problem by minimization over two-dimensional subspaces, Math. Programming, 40 (1988), pp. 247-263.
7. J. D. Carroll and J.-J. Chang, Analysis of individual differences in multidimensional scaling via an n-way generalization of "Eckart-Young" decomposition, Psychometrika, 35 (1970), pp. 283-319.
8. A. R. Conn, N. I. M. Gould, and Ph. L. Toint, Trust-region methods, MPS-SIAM Ser. Optim., SIAM and MPS, Philadelphia, 2000.
9. J. E. Dennis, Jr., and H. H. W. Mei, Two new unconstrained optimization algorithms which use function and gradient values, J. Optim. Theory Appl., 28 (1979), pp. 453-482.
10. J. E. Dennis, Jr., and J. J. Moré, Quasi-Newton methods, motivation and theory, SIAM Rev., 19 (1977), pp. 46-89.
11. R. Fletcher, A new approach to variable metric algorithms, Comput. J., 13 (1970), pp. 317-322.
12. R. Fletcher and C. M. Reeves, Function minimization by conjugate gradients, Comput. J., 7 (1964), pp. 149-154.
13. J. C. Gilbert and C. Lemaréchal, Some numerical experiments with variable-storage quasi- Newton algorithms, Math. Programming, 45 (1989), pp. 407-435.
14. J. C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2 (1992), pp. 21-42.
15. D. Goldfarb, A family of variable-metric methods derived by variational means, Math.Comp., 24 (1970), pp. 23-26.
16. W. W. Hager and H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16 (2005), pp. 170-192.
17. W. W. Hager and H. Zhang, A survey of nonlinear conjugate gradient methods, Pac. J. Optim., 2 (2006), pp. 35-58.
18. R. A. Harshman, Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-modal factor analysis, UCLA Working Papers in Phonetics, 16 (1970), pp. 1-84.
19. M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Research Nat. Bur. Standards, 49 (1952), pp. 409-436.
20. F. L. Hitchcock, The expression of a tensor or a polyadic as a sum of products, J. Math. Phys., 6 (1927), pp. 164-189.
21. F. L. Hitchcock, Multiple invariants and generalized rank of a p-way matrix or tensor, J. Math. Phys., 7 (1927), pp. 39-79.
22. R. C. James, Advanced Calculus, Wadsworth, Belmont, CA, 1966.
23. T. Kailath and A. H. Sayed, eds., Fast Reliable Algorithms for Matrices with Structure, SIAM, Philadelphia, 1999.
24. C. G. Khatri and C. R. Rao, Solutions to some functional equations and their applications to characterization of probability distributions, Sankhya Indian J. of Stat., 30 (1968), pp. 167-180.
25. T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455-500.
26. K. Kreutz-Delgado, The complex gradient operator and the CR-calculus, ArXiv e-prints, arXiv:0906.4835v1, 2009.
27. D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Programming, 45 (1989), pp. 503-528.
28. J. J. Moré and D. J. Thuente, Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Software, 20 (1994), pp. 286-307.
29. L. W. Nagel, SPICE2: A Computer Program to Simulate Semiconductor Circuits, Memorandum ERL-M520, Electronics Research Lab, University of California, Berkeley, CA, 1975.
30. M. S. Nakhla and J. Vlach, A piecewise harmonic balance technique for determination of periodic response of nonlinear systems, IEEE Trans. Circuits and Systems, 23 (1976), pp. 85-91.
31. Z. Nehari, Introduction to Complex Analysis, Allyn & Bacon, Inc., Boston, 1961.
32. J. Nocedal, Updating quasi-Newton matrices with limited storage, Math. Comp., 35 (1980), pp. 773-782.
33. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed., Springer Ser. Oper. Res., Springer-Verlag, New York, 2006.
34. C. C. Paige and M. A. Saunders, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8 (1982), pp. 43-71.
35. E. Polak and G. Ribière, Note sur la convergence de methods de directions conjugres, Rev. Française Informat. Recherche Operationnelle, 3 (1969), pp. 35-43.
36. M. J. D. Powell, A hybrid method for nonlinear equations, in Numerical Methods for Nonlinear Algebraic Equations, P. Robinowitz, ed., Gordon and Breach, London, 1970, pp. 87-144.
37. M. J. D. Powell, A new algorithm for unconstrained optimization, in Nonlinear Programming, J. B. Rosen, O. L. Mangasarian, and K. Ritter, eds., Academic Press, New York, 1970, pp. 31-66.
38. M. J. D. Powell, Restart procedures for the conjugate gradient method, Math. Programming, 12 (1977), pp. 241-254.
39. M. J. D. Powell, Nonconvex Minimization Calculations and the Conjugate Gradient Method, Numerical Analysis, Lecture Notes in Math. 1066, Springer, Berlin, 1984.
40. T. L. Quarles, Analysis of Performance and Convergence Issues for Circuit Simulation, Memorandum ERL-M89/42, Electronics Research Lab, University of California, Berkeley, CA, 1989.
41. C. R. Rao and M. B. Rao, Matrix Algebra and Its Applications to Statistics and Econometrics, World Scientific, River Edge, NJ, 1998.
42. R. Remmert, Theory of Complex Functions, Springer-Verlag, New York, 1991.
43. V. Rizzoli, F. Mastri, C. Cecchetti, and F. Sgallari, Fast and robust inexact Newton approach to the harmonic-balance analysis of nonlinear microwave circuits, IEEE Microwave Guided Waves Lett., 7 (1997), pp. 359-361.
44. R. B. Schnabel and J. E. Dennis, Jr., A New Derivation of Symmetric Positive Definite Secant Updates, Tech. report, Department of Computer Science, University of Colorado, Boulder, CO, 1980.
45. D. F. Shanno, Conditioning of quasi-Newton methods for function minimization, Math. Comp., 24 (1970), pp. 647-656.
46. J. Sherman and W. J. Morrison, Adjustment of an inverse matrix corresponding to a change in the elements of a given column or a given row of the original matrix, Ann. Math. Statistics, 20 (1949), p. 621.
47. J. Sherman and W. J. Morrison, Adjustment of an inverse matrix corresponding to a change in one element of a given matrix, Ann. Math. Statistics, 21 (1950), pp. 124-127.
48. L. Sorber, M. Van Barel, and L. De Lathauwer, Optimization-based Algorithms for Tensor Decompositions: Canonical Polyadic Decomposition, Decomposition in Rank-(Lr, Lr, 1) Terms and a New Generalization, ESAT-SISTA Internal Report 12-37, Department of Electrical Engineering (ESAT), KU Leuven, Leuven, Belgium, 2012.
49. A. Suárez, Analysis and Design of Autonomous Microwave Circuits, Wiley Ser. Microw. Opt. Eng. 190, Wiley-IEEE Press, Hoboken, NJ, Piscataway, NJ, 2009.
50. A. van den Bos, Complex gradient and Hessian, IEE Proc. Vis. Image & Signal Process., 141 (1994), pp. 380-383.
51. Ph. Wolfe, Convergence conditions for ascent methods, SIAM Rev., 11 (1969), pp. 226-235.