Hessian approximation algorithms for hybrid optimization methods

Engineering Optimization

Volumn 41, Issue 7, 2009, Pages 609-633

  Tahk, Min Jea ^a   Park, Moon Su ^b   Woo, Hyun Wook ^c   Kim, Hyoun Jin ^d  

^a Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^b Korea Aerospace Research Institute (KARI)   (South Korea)

^c LIG Nex1   (South Korea)

^d Seoul National University   (South Korea)

Author keywords

Evolution strategy; Hessian approximation; Hybrid method; Quasi Newton method

Indexed keywords

BROYDEN-FLETCHER-GOLDFARB-SHANNO; CONTINUOUS OPTIMIZATION PROBLEMS; CONTINUOUS PARAMETERS; DAVIDON-FLETCHER-POWELL; ESTIMATION PROCESS; EVOLUTION STRATEGY; EVOLUTIONARY SEARCH; FUNCTION VALUES; GRADIENT INFORMATIONS; GRADIENT VECTORS; GRADIENT-BASED METHOD; HESSIAN APPROXIMATION; HESSIAN MATRICES; HYBRID ALGORITHMS; HYBRID METHOD; HYBRID OPTIMIZATION METHOD; NUMERICAL EXPERIMENTS; OPTIMIZATION PROBLEMS; QUASI-NEWTON; QUASI-NEWTON METHOD; QUASI-NEWTON METHODS; SEARCH DIRECTION; SMALL POPULATION;

ESTIMATION; EVOLUTIONARY ALGORITHMS; HYBRID SENSORS; MATRIX ALGEBRA; NEWTON-RAPHSON METHOD; OPTIMIZATION;

APPROXIMATION ALGORITHMS;

EID: 70449556673     PISSN: 0305215X     EISSN: 10290273     Source Type: Journal    
DOI: 10.1080/03052150902736879     Document Type: Article

References (61)

1. Attaviriyanupap, P., et al., 2002.A hybrid EP and SQP for dynamic economic dispatch with nonsmooth fuel cost function. IEEE Transactions on Power Systems, 17 (2), 411-416.
2. Auger, A. and Hansen, N. 2005a. A restart CMA evolution strategy with increasing population size. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA: IEEE, 1769-1776.
3. Auger, A. and Hansen, N. 2005b. Performance evaluation of an advanced local search evolutionary algorithm. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA: IEEE, 1777-1784.
4. Baali, M.A., Fuduli, A., and Musmanno, R. 2004. On the performance of switching BFGS/SR1 algorithms for unconstrained optimization. Optimization Methods and Software, 19 (2), 153-164.
5. Ballester, P.J., et al., 2005. Real-parameter optimization performance study on the CEC-2005 benchmark with SPC-PNX. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA: IEEE, 498-505.
6. Bazaraa, M.S., Sherali, H.D., and Shetty, C.M. 1993. Nonlinear programming theory and algorithms. 2nd ed. NewYork: JohnWiley & Sons.
7. Beyer, H.-G. and Deb, K. 2001. On self-adaptive features in real-parameter evolutionary algorithms. IEEE Transactions on Evolutionary Computation. 5 (3), 250-270.
8. Box, G.E.P. andWilson, K.B. 1951. On the experimental attainment of optimum conditions. Journal of the Royal Statistical Society, Series B, 13, 1-45.
9. Chellapilla, K. 1998. Combining mutation operators in evolutionary programming. IEEE Transactions on Evolutionary Computation, 2 (3), 91-96.
10. Chunduru, R.K., Sen, M.K., and Stoffa, P.L. 1997. Hybrid optimization methods for geophysical inversion. Geophysics, 62 (4), 1196-1207.
11. Coley, D.A. 1999. An introduction to genetic algorithms for scientists and engineers. Singapore:World Scientific.
12. Crain, T., et al., 2000. Interplanetary flyby mission optimization using a hybrid global-local search method. Journal of Spacecraft and Rockets, 37 (4), 468-474.
13. Daescu, D.N. and Navon, I.M. 2003. An analysis of a hybrid optimization method for variational data assimilation. International Journal of Computational Fluid Dynamics, 17 (4), 299-306.
14. Daescu, D.N. and Navon, I.M. 2007. Efficiency of a POD-based reduced second-order adjoint model in 4D-var data assimilation. International Journal for Numerical Method in Fluids. 53 (6), 985-1004.
15. Deb, K.,Anand,A., and Joshi, D. 2002.Acomputationally efficient evolutionary algorithm for real-parameter optimization. Evolutionary Computation, 10 (4), 371-395.
16. Digalakis, J. and Margaritis, K. 2004. Performance comparison of memetic algorithms. Applied Mathematics and Computation, 158 (1), 237-252.
17. García, S., et al., 2008. A study on the use of non-parametric tests for analyzing the evolutionary algorithms' behaviour: A case study on the CEC'2005 Special Session on Real Parameter Optimization. Journal of Heuristics, doi:10.1007/s10732-008-9080-4.
18. García-Martínez, C. and Lozano, M. 2005. Hybrid real-coded genetic algorithms with female and male differentiation. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA: IEEE, 896-903. (Pubitemid 41495945)
19. Gelb, A. 1974. Applied optimal estimation. Cambridge, USA: MIT Press.
20. Gill, P.E., Murray,W., and Wright, M.H. 1981. Practical optimization. London: Academic Press.
21. Hart,W.E. 1994. Adaptive global optimization with local search. Thesis (PhD). University of California.
22. Hart,W.E., Krasnoger, N., and Smith, J. 2004. Recent advances in memetic algorithms. Berlin: Springer.
23. Herrera, F., Lozano, M., and Verdegay, J.L. 1998. Tackling real-coded genetic algorithms: Operators and tools for behavioural analysis. Artificial Intelligent Review, 12 (4), 265-319.
24. Herrera, F., Lozano, M., and Sánchez, A.M. 2003. A taxonomy for the crossover operator real-coded genetic algorithms: An experimental study. International Journal of Intelligent Systems, 18 (3), 309-338.
25. Jones, D.R., Schonlau, M., andWelch,W.J. 1998. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13, 455-492.
26. Kita, H. 2001.Acomparison study of self-adaptation in evolution strategies and real-coded genetic algorithms. Evolutonary Computational Journal, 9 (2), 223-241.
27. Krasnogor, N. and Smith, J. 2005. A tutorial for competent memetic algorithms: Model, taxonomy, and design issues. IEEE Transactions on Evolutionary Computation, 9 (5), 474-488.
28. LeDimet, F.-X., Navon, I.M., and Daescu, D.N. 2002. Second order information in data assimilation. Monthly Weather Review, 130 (3), 629-648.
29. Leung, Y.W. and Wang, Y. 2001. An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Transactions on Evolutionary Computation, 5 (1), 41-53.
30. Liang, K.H.,Yao, X., and Newton C. 2000. Evolutionary search of approximated n-dimensional landscapes. International Journal of Knowledge-Based Intelligent Engineering Systems, 4 (3), 172-183.
31. Liang, J.J. and Suganthan, P.N. 2005. Dynamic multi-swarm particle swarm optimizer with local search. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh, Piscataway, NJ: IEEE, 522-528.
32. Liu, D.C. and Nocedal, J. 1989. On the limited memory BFGS method for large scale minimization. Mathematical Programming, 45 (3), 503-528.
33. Liu, G.R., Han, X., and Lam, K.Y. 2002. A combined genetic algorithm and nonlinear least squares method for material characterization using elastic waves. Computer Methods in Applied Mechanics and Engineering, 191 (17/18), 1909-1921.
34. Lozano, M., et al., 2004. Real-coded memetic algorithms with crossover hill-climbing. Evolutionary Computation, 12 (3), 273-302.
35. Merz, P. and Freisleben, B. 1999. Fitness landscapes and memetic algorithm design: Chapter 3. In: D. Corne, F. Glover, and M. Dorigo, eds. New ideas in optimization. NewYork: McGraw-Hill.
36. Molina, D., Herrera, F., and Lozano, M. 2005. Adaptive local search parameters for real-coded memetic algorithms. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA: IEEE, 888-895.
37. Moscato, P. 1989. On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms. CalTech Concurrent Computation Program Report 826, CA, USA: CalTech.
38. Moscato, P. 1999. Memetic Algorithms: A Short Introduction. In: D. Corne, F. Glover, and M. Dorigo, Eds. New Ideas in Optimization. NewYork: McGraw-Hill.
39. Nash, S.G. and Nocedal, J. 1991. A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization. SIAM Journal on Optimization, 1 (3), 358-372.
40. Navon, I.M., Brown, F.B., and Robertson D.H. 1990.A combined simulated annealing and quasi-Newton-like conjugategradient method for determining the structure of mixed argon-xenon clusters. Computers and Chemistry, 14 (4), 305-311.
41. Nocedal, J. and Wright, S.J. 2006. Numerical optimization. 2nd ed. newYork: Springer-Verlag.
42. Noman, N. and Iba, H. 2008. Accelerating differential evolution using an adaptive local search. IEEE Transactions on Evolutionary Computation, 12 (1), 107-125.
43. Ogata, K. 1994. Discrete-time control. 2nd ed. New Jersey, USA: Prentice Hall.
44. Pošík, P. 2005. Real-parameter optimization using the mutation step co-evolution. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA: IEEE, 872-879.
45. Price, K.V., Storn, R.M., and Lampinen, J.A. 2005. Differential evolution: A practical approach to global optimization. Berlin: Springer-Verlag.
46. Qin, A.K. and Suganthan, P.N. 2005. Self-adaptive differential evolution algorithm for numerical optimization. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA: IEEE, 1785-1791.
47. Rönkkönen, J., Kukkonen, S., and Price, K.V. 2005. Real-parameter evolution with differential evolution. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA : IEEE, 506-513.
48. Ruiz-Andino, A., et al., 2000. A hybrid evolutionary approach for solving constrained optimization problems over finite domains. IEEE Transactions on Evolutionary Computation, 4 (4), 353-371.
49. Sacks, J., et al., 1989. Design and analysis of computer experiments. Statistical Science, 4 (4), 409-435.
50. Sherali, H.D. 1990. Conjugate gradient methods using quasi-Newton updates with inexact line searches. Journal of Numerical Analysis and Applications, 150 (2), 359-377.
51. Shinha, A., Tiwari, S., and Deb, K. 2005. A population-based, steady-state procedure for real-parameter optimization. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA: IEEE, 514-521. (Pubitemid 41495896)
52. Storn, R.M. and Price, K.V. 1997. Differential evolution-A simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11 (4), 341-359.
53. Suganthan, P.N., et al., 2005. Problem definitions and evaluation criteria for the CEC 2005 Special Session on Real- Parameter Optimization: Technical report, Singapore: Nanyang Technological University.
54. Tahk, M.J., Woo, H.W., and Park, M.S. 2007. A hybrid optimization of evolutionary and gradient search. Engineering Optimization, 39 (1), 87-104.
55. Tahk, M. and Sun, B. 2000. Coevolutionary augmented Lagrangian methods for constrained optimization and applications. IEEE Transactions On Evolutionary Computation, 4 (2), 114-124.
56. Vicini, A. and Quagliarella, D. 1999. Airfoil and wing design through hybrid optimization strategies. AIAA Journal, 37 (5), 634-641.
57. Wang, Z., et al., 1995. A truncated-Newton optimization algorithm in meteorology applications with analytic Hessian/vector products. Computational Optimization and Applications, 4 (3), 241-262.
58. Winfield, D. 1973. Function minimization by interpolation in a data table. Journal of the Institute of Mathematics and its Applications, 12, 339-347.
59. Yao, X., Liu, Y., and Lin, G. 1999. Evolutionary programming made faster. IEEE Transactions on Evolutionary Computation, 3 (2), 82-102.
60. Yuan, B. and Gallagher, M. 2005. Experimental results for the Special Session on Real-Parameter Optimization at CEC 2005: A simple, continuous EDA. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, Edinburgh. New Jersey, USA:IEEE, 1792-1799.
61. Zar, J.H. 1999. Biostatistical analysis, 4th ed. New Jersey, USA: Prentice Hall.