Stochastic optimization

Handbook of Computational Statistics: Concepts and Methods: Second Edition

Volumn , Issue , 2012, Pages 173-201

  Spall, James C ^a  

^a JOHNS HOPKINS UNIVERSITY APPLIED PHYSICS LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

STOCHASTIC OPTIMIZATIONS;

OPTIMIZATION;

EID: 85028288109     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-3-642-21551-3__7     Document Type: Chapter

References (55)

1. Arsham, H.: Techniques for Monte Carlo optimizing. Monte Carlo Methods Appl. 4, 181-229 (1998)
2. Baba, N., Shoman, T., Sawaragi, Y.: A modified convergence theorem for a random optimization method. Inf. Sci. 13, 159-166 (1977)
3. Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear programming: theory and algorithms. Wiley, New York (1993)
4. Blum, J.R.: Multidimensional stochastic approximation methods. Ann. Math. Stat. 25, 737-744 (1954)
5. Box, G.E.P.: Evolutionary operation: a method for increasing industrial productivity. J. R. Stat. Soc. Ser. C: Appl. Stat. 6, 81-101 (1957)
6. Cochran, J.J. (ed.): Encyclopedia of operations research and management science. Wiley, Hoboken (2011)
7. Das, S., Spall, J.C., Ghanem, R.: Efficient Monte Carlo computation of Fisher information matrix using prior information. Comput. Stat. Data Anal. 54(2), 272-289 (2010)
8. De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems. Ph.D. dissertation, University of Michigan (University Microfilms no. 76-9381) (1975)
9. Dippon, J., Renz, J.: Weighted means in stochastic approximation of minima. SIAM J. Contr. Optim. 35, 1811-1827 (1997)
10. Fabian, V.: Stochastic approximation. In: Rustigi, J.S. (ed.) Optimizing methods in statistics, pp. 439-470. Academic, New York (1971)
11. Fogel, D.B.: Evolutionary computation: toward a new philosophy of machine intelligence. IEEE, Piscataway (2000)
12. Fouskakis, D., Draper, D.: Stochastic optimization: a review. Int. Stat. Rev. 70, 315-349 (2002)
13. Fu, M.C.: Optimization for simulation: theory vs. practice. INFORMS J. Comput. 14, 192-227 (2002)
14. Gentle, J.E.: Random number generation and Monte Carlo methods. Springer, New York (2003)
15. Gerencsér, L.: Convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting. IEEE Trans. Autom. Control 44, 894-905 (1999)
16. Gerencsér, L., Hill, S.D., Vágó, Z.: Optimization over discrete sets via SPSA. In: Proceedings of the IEEE Conference on Decision and Control, Phoenix, 7-10 December 1999, 1791-1795
17. Goldberg, D.E.: Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading (1989)
18. Goldberg, D.E.: The design of innovation: lessons from and for competent genetic algorithms. Kluwer Academic, Boston (2002)
19. Gosavi, A.: Simulation-based optimization: parametric optimization techniques and reinforcement learning. Kluwer Academic, Boston (2003)
20. Hill, S.D.: Discrete stochastic approximation with application to resource allocation. Johns Hopkins APL Tech. Dig. 26, 15-21 (2005)
21. Holland, J.H.: Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor (1975)
22. Jeffreys, H.: An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. A: Math. Phys. Sci. 186, 453-461 (1946)
23. Karnopp, D.C.: Random search techniques for optimization problems. Automatica 1, 111-121 (1963)
24. Kiefer, J., Wolfowitz, J.: Stochastic estimation of a regression function. Ann. Math. Stat. 23, 462-466 (1952)
25. Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385-482 (2003)
26. Kushner, H.J., Yin, G.G.: Stochastic approximation and recursive algorithms and applications. Springer, New York (2003)
27. Ljung, L.: System identification - theory for the user. Prentice Hall PTR, Upper Saddle River (1999)
28. Maryak, J.L., Chin, D.C.: Global random optimization by simultaneous perturbation stochastic approximation. IEEE Trans. Autom. Control 53, 780-783 (2008)
29. Matyas, J.: Random optimization. Autom. Remote Control 26, 244-251 (1965)
30. Michalewicz, Z.: Genetic algorithms + data structures = evolution programs. Springer, New York (1996)
31. Michalewicz, Z., Fogel, D.B.: How to solve it: modern heuristics. Springer, New York (2000)
32. Mitchell, M.: An introduction to genetic algorithms. MIT Press, Cambridge (1996)
33. Nelder, J. A., Mead, R., A simplex method for function minimization. Comput. J., 7, 308-313 (1965)
34. Pflug, G.C.h.: Optimization of stochastic models: the interface between simulation and optimization. Kluwer Academic, Boston (1996)
35. Reeves, C.R., Rowe, J.E.: Genetic algorithms - principles and perspectives: a guide to GA theory. Kluwer Academic, Boston (2003)
36. Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400-407 (1951)
37. Rudolph, G.: Convergence analysis of Canonical genetic algorithms. IEEE Trans. Neural Netw. 5, 96-101 (1994)
38. Rudolph, G.: Convergence properties of evolutionary algorithms. Kovac, Hamburg (1997)
39. Ruppert, D.: Stochastic approximation. In: Ghosh, B.K., Sen, P.K. (eds.) Handbook of sequential analysis, pp. 503-529. Dekker, New York (1991)
40. Schwefel, H.P.: Evolution and optimum seeking. Wiley, New York (1995)
41. Solis, F.J., Wets, J.B.: Minimization by random search techniques. Math. Oper. Res. 6, 19-30 (1981)
42. Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37, 332-341 (1992)
43. Spall, J.C.: Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Autom. Control 45, 1839-1853 (2000)
44. Spall, J.C.: Introduction to stochastic search and optimization: estimation, simulation, and control. Wiley, Hoboken (2003)
45. Spall, J.C.: Monte Carlo computation of the Fisher information matrix in nonstandard settings. J. Comput. Graph. Stat. 14(4), 889-909 (2005)
46. Spall, J.C.: Feedback and weighting mechanisms for improving Jacobian estimates in the adaptive simultaneous perturbation algorithm. IEEE Trans. Autom. Control 54(6), 1216-1229 (2009)
47. Spall, J.C.: Factorial design for choosing input values in experimentation: generating informative data for system identification. IEEE Control Syst. Mag. 30(5), 38-53 (2010)
48. Stark, D.R., Spall, J.C.: Rate of convergence in evolutionary computation. In: Proceedings of the American Control Conference, Denver, 4-6 June 2003
49. Suzuki, J.: AMarkov chain analysis on simple genetic algorithms. IEEE Trans. Syst.Man. Cybern. 25, 655-659 (1995)
50. Vose, M.: The simple genetic algorithm. MIT Press, Cambridge (1999)
51. Wang, Q., Spall, J.C.: Discrete simultaneous perturbation stochastic approximation on loss functions with noisy measurements. In: Proceedings of the American Control Conference, San Francisco, 29 June-1 July 2011, pp. 4520-4525 (paper FrB10.3) (2011)
52. Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67-82 (1997)
53. Yakowitz, S.J., Fisher, L.: On sequential search for the maximum of an unknown function. J. Math. Anal, Appl. 41, 234-259 (1973)
54. Yin, G.: Rates of convergence for a class of global stochastic optimization algorithms. SIAM J. Optim. 10, 99-120 (1999)
55. Zhigljavsky, A.A.: Theory of global random search. Kluwer Academic, Boston (1991)