Stochastic approximation algorithms for constrained optimization via simulation

ACM Transactions on Modeling and Computer Simulation

Volumn 21, Issue 3, 2011, Pages

  Bhatnagar, Shalabh ^a   Hemachandra, N ^b   Mishra, Vivek Kumar ^a  

^a INDIAN INSTITUTE OF SCIENCE   (India)

^b INDIAN INSTITUTE OF TECHNOLOGY BOMBAY   (India)

Author keywords

Inequality constraints; Lagrange multiplier; SF estimates; Simulation based constrained optimization; SPSA estimates; Stochastic approximation

Indexed keywords

INEQUALITY CONSTRAINT; LAGRANGE; SF ESTIMATES; SIMULATION-BASED; SPSA ESTIMATES; STOCHASTIC APPROXIMATIONS;

APPROXIMATION THEORY; COERCIVE FORCE; CONSTRAINED OPTIMIZATION; CONVERGENCE OF NUMERICAL METHODS; ESTIMATION; FREQUENCY MULTIPLYING CIRCUITS; LAGRANGE MULTIPLIERS; STOCHASTIC SYSTEMS;

APPROXIMATION ALGORITHMS;

EID: 79952548696     PISSN: 10493301     EISSN: 15581195     Source Type: Journal    
DOI: 10.1145/1921598.1921599     Document Type: Article

References (25)

1. Bertsekas, D.P. and Tsitsiklis, J. N. 1996. Neuro-Dynamic Programming. Athena Scientific, Belmont, MA.
2. Bhatnagar, S. 2005. Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization. ACM Trans. Model. Comput. Simul. 15, 1, 74-107. (Pubitemid 40564808)
3. Bhatnagar, S. 2007. Adaptive Newton-based smoothed functional algorithms for simulation optimization. ACM Trans. Model. Comput. Simul. 18, 1, 27-62.
4. Bhatnagar, S. 2010. An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes. Syst. Control Lett. 59, 760-766.
5. Bhatnagar, S. and Borkar, V. S. 1998. A two time scale stochastic approximation scheme for simulation based parametric optimization. Prob. Eng. Inf. Sci. 12, 519-531. (Pubitemid 128544961)
6. Bhatnagar, S. and Borkar, V. S. 2003. Multiscale chaotic SPSA and smoothed functional algorithms for simulation optimization. Simulation 79, 10, 568-580.
7. Bhatnagar, S., Fu, M. C., Marcus, S. I., and Bhatnagar, S. 2001. Two timescale algorithms for simulation optimization of hidden Markov models. IIE Trans. 33, 3, 245-258. (Pubitemid 32271551)
8. Borkar, V. S. 1997. Stochastic approximation with two timescales. Syst. Control Lett. 29, 291-294.
9. Borkar, V. S. 2005. An actor-critic algorithm for constrained Markov decision processes. Syst. Control Lett. 54, 207-213. (Pubitemid 40183502)
10. Brandiere, O. 1998. Some pathological traps for stochastic approximation. SIAM J. Control Optim. 36, 1293-1314. (Pubitemid 128490849)
11. Cassandras, C. G. 1993. Discrete Event Systems: Modeling and Performance Analysis. Aksen Associates, Boston, MA.
12. Chen, H. F., Duncan, T. E., and Pasik-Duncan, B. 1999. A Kiefer-Wolfowitz algorithm with randomized differences. IEEE Trans. Auto. Cont. 44, 3, 442-453.
13. Hirsch, M. W. 1989. Convergent activation dynamics in continuous time networks. Neural Netw. 2, 331-349.
14. Ho, Y. C. and Cao, X. R. 1991. Perturbation Analysis of Discrete Event Dynamical Systems. Kluwer, Boston.
15. Katkovnik, V. Y. and Kulchitsky, Y. 1972. Convergence of a class of random search algorithms. Autom. Remote Control 8, 1321-1326.
16. Kushner, H.J. and Yin, G. G. 1997. Stochastic Approximation Algorithms and Applications. Springer, Berlin.
17. Mas-Colell, A., Whinston, M. D., and Green, J. R. 1995. Microeconomic Theory. Oxford University Press, Oxford, UK.
18. Meyn, S.P. and Tweedie, R. L. 1993. Markov Chains and Stochastic Stability. (2nd ed.), Springer, Berlin.
19. Sadegh, P. 1997. Constrained optimization via stochastic approximation with a simultaneous perturbation gradient approximation. Automatica 33, 889-892. (Pubitemid 127413705)
20. Spall, J. C. 1992. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37, 3, 332-341.
21. Spall, J. C. 2000. Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Autom. Control 45, 1839-1853. (Pubitemid 32067073)
22. Styblinski, M.A. and Tang, T.-S. 1990. Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing. Neural Netw. 3, 467-483. (Pubitemid 20721119)
23. Tadic, V. B., Doucet, A., and Singh, S. 2003. Two time-scale stochastic approximation for constrained stochastic optimization and constrained Markov decision problems. In Proceedings of the American Control Conference, 4736-4741.
24. Wang, I.-J. and Spall, J. C. 2008. Stochastic optimization with inequality constraints using simultaneous perturbations and penalty functions. Int. J. Control 81, 8, 1232-1238.
25. Whitney, J. E. and Hill, S. D. 2001. Constrained optimization over discrete sets via SPSA with application to non-separable resource allocation. In Proceedings of the Winter Simulation Conference, 313-317. (Pubitemid 34091309)