A dynamical system approach to stochastic approximations

SIAM Journal on Control and Optimization

Volumn 34, Issue 2, 1996, Pages 437-472

  Benaim, Michel ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Chain recurrence; Neural networks; Ordinary differential equations; Stochastic approximations

Indexed keywords

ASYMPTOTIC STABILITY; DIFFERENTIAL EQUATIONS; NEURAL NETWORKS; RANDOM PROCESSES;

CHAIN RECURRENCE; ORDINARY DIFFERENTIAL EQUATIONS; STOCHASTIC APPROXIMATIONS;

APPROXIMATION THEORY;

EID: 0030104967     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012993253534     Document Type: Article

References (35)

1. M. BENAIM (1994a), Un théorème de Poincaré-Bendixson pour une classe d'equations différentielles stochastiques, C. R. Acad. Sci. Paris Sér. I, 318, pp. 837-839.
2. _ (1994b), On functional approximation with normalized Gaussian units, Neural Computation, 6, pp. 319-333.
3. M. BENAIM AND M. W. HIRSCH (1995a), Dynamics of Morse-Smale urns processes, Ergodic Theory Dynamical Systems, to appear.
4. _ (1995b), Asymptotic pseudo-trajectories, chain-recurrent flows and stochastic approximations, J. Dynamics Differential Equations, to appear.
5. _ (1995c), Chain recurrence in surface flows, Discrete and Continuous Dynamics Systems, 1 (1995), pp. 1-17.
6. M. BENAIM AND L. TOMASINI (1991), Competitive and self-organizing algorithms based on the minimization of an information criterion, in Artificial Neural Networks I, Vol. 1, T. Kohonen, ed., North-Holland, Amsterdam, pp. 391-396.
7. _ (1992), Approximating function and predicting time series with multisigmoidal basis functions, in Artificial Neural Networks II, Vol. 1, I. Aleksander and J. Taylor, eds., Elsevier, Amsterdam, pp. 407-411.
8. A. BENVENISTE, M. MÉTIVIER, AND P. PRIOURET (1990), Stochastic Approximations and Adaptive Algorithms, Springer-Verlag, Berlin, Heidelberg, New York. Translated from Algorithmes adaptatifs et approximations stochastiques, Masson, Paris, 1987.
9. A. BENVENISTE, M. MÉTIVIER, AND P. PRIOURET (1990), Stochastic Approximations and Adaptive Algorithms, Springer-Verlag, Berlin, Heidelberg, New York. Translated from Algorithmes adaptatifs et approximations stochastiques, Masson, Paris, 1987.
10. P. BILLINGSLEY (1968), Convergence of Probability Measures, Wiley, London, New York.
11. C. C. CONLEY (1978), Isolated Invariant Sets and the Morse Index, Regional Conference Series in Mathematics 38, American Mathematical Society, Providence, RI.
12. E. EWEDA AND O. MACCHI (1983), Quadratic mean and almost sure convergence of unbounded stochastic approximation algorithms with correlated observations, Ann. Inst. H. Poincaré, 19, pp. 235-255.
13. J. C. FORT AND G. PAGÈS (1994), Convergences d'algorithmes stochastiques: le théorème de Kushner et Clark revisité, pré-pub, labo. de proba., Univ. Paris 6.
14. E. G. GLADYSHEV (1965), On stochastic approximation, Theory Probab. Appl., 10, pp. 275-278.
15. D. HALL AND C. C. HEYDE (1980), Martingale Limit Theory and Applications, Academic Press, New York.
16. M. W. HIRSCH (1993), Asymptotic phase, shadowing and reaction-diffusion systems, in Control Theory, Dynamical Systems and Geometry of Dynamics, K. D. Elworthy, W. N. Everitt, and E. B. Lee, eds., Marcel Dekker, New York.
17. M. W. HIRSCH AND C. PUGH (1970), Stable manifolds for hyperbolic sets, in Global Analysis, Proceedings of Symposia in Pure Mathematics 14, American Mathematical Society, Providence, RI, pp. 133-163.
18. J. KIEFER AND J WOLFOWITZ (1952), Stochastic estimation of the maximum of a regression function, Ann. Math. Statist., 23, pp. 462-466.
19. C. M. KUAN AND H. WHITE (1992), Artificial neural networks: An econometric perspective, Econometric Reviews, to be published.
20. H. J. KUSHNER AND D. S. CLARK (1978), Stochastic Approximation Methods for Constrained and Unconstrained Systems, Springer-Verlag, Berlin, Heidelberg, New York.
21. L. LJUNG (1977), Analysis of recursive stochastic algorithms, IEEE Trans. Automat. Control, AC-22, pp. 551-575.
22. D. L. MCLEISH (1975), A maximal inequality and dependent strong laws, Ann. Probab., 3, pp. 829-839
23. M. MÉTIVIER AND P. PRIOURET (1984), Application of a Kushner and Clark lemma to general classes of stochastic algorithms, IEEE Trans. Inform. Theory, IT-30, pp. 140-150
24. _ (1987), Théorèmes de convergence presque sure pour une classe d'algorithmes stochastiques à pas décroissant, Probab. Theory Related Fields, 74, pp. 403-428.
25. J. MOODY AND C. DARKEN (1989), Fast learning in networks of locally-tuned processing units, Neural Computation, 1, pp. 281-582.
26. M. B. NEVEL'SON AND R. Z. HAS'MINSKII (1974), Stochastic Approximation and Recursive Estimation, Translation of Math. Monographs 47, American Mathematical Society, Providence, RI.
27. J. NEVEU (1964), Bases mathematiques du calcul des probabilites, Masson, Paris.
28. S. NOWLAN (1990), Maximum likelihood competitive learning, in Proceedings of Neural Information Processing Systems, pp. 574-582.
29. J. PALIS (1969), On Morse-Smale dynamical systems, Topology, 8, pp. 385-405.
30. R. PEMANTLE (1990), Nonconvergence to unstable points in urn models and stochastic approximations, Ann. Probab., 18, pp. 698-712.
31. T. POGGIO AND F. GIROSI (1990), Regularization algorithms for learning that are equivalent to multilayer networks, Science, 247, pp. 979-982.
32. H. ROBBINS AND S. MONRO (1951), A stochastic approximation method, Ann. Math. Statist., 22, pp. 400-407.
33. C. ROBINSON (1977), Stability theorems and hyperbolicity in dynamical systems, Rocky J. Math., 7, pp. 425-434.
34. M. SHUB (1986), Global Stability of Dynamical Systems, Springer-Verlag, New York.
35. H. WHITE (1992), Artificial Neural Networks, Approximation and Learning Theory, Blackwell, Oxford, Cambridge.