Incremental least squares methods and the extended Kalman filter

SIAM Journal on Optimization

Volumn 6, Issue 3, 1996, Pages 807-822

  Bertsekas, Dimitri P ^a  

^a USA   (United States)

Author keywords

Kalman filter; Least squares; Optimization

Indexed keywords

EID: 0030303864     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/S1052623494268522     Document Type: Article

References (34)

1. M. ATHANS, R. P. WISHNER, AND A. BERTOLINI, Suboptimal state estimation for continuous time nonlinear systems from discrete noisy measurements, IEEE Trans. Automat. Control, AC-13 (1968), pp. 504-514.
2. B. D. O. ANDERSON AND J. B. MOORE, Optimal Filtering, Prentice-Hall, Englewood Cliffs, NJ, 1979.
3. D. P. BERTSEKAS AND J. N. TSITSIKLIS, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, Englewood Cliffs, NJ, 1989.
4. _, Neuro-Dynamic Programming, Athena Scientific, Belmont, MA, 1996.
5. B. M. BELL, The iterated Kalman smoother as a Gauss-Newton method, SIAM J. Optim., 4 (1994), p. 626.
6. D. P. BERTSEKAS, Projected Newton methods for optimization problems with simple constraints, SIAM J. Control Optim., 20 (1982), pp. 221-246.
7. _, Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, 1982.
8. _, Nonlinear Programming, Athena Scientific, Belmont, MA, 1995.
9. W. C. DAVIDON, New least squares algorithms, J. Optim. Theory Appl., 18 (1976), pp. 187-197.
10. A. A. GAIVORONSKI, Convergence Analysis of Parallel Backpropagation Algorithm for Neural Networks, Symposium on Parallel Optimization 3, Madison, WI, July 7-9, 1993.
11. L. GRIPPO, A Class of Unconstrained Minimization Methods for Neural Network Training, Symposium on Parallel Optimization 3, Madison, WI, July 7-9, 1993.
12. A. H. JAZWINSKI, Stochastic Processes and Filtering Theory, Academic Press, New York, 1970.
13. H. J. KUSHNER AND D. S. CLARK, Stochastic Approximation Methods for Constrained and Unconstrained Systems, Springer-Verlag, New York, 1978.
14. L. LJUNG, Analysis of recursive stochastic algorithms, IEEE Trans. Automat. Control, AC-22 (1977), pp. 551-575.
15. _, Asymptotic behavior of the extended Kalman filter as a parameter estimator for linear systems, IEEE Trans. on Automat. Control, AC-24 (1979), pp. 36-50.
16. Z. Q. LUO AND P. TSENG, Analysis of an approximate gradient projection method with applications to the backpropagation algorithm, Optim. Meth. Software, 4 (1994), pp. 85-101.
17. Z. Q. LUO, On the convergence of the LMS algorithm with adaptive learning rate for linear feedforward networks, Neural Comput., 3 (1991), pp. 226-245.
18. O. L. MANGASARIAN AND M. V. SOLODOV, Serial and parallel backpropagation convergence via nonmonotone perturbed minimization, Optim. Methods Software, 4 (1994), pp. 103-116.
19. O. L. MANGASARIAN, Mathematical programming in neural networks, ORSA J. Comput., 5 (1993), pp. 349-360.
20. R. K. MEHRA, A comparison of several nonlinear filters for reentry vehicle tracking, IEEE Trans. Automat. Control, AC-16 (1971), pp. 307-319.
21. T. N. PAPPAS, Solution of Nonlinear Equations by Davidon's Least Squares Method, M.S. thesis, Dept. of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, 1982.
22. B. T. POLJAK AND Y. Z. TSYPKIN, Pseudogradient adaptation and training algorithms, Automation Remote Control, 12 (1973), pp. 45-68.
23. B. T. POLJAK, Introduction to Optimization, Optimization Software Inc., NY, 1987.
24. D. W. RUCK, S. K. ROGERS, M. KABRISKY, P. S. MAYBECK, AND M. E. OXLEY, Comparative analysis of backpropagation and the extended Kalman filter for training multilayer perceptrons, IEEE Trans. Pattern Analysis and Machine Intelligence, 14 (1992), pp. 686-691.
25. J. N. TSITSIKLIS, D. P. BERTSEKAS, AND M. ATHANS, Distributed asynchronous deterministic and stochastic gradient optimization algorithms, IEEE Trans. Automat. Control, AC-31 (1986), pp. 803-812.
26. R. R. TENNEY, R. S. HEBBERT, AND N. R. SANDELL, JR., Tracking filter for maneuvering sources, IEEE Trans. Automat. Control, AC-22 (1977), pp. 246-251.
27. J. N. TSITSIKLIS, Problems in Decentralized Decision Making and Computation, Ph.D. dissertation, Dept. of Electrical Enginering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, 1984.
28. B. URSIN, Asymptotic convergence properties of the extended Kalman filter using filtered state estimates, IEEE Trans. Automat. Control, AC-25 (1980), pp. 1207-1211.
29. H. WHITE, Some asymptotic results for learning in single hidden-layer feedforward network models, J. Amer. Statist. Assoc., 84 (1989), pp. 1003-1013.
30. _, Learning in artificial neural networks: A statistical perspective, Neural Comput., 1 (1989), pp. 425-464.
31. K. WATANABE AND S. G. TZAFESTAS, Learning algorithms for neural networks with the Kalman filters, J. Intelligent Robotic Systems, 3 (1990), pp. 305-319.
32. H. WEISS AND J. B. MOORE, Improved extended Kalman filter design for passive tracking, IEEE Trans. Automat. Control, AC-25 (1980), pp. 807-811.
33. B. WIDROW AND M. E. HOFF, Adaptive Switching Circuits, Institute of Radio Engineers, Western Electronic Show and Convention, Convention Record, part 4, 1960, pp. 96-104.
34. B. WIDROW AND S. D. STEARNS, Adaptive Signal Processing, Prentice-Hall, Englewood Cliffs, NJ, 1985.