The variational Kalman filter and an efficient implementation using limited memory BFGS

International Journal for Numerical Methods in Fluids

Volumn 64, Issue 3, 2010, Pages 314-335

  Auvinen, H ^a   Bardsley, J M ^b   Haario, H ^a   Kauranne, T ^a  

^a LAPPEENRANTA UNIVERSITY OF TECHNOLOGY   (Finland)

^b UNIVERSITY OF MONTANA   (United States)

Author keywords

Bayesian inversion; Kalman filter; Large scale optimization; Nonlinear dynamics; Optimization; Quasi newton methods; Variational methods

Indexed keywords

BAYESIAN INVERSION; LARGE-SCALE OPTIMIZATION; NON-LINEAR DYNAMICS; QUASI-NEWTON METHODS; VARIATIONAL METHODS;

BAYESIAN NETWORKS; DATA PROCESSING; DYNAMICS; KALMAN FILTERS; NEWTON-RAPHSON METHOD;

OPTIMIZATION;

EID: 77956576458     PISSN: 02712091     EISSN: 10970363     Source Type: Journal    
DOI: 10.1002/fld.2153     Document Type: Article

References (30)

1. Kalman RE. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering 1960; 82(Series D):35-45.
2. Dee DP. Simplification of the Kalman filter for meteorological data assimilation. Quarterly Journal of the Royal Meteorological Society 1990; 117:365-384.
3. Cane MA, Miller RN, Tang B, Hackert EC, Busalacchi AJ. Mapping tropical Pacific sea level: data assimilation via reduced state Kalman filter. Journal of Geophysical Research 1996; 101:599-617.
4. Voutilainen A, Pyhälahti T, Kallio K, Pulliainen J, Haario H, Kaipio J. A filtering approach for estimating lake water quality from remote sensing data. International Journal of Applied Earth Observation and Geoinformation 2007; 9(1):50-64. ISSN: 0303-2434.
5. Fisher M. Development of a simplified Kalman filter. ECMWF Technical Memorandum 260, 1998.
6. Gejadze I-Y, Le Dimet FX, Shutyaev V. On analysis error covariances in variational data assimilation. SIAM Journal on Scientific Computing 2008; 30:1847-1874.
7. Tian X, Xie Z, Dai A. An ensemble-based explicit four-dimensional variational assimilation method. Journal of Geophysical Research 2008; 113:D21124-1-D21124-13.
8. Veersé F, Auroux D, Fisher M. Limited-memory BFGS diagonal predonditioners for a data assimilation problem in meteorology. Optimization and Engineering 2000; 1:323-339.
9. Fisher M, Andersson E. Developments in 4D-var and Kalman filtering. ECMWF Technical Memorandum 347, 2001.
10. Evensen G. Sequential data assimilation with a non-linear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. Journal of Geophysical Research 1994; 99:10143-10162.
11. Auvinen H, Bardsley JM, Haario H, Kauranne T. Large-scale Kalman filtering using the limited memory BFGS method, 2008; submitted.
12. Yang W, Navon IM, Courtier P. A new Hessian preconditioning method applied to variational data assimilation experiments using NASA general circulation model. Monthly Weather Review 1996; 124:1000-1017.
13. Fisher Michael, Courtier Philippe. Estimating the covariance matrices of analysis and forecast error in variational data assimilation. ECMWF Technical Memorandum 220, 1995.
14. Veersé F. Variable-storage quasi-Newton operators as inverse forecast/analysis error covariance matrices in variational data assimilation. INRIA Report No 3685, 26 April 1999.
15. Veersé F. Variable-storage quasi-Newton operators for modeling error covariances. Proceedings of the Third WMO International Symposium on Assimilation of Observations in Meteorology and Oceanography, Que., Canada. World Meteorological Organization: Geneva, 7-11 June 1999.
16. LeDimet F-X, Talagrand O. Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects. Tellus A Series 1986; 38:97-110.
17. Auvinen H, Haario H, Kauranne T. Optimal approximation of Kalman filtering with a temporally local 4D-Var in operational weather forecasting. In Proceedings of the 11th ECMWF Workshop on Use of High-performance Computing in Meteorology, Zwiefelhofer W, Mozdzynski G (eds). World Scientific: Singapore, 2005.
18. Nocedal Jorge, Wright Stephen. Numerical Optimization. Springer: Berlin, 1999.
19. Derber J. A variational continuous assimilation technique. Monthly Weather Review 1989; 117:2437-2446.
20. Greenstadt J. Variations on variable-metric methods. Mathematics of Computation 1970; 24:1-22.
21. Lorenz EN. Predictability: a problem partly solved. Proceedings of the Seminar on Predictability, ECMWF, vol. 1, Reading, Berkshire, U.K., 1996; 1-18.
22. Lorenz EN, Emanuel KA. Optimal sites for supplementary weather observations: simulation with a small model. Journal of Atmospheric Science 1998; 399-414.
23. Leutbecher M. A Data Assimilation Tutorial Based on the Lorenz-95 System, European Centre for Medium-Range Weather Forecasts Web Tutorial. Available from: www.ecmwf.int/newsevents/training/lecture_notes/pdffiles/ASSIM/Tutorial.pdf.
24. Zupanski M. Maximum likelihood ensemble filter: theoretical aspect. Monthly Weather Review 2005; 133: 1710-1726.
25. Rabier F, Järvinen H, Klinker E, Mahfouh JF, Simmons A. The ECMWF operational implementation of four dimensional variational assimilation. Part I: experimental results with simplified physics. Quarterly Journal of the Royal Meteorological Society 2000; 126:1143-1170.
26. Auvinen H, Haario H, Kauranne T. Variational Kalman filtering on parallel computers. In Proceedings of the 12th ECMWF Workshop on Use of High-Performance Computing in Meteorology, Zwiefelhofer W, Mozdzynski G (eds). World Scientific: Singapore, 2007.
27. Zhang F, Zhang M, Hansen JA. Coupling ensemble Kalman filter with four-dimensional data assimilation. Advances in Atmospheric Sciences 2009; 26:1-8.
28. Hamill TM, Snyder C. A hybrid ensemble Kalman filter-3D variational analysis scheme. Monthly Weather Review 2000; 128:2905-2919.
29. Nocedal J. Updating quasi-Newton matrices with limited storage. Mathematics of Computation 1980; 35(151): 773-782.
30. Byrd RH, Nocedal J, Schnabel RB. Representations of quasi-Newton matrices and their use in limited memory methods. Mathematical Programming 1994; 63(1-3):129-156.