Controlling overestimation of error covariance in ensemble kalman filters with sparse observations: A variance-limiting kalman filter

Monthly Weather Review

Volumn 139, Issue 8, 2011, Pages 2650-2667

  Gottwald, Georg A ^a   Mitchell, Lewis ^a   Reich, Sebastian ^b  

^a UNIVERSITY OF SYDNEY   (Australia)

^b UNIVERSITY OF POTSDAM   (Germany)

Author keywords

Data assimilation; Ensembles; Error analysis; Filtering techniques; Kalman filters; Variational analysis

Indexed keywords

DATA ASSIMILATION; ENSEMBLE KALMAN FILTER; ENSEMBLES; ERROR COVARIANCES; FILTERING TECHNIQUES; OBSERVATIONAL ERRORS; OPTIMAL PERFORMANCE; PARTIAL OBSERVATION; TOY MODELS; UNOBSERVABLE; VARIATIONAL ANALYSIS;

DATA PROCESSING; ERROR ANALYSIS; METADATA;

KALMAN FILTERS;

CLIMATE VARIATION; COVARIANCE ANALYSIS; DATA ASSIMILATION; ENSEMBLE FORECASTING; ERROR ANALYSIS; KALMAN FILTER; NUMERICAL MODEL;

EID: 80052422363     PISSN: 00270644     EISSN: None     Source Type: Journal    
DOI: 10.1175/2011MWR3557.1     Document Type: Article

References (53)

1. Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884-2903.
2. Anderson, J. L., 2007: An adaptive covariance inflation error correction algorithm for ensemble filters. Tellus, 59A, 210-224.
3. Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72-83.
4. Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 2741-2758.
5. Bengtsson, L., and Coauthors, 2007: The need for a dynamical climate reanalysis. Bull. Amer. Meteor. Soc., 88, 495-501.
6. Bergemann, K., G. Gottwald, and S. Reich, 2009: Ensemble propagation and continuous matrix factorization algorithms. Quart. J. Roy. Meteor. Soc., 135, 1560-1572.
7. Bocquet, M., C. A. Pires, and L. Wu, 2010: Beyond Gaussian statistical modeling in geophysical data assimilation. Mon. Wea. Rev., 138, 2997-3023.
8. Buizza, R., M. Miller, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 125, 2887-2908.
9. Charron, M., G. Pellerin, L. Spacek, P. L. Houtekamer,N. Gagnon, H. L. Mitchell, and L. Michelin, 2010: Toward random sampling of model error in the Canadian ensemble prediction system. Mon. Wea. Rev., 138, 1877-1901.
10. Compo, G. P., and Coauthors, 2011: The twentieth century reanalysis project. Quart. J. Roy. Meteor. Soc., 137, 1-28, doi:10.1002/qj.776.
11. Durran, D. R., 1999: Numerical Methods for Wave Equations in Geophysical Fluid Dynamics. Springer, 482 pp.
12. Eckermann, S. D., and Coauthors, 2009: High-altitude data assimilation system experiments for the northern summer mesosphere season of 2007. J. Atmos. Sol.-Terr. Phys., 71, 531-551.
13. Ehrendorfer, M., 2007: A review of issues in ensemble-based Kalman filtering. Meteor. Z., 16, 795-818.
14. Evensen, G., 2006: Data Assimilation: The Ensemble Kalman Filter. Springer, 280 pp.
15. Fisher, M., M. Leutbecher, and G. A. Kelly, 2005: On the equivalence between Kalman smoothing and weak-constraint four-dimensional variational data assimilation. Quart. J. Roy. Meteor. Soc., 131, 3235-3246.
16. Gardiner, C. W., 2004: Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences. 3rd ed. Springer, 415 pp.
17. Golub, G. H., and C. F. Van Loan, 1996: Matrix Computations. 3rd ed. The Johns Hopkins University Press, 728 pp.
18. Gottwald, G. A., and I. Melbourne, 2005: Testing for chaos in deterministic systems with noise. Physica D, 212, 100-110.
19. Hamill, T. M., and J. S. Whitaker, 2005: Accounting for the error due to unresolved scales in ensemble data assimilation: A comparison of different approaches. Mon. Wea. Rev., 133, 3132-3147.
20. Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 2776-2790.
21. Harlim, J., and A. J. Majda, 2010: Catastrophic filter divergence in filtering nonlinear dissipative systems. Comm.Math. Sci., 8, 27-43.
22. Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796-811.
23. Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123-136.
24. Houtekamer, P. L., and H. L. Mitchell, 2005: Ensemble Kalman filtering. Quart. J. Roy. Meteor. Soc., 131, 3269-3289.
25. Houtekamer, P. L., H. L. Mitchell, G. Pellerin, M. Buehner, M. Charron, L. Spacek, and B. Hansen, 2005: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Wea. Rev., 133, 604-620.
26. Houtekamer, P. L., H. L. Mitchell, and X. Deng, 2009: Model error representation in an operational ensemble Kalman filter. Mon. Wea. Rev., 137, 2126-2143.
27. Ide, K., P. Courtier, M. Ghil, and A. C. Lorenc, 1997: Unified notation for data assimilation: Operational, sequential and variational. J. Meteor. Soc. Japan, 75, 181-189.
28. Kalnay, E., 2002: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, 364 pp.
29. Kepert, J. D., 2004: On ensemble representation of the observation- error covariances in the ensemble Kalman filter. Ocean Dyn., 54, 561-569.
30. Kepert, J. D., 2009: Covariance localisation and balance in an Ensemble Kalman Filter. Quart. J. Roy. Meteor. Soc., 135, 1157-1176.
31. Leimkuhler, B., and S. Reich, 2005: Simulating Hamiltonian Dynamics. Cambridge University Press, 379 pp.
32. Li, H., E. Kalnay, and T. Miyoshi, 2009: Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Quart. J. Roy. Meteor. Soc., 135, 523-533.
33. Liu, J., E. J. Fertig, H. Li, E. Kalnay, B. R. Hunt, E. J. Kostelich, I. Szunyogh, and R. Todling, 2008: Comparison between local ensemble transform Kalman filter and PSAS in the NASA finite volume GCM-Perfect model experiments. Nonlinear Processes Geophys., 15, 645-659.
34. Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWP-A comparison with 4DVAR. Quart. J. Roy. Meteor. Soc., 129, 3183-3203.
35. Lorenz, E. N., 1996: Predictability-A problem partly solved. Predictability, T. Palmer, Ed., European Centre for Medium- Range Weather Forecasts, 1-18.
36. Lorenz, E. N., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399-414.
37. Mitchell, H. L., and P. L. Houtekamer, 2000: Anadaptive ensemble Kalman filter. Mon. Wea. Rev., 128, 416-433.
38. Neef, L., S. M. Polavarapu, and T. G. Shepherd, 2006: Fourdimensional data assimilation and balanced dynamics. J. Atmos. Sci., 63, 1840-1850.
39. Orrell, D., and L. Smith, 2003: Visualising bifurcations in high dimensional systems: The spectral bifurcation diagram. Int. J. Bifurcation Chaos, 13, 3015-3028.
40. Ott, E., B. Hunt, I. Szunyogh, A. Zimin, E. Kostelich, M. Corrazza, E. Kalnay, and J. Yorke, 2004: A local ensemble Kalman filter for atmospheric data assimilation. Tellus, 56A, 415-428.
41. Pires, C. A., O. Talagrand, and M. Bocquet, 2010: Diagnosis and impacts of non-Gaussianity of innovations in data assimilation. Physica D, 239, 1701-1717.
42. Polavarapu, S., T. G. Shepherd, Y. Rochon, and S. Ren, 2005: Some challenges of middle atmosphere data assimilation. Quart. J. Roy. Meteor. Soc., 131, 3513-3527.
43. Sankey, D., S. Ren, S. Polavarapu, Y. Rochon, Y. Nezlin, and S. Beagley, 2007: Impact of data assimilation filtering methods on the mesosphere. J. Geophy. Res., 112, D24104, doi:10.1029/2007JD008885.
44. Sasaki, Y., 1970: Some basic formalisms on numerical variational analysis. Mon. Wea. Rev., 98, 875-883.
45. Shutts, G. J., 2005: A stochastic kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 131, 3079-3102.
46. Simon, D. J., 2006: Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches. John Wiley & Sons, Inc., 552 pp.
47. Szunyogh, I.,E.Kostelich,G.Gyarmati,D. J. Patil,B.Hunt, E.Kalnay, E. Ott, and J. Yorke, 2005: Assessing a local ensemble Kalman filter: Perfect model experiments with the National Centers for Environmental Prediction global model. Tellus, 57A, 528-545.
48. Tippett, M. K., J. L. Anderson, C.H. Bishop, T. M. Hamill, and J. S. Whitaker, 2003: Ensemble square root filters. Mon. Wea. Rev., 131, 1485-1490.
49. Wang, X., C. H. Bishop, and S. J. Julier, 2004: Which is better, an ensemble of positive-negative pairs or a centered spherical simplex ensemble? Mon. Wea. Rev., 132, 1590-1605.
50. Whitaker, J. S., G. P. Compo, X. Wei, and T. M. Hamill, 2004: Reanalysis without radiosondes using ensemble data assimilation. Mon. Wea. Rev., 132, 1190-1200.
51. Whitaker, J. S., G. P. Compo, and J. N. Thépaut, 2009: A comparison of variational and ensemble-based data assimilation systems for reanalysis of sparse observations. Mon. Wea. Rev., 137, 1991-1999.
52. Wolfram Research, Inc., 2008: Mathematica Version 7.0.Wolfram Research, Inc., Champaign, IL. [Available online at http://www.wolfram.com/mathematica/.]
53. Zupanski, D., 1997: A general weak constraint applicable to operational 4DVar data assimilation systems. Mon. Wea. Rev., 125, 2274-2292.