A localized particle filter for high-dimensional nonlinear systems

Monthly Weather Review

Volumn 144, Issue 1, 2016, Pages 59-76

  Poterjoy, Jonathan ^a  

^a NATIONAL CENTER FOR ATMOSPHERIC RESEARCH   (United States)

Author keywords

Bayesian methods; Filtering techniques; Kalman filters; Mathematical and statistical techniques

Indexed keywords

BANDPASS FILTERS; BAYESIAN NETWORKS; DISTRIBUTED COMPUTER SYSTEMS; DYNAMICAL SYSTEMS; KALMAN FILTERS; PROBABILITY DENSITY FUNCTION; PROBABILITY DISTRIBUTIONS; SIGNAL FILTERING AND PREDICTION; TARGET TRACKING;

BAYESIAN METHODS; ENSEMBLE KALMAN FILTER; FILTERING TECHNIQUE; HIGH-DIMENSIONAL NONLINEAR SYSTEMS; LOCALIZATION FUNCTIONS; MONTE-CARLO APPROXIMATIONS; PROBABILITY DENSITIES; STATISTICAL TECHNIQUES;

MONTE CARLO METHODS;

BAYESIAN ANALYSIS; DATA ASSIMILATION; ENSEMBLE FORECASTING; GAUSSIAN METHOD; KALMAN FILTER; MONTE CARLO ANALYSIS; WEATHER FORECASTING;

EID: 84957812807     PISSN: 00270644     EISSN: 15200493     Source Type: Journal    
DOI: 10.1175/MWR-D-15-0163.1     Document Type: Article

References (55)

1. Ades, M., and P. J. van Leeuwen, 2015: The equivalent-weights particle filter in a high-dimensional system. Quart. J. Roy. Meteor. Soc., 141, 484-503, doi:10.1002/qj.2370.
2. Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from ensemble model integrations. J. Climate, 9, 1518-1530, doi:10.1175/1520-0442(1996)009<1518:AMFPAE>2.0.CO;2.
3. Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884-2903, doi:10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2.
4. Anderson, J. L., 2007: An adaptive covariance inflation error correction algorithm for ensemble filters. Tellus, 59A, 210-224, doi:10.1111/j.1600-0870.2006.00216.x.
5. Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72-83, doi:10.1111/j.1600-0870.2008.00361.x.
6. Anderson, J. L., 2010: A non-Gaussian ensemble filter update for data assimilation. Mon. Wea. Rev., 138, 4186-4198, doi:10.1175/2010MWR3253.1.
7. Anderson, J. L., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: Acommunity facility. Bull. Amer. Meteor. Soc., 90, 1283-1296, doi:10.1175/2009BAMS2618.1.
8. Bengtsson, T., C. Snyder, and D. Nychka, 2003: Toward a nonlinear ensemble filter for high-dimensional systems. J. Geophys. Res., 108, 8775, doi:10.1029/2002JD002900.
9. Bengtsson, T., P. Bickel, and B. Li, 2008: Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems. Probability and Statistics: Essays in Honor of David A. Freedman, D. Nolan and T. Speed, Eds., Vol. 2, Institute of Mathematical Statistics, 316-334.
10. Bickel, P., B. Li, and T. Bengtsson, 2008: Sharp failure rates for the bootstrap particle filter in high dimensions. Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, B. Clarke and S. Ghosal, Eds., Vol. 3, Institute of Mathematical Statistics, 318-329.
11. Bishop, C. H., and D. Hodyss, 2011: Adaptive ensemble covariance localization in ensemble 4D-VAR state estimation. Mon. Wea. Rev., 139, 1241-1255, doi:10.1175/2010MWR3403.1.
12. Buehner, M., 2005: Ensemble-derived stationary and flowdependent background-error covariances: Evaluation in a quasi-operational NWP setting. Quart. J. Roy. Meteor. Soc., 131, 1013-1043, doi:10.1256/qj.04.15.
13. Buehner, M., P. L. Houtekamer, C. Charette, H. Mitchell, and B. He, 2010: Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part II: One-month experiments with real observations. Mon. Wea. Rev., 138, 1567-1586, doi:10.1175/2009MWR3158.1.
14. Chorin, A., M. Morzfeld, and X. Tu, 2010: Implicit particle filters for data assimilation. Commun. Appl. Math. Comput. Sci., 5, 221-240, doi:10.2140/camcos.2010.5.221.
15. Clayton, A. M., A. C. Lorenc, and D. M. Barker, 2013: Operational implementation of a hybrid ensemble/4D-Var global data assimilation system at the Met Office. Quart. J. Roy. Meteor. Soc., 139, 1445-1461, doi:10.1002/qj.2054.
16. Doucet, A., N. de Freitas, and N. Gordon, Eds., 2001: An introduction to sequential Monte Carlo methods. Sequential Monte Carlo Methods in Practice, Springer-Verlag, 3-14, doi:10.1007/978-1-4757-3437-9_1.
17. Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 10 143-10 162, doi:10.1029/94JC00572.
18. Evensen, G., and P. J. van Leeuwen, 2000: An ensemble Kalman smoother for nonlinear dynamics. Mon. Wea. Rev., 128, 1852-1867, doi:10.1175/1520-0493(2000)128<1852:AEKSFN>2.0.CO;2.
19. Frei, M., and H. R. Künsch, 2013: Bridging the ensemble Kalman and particle filters. Biometrika, 100, 781-800, doi:10.1093/biomet/ast020.
20. Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723-757, doi:10.1002/qj.49712555417.
21. Gordon, N. J., D. J. Salmond, and A. F. M. Smith, 1993: Novel approach to nonlinear/non-Gaussian state estimation. IEE Proc., F, Radar Signal Process., 140, 107-113, doi:10.1049/ip-f-2.1993.0015.
22. Haerter, J. O., S. Hagemann, C. Moseley, and C. Piani, 2011: Climate model bias correction and the role of timescales. Hydrol. Earth Syst. Sci., 15, 1065-1079, doi:10.5194/hess-15-1065-2011.
23. Hamill, T. M., and S. J. Colucci, 1996: Random and systematic error in NMC's short-range Eta ensembles. Preprints, 13th Conf. on Probability and Statistics in the Atmospheric Sciences, San Francisco, CA, Amer. Meteor. Soc., 51-56.
24. Hamill, T. M., and C. Snyder, 2000: A hybrid ensemble Kalman filter-3D variational analysis scheme. Mon. Wea. Rev., 128, 2905-2919, doi:10.1175/1520-0493(2000)128<2905:AHEKFV>2.0.CO;2.
25. Hamill, T. M., and J. S. Whitaker, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 2776-2790, doi:10.1175/1520-0493(2001)129<2776:DDFOBE>2.0.CO;2.
26. Houtekamer, P. L., and H. L.Mitchell, 1998:Data assimilation using an ensemble Kalman filter technique. Mon.Wea. Rev., 126, 796-811, doi:10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.
27. Houtekamer, P. L., and H. L.Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123-137, doi:10.1175/1520-0493(2001)129<0123:ASEKFF>2.0.CO;2.
28. Ines, A. V. M., and J. W. Hansen, 2006: Bias correction of daily GCM rainfall for crop simulation studies. Agric. For. Meteor., 138, 44-53, doi:10.1016/j.agrformet.2006.03.009.
29. Kuhl, D. D., T. E. Rosmond, C. H. Bishop, J. McLay, and N. L. Baker, 2013: Comparison of hybrid ensemble/4DVar and 4DVar within the NAVDAS-AR data assimilation framework. Mon. Wea. Rev., 141, 2740-2758, doi:10.1175/MWR-D-12-00182.1.
30. Lei, J., and P. Bickel, 2011:A moment matching ensemble filter for nonlinear non-Gaussian data assimilation. Mon. Wea. Rev., 139, 3964-3973, doi:10.1175/2011MWR3553.1.
31. Lei, J., P. Bickel, and C. Snyder, 2010: Comparison of ensemble Kalman filters under non-Gaussianity. Mon. Wea. Rev., 138, 1293-1306, doi:10.1175/2009MWR3133.1.
32. Liu, C., Q. Xiao, and B. Wang, 2008: An ensemble-based fourdimensional variational data assimilation scheme. Part I: Technical formulation and preliminary test. Mon. Wea. Rev., 136, 3363-3373, doi:10.1175/2008MWR2312.1.
33. Lorenc, A. C., 2003: The potential of the ensemble Kalman filter for NWP: A comparison with 4D-Var. Quart. J. Roy. Meteor. Soc., 129, 3183-3203, doi:10.1256/qj.02.132.
34. Lorenz, E. N., 1996: Predictability: A problem partly solved. Proc. Seminar on Predictability, Vol. 1, Reading, United Kingdom, ECMWF, 1-18.
35. Lorenz, E. N., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399-414, doi:10.1175/1520-0469(1998)055<0399:OSFSWO>2.0.CO;2.
36. Majda, A. J., D. Qi, and T. P. Sapsis, 2014: Blended particle filters for large-dimensional chaotic dynamical systems. Proc. Natl. Acad. Sci. USA, 111, 7511-7516, doi:10.1073/pnas.1405675111.
37. McGinnis, S., D. Nychka, and L. O. Mearns, 2015: A new distribution mapping technique for climate model bias correction. Machine Learning and Data Mining Approaches to Climate Science, V. Lakshmananet al., Eds., Springer, 91-99, doi:10.1007/978-3-319-17220-0_9.
38. Mitchell, H. L., P. L. Houtekamer, and G. Pellerin, 2002: Ensemble size, balance, and model-error representation in an ensemble Kalman filter. Mon. Wea. Rev., 130, 2791-2808, doi:10.1175/1520-0493(2002)130<2791:ESBAME>2.0.CO;2.
39. Miyoshi, T., K. Kondo, and T. Imamura, 2014: The 10, 240-member ensemble Kalman filtering with an intermediate AGCM. Geophys. Res. Lett., 41, 5264-5271, doi:10.1002/2014GL060863.
40. Nakano, S., G. Ueno, and T. Higuchi, 2007: Merging particle filter for sequential data assimilation. Nonlinear Processes Geophys., 14, 395-408, doi:10.5194/npg-14-395-2007.
41. Piani, C., J. O. Haerter, and E. Coppola, 2010: Statistical bias correction for daily precipitation in regional climate models over Europe. Theor. Appl. Climatol., 99, 187-192, doi:10.1007/s00704-009-0134-9.
42. Pires, C. A., O. Talagrand, and M. Bocquet, 2010: Diagnosis and impacts of non-Gaussianity of innovations in data assimilation. Physica D, 239, 1701-1717, doi:10.1016/j.physd.2010.05.006.
43. Reich, S., 2013: A nonparametric ensemble transform method for Bayesian inference. SIAM J. Sci. Comput., 35, A2013-A2024, doi:10.1137/130907367.
44. Slivinski, L., E. Spiller, A. Apte, and B. Sandstede, 2015: A hybrid particle-ensemble Kalman filter for Lagrangian data assimilation. Mon. Wea. Rev., 143, 195-211, doi:10.1175/MWR-D-14-00051.1.
45. Snyder, C., T. Bengtsson, P. Bickel, and J. Anderson, 2008: Obstacles to high-dimensional particle filtering. Mon. Wea. Rev., 136, 4629-4640, doi:10.1175/2008MWR2529.1.
46. Stewart, L. M., S. L. Dance, and N. K. Nichols, 2008: Correlated observation errors in data assimilation. Int. J. Numer. Methods Fluids, 56, 1521-1527, doi:10.1002/fld.1636.
47. Talagrand, O., R. Vautard, and B. Strauss, 1997: Evaluation of probabilistic prediction systems. Proc. ECMWF Workshop on Predictability, Reading, United Kingdom, ECMWF, 1-25.
48. van Leeuwen, P. J., 2003: A variance-minimizing filter for largescale applications.Mon.Wea. Rev., 131, 2071-2084, doi:10.1175/1520-0493(2003)131<2071:AVFFLA>2.0.CO;2.
49. van Leeuwen, P. J., 2009: Particle filtering in geophysical systems. Mon. Wea. Rev., 137, 4089-4114, doi:10.1175/2009MWR2835.1.
50. van Leeuwen, P. J., 2010: Nonlinear data assimilation in geosciences: An extremely efficient particle filter. Quart. J. Roy. Meteor. Soc., 136, 1991-1999, doi:10.1002/qj.699.
51. Wang, X., and T. Lei, 2014: GSI-based four-dimensional ensemblevariational (4DEnsVar) data assimilation: Formulation and single-resolution experiments with real data for NCEP Global Forecast System. Mon. Wea. Rev., 142, 3303-3325, doi:10.1175/MWR-D-13-00303.1.
52. Whitaker, J. S., and T. M. Hamill, 2012: Evaluating methods to account for system errors in ensemble data assimilation. Mon. Wea. Rev., 140, 3078-3089, doi:10.1175/MWR-D-11-00276.1.
53. Xiong, X., I. M. Navon, and B. Uzunoglu, 2006: A note on the particle filter with posterior Gaussian resampling. Tellus, 58A, 456-460, doi:10.3402/tellusa.v58i4.14798.
54. Zhang, F., C. Snyder, and J. Sun, 2004: Impacts of initial estimate and observation availability on convective-scale data assimilation with an ensemble Kalman filter.Mon. Wea. Rev., 132, 1238-1253, doi:10.1175/1520-0493(2004)132<1238:IOIEAO>2.0.CO;2.
55. Zhang, F., M. Zhang, and J. A. Hansen, 2009: Coupling ensemble Kalman filter with four-dimensional variational data assimilation. Adv. Atmos. Sci., 26, 1-8, doi:10.1007/s00376-009-0001-8.