Obstacles to high-dimensional particle filtering

Monthly Weather Review

Volumn 136, Issue 12, 2008, Pages 4629-4640

  Snyder, Chris ^a   Bengtsson, Thomas ^b   Bickel, Peter ^c   Anderson, Jeff ^a  

^a NATIONAL CENTER FOR ATMOSPHERIC RESEARCH   (United States)

^b LUCENT TECHNOLOGIES   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC THEORIES; ENSEMBLE KALMAN FILTERS; GAUSSIAN; GAUSSIAN ERRORS; HIGH-DIMENSIONAL; LOG LIKELIHOODS; NONLINEAR AND NON-GAUSSIAN; OBSERVATION ERRORS; PARTICLE FILTERING; PARTICLE FILTERS; PROBABILITY DISTRIBUTION FUNCTIONS; PROBLEM SIZES; STATE COMPONENTS; STATE VECTORS;

AIR FILTERS; ASYMPTOTIC ANALYSIS; CONTROL THEORY; DISTRIBUTED COMPUTER SYSTEMS; DISTRIBUTION FUNCTIONS; ERRORS; NONLINEAR FILTERING; TARGET TRACKING; TRELLIS CODES;

PROBABILITY DISTRIBUTIONS;

COMPUTER SIMULATION; DATA ASSIMILATION; ENSEMBLE FORECASTING; ERROR ANALYSIS; GAUSSIAN METHOD; KALMAN FILTER; OBSERVATIONAL METHOD; PROBABILITY DENSITY FUNCTION; THEORETICAL STUDY;

EID: 64149083162     PISSN: 00270644     EISSN: None     Source Type: Journal    
DOI: 10.1175/2008MWR2529.1     Document Type: Article

References (36)

1. Anderson, J. L., 2001: An ensemble adjustment filter for data assimilation. Mon. Wea. Rev., 129, 2884-2903.
2. Anderson, J. L., and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Mon. Wea. Rev., 127, 2741-2758.
3. Bender, M., and S. Orszag, 1978: Advanced Mathematical Methods for Scientists and Engineers. McGraw-Hill, 593 pp.
4. Bengtsson, T., C. Snyder, and D. Nychka, 2003: Toward a non-linear ensemble filter for high-dimensional systems. J. Geophys. Res., 108 (D24), 8775-8785.
5. Bengtsson, T., P. Bickel, and B. Li, 2008: Curse of dimensionality revisited: Collapse of the particle filter in very large scale systems. Probability and Statistics: Essays in Honor of David A. Freedman, D. Nolan and T. Speed, Eds., Vol. 2, Institute of Mathematical Statistics, 316-334, doi:10.1214/193940307000000518. [Available online at http://projecteuclid.org/DPubS/Repository/1.0/ Disseminate?viewbody&idpdf=1&handleeuclid.imsc/1207580091.]
6. Bickel, P., and E. Levina, 2008: Regularized estimation of large covariance matrices. Ann. Stat., 36, 199-227.
7. Bickel, P., B. Li, and T. Bengtsson, 2008: Sharp failure rates for the bootstrap particle filter in high dimensions. Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, Vol. 3, B. Clarke and S. Ghosal Eds., Institute of Mathematical Statistics, 318-329, doi:10.1214/074921708000000228.
8. Chin, T. M., M. J. Turmon, J. B. Jewell, and M. Ghil, 2007: An ensemble-based smoother with retrospectively updated weights for highly nonlinear systems. Mon. Wea. Rev., 135, 186-202.
9. David, H. A., and H. N. Nagaraja, 2003: Order Statistics. 3rd ed. John Wiley and Sons, 458 pp.
10. Doucet, A., N. de Freitas, and N. Gordon, 2001: An introduction to sequential Monte Carlo methods. Sequential Monte Carlo Methods in Practice, A. Doucet, N. de Freitas, and N. Gordon, Eds., Springer-Verlag, 2-14.
11. Durret, R., 2005: Probability: Theory and Examples. 3rd ed. Duxbury Press, 512 pp.
12. Furrer, R., and T. Bengtsson, 2007: Estimation of high-dimensional prior and posteriori covariance matrices in Kalman filter variants. J. Multivar. Anal., 98 (2), 227-255.
13. Gordon, N. J., D. J. Salmond, and A. F. M. Smith, 1993: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEEE Proc., 140, 107-113.
14. Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550-560.
15. Hamill, T. M., J. S. Whitaker, and C. Snyder, 2001: Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Wea. Rev., 129, 2776-2790.
16. Harlim, J., and B. R. Hunt, 2007: A non-Gaussian ensemble filter for assimilating infrequent noisy observations. Tellus, 59A, 225-237.
17. Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796-811.
18. Houtekamer, P. L., and H. L. Mitchell, 2001: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev., 129, 123-137.
19. Ide, K., P. Courtier, M. Ghil, and A. C. Lorenc, 1997: Unified notation for data assimilation: operational, sequential, and variational. J. Meteor. Soc. Japan, 75 (Special Issue), 181-189.
20. Keppenne, C. L., M. M. Rienecker, N. P. Kurkowski, and D. A. Adamec, 2005: Ensemble Kalman filter assimilation of temperature and altimeter data with bias correction and application to seasonal prediction. Nonlinear Processes Geophys., 12, 491-503.
21. Kim, S., G. L. Eyink, J. M. Restrepo, F. J. Alexander, and G. Johnson, 2003: Ensemble filtering for nonlinear dynamics. Mon. Wea. Rev., 131, 2586-2594.
22. Liu, J. S., 2001: Monte Carlo Strategies in Scientific Computing. Springer-Verlag, 364 pp.
23. Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130-148.
24. Lorenz, E. N., 1996: Predictability: A problem partly solved. Proc. Seminar on Predictability, Vol. 1, Reading, Berkshire, United Kingdom, ECMWF, 1-18.
25. Moradkhani, H., K.-L. Hsu, H. Gupta, and S. Sorooshian, 2005: Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter. Water Resour. Res., 41, W05012, doi:10.1029/2004WR003604.
26. Nakano, S., G. Ueno, and T. Higuchi, 2007: Merging particle filter for sequential data assimilation. Nonlinear Processes Geophys., 14, 395-408.
27. Pham, D. T., 2001: Stochastic methods for sequential data assimilation in strongly nonlinear systems. Mon. Wea. Rev., 129, 1194-1207.
28. Pitt, M. K., and N. Shephard, 1999: Filtering via simulation: Auxilliary particle filters. J. Amer. Stat. Assoc., 94, 590-599.
29. Reichle, R. H., D. B. McLaughlin, and D. Entekhabi, 2002: Hydrologic data assimilation with the ensemble Kalman filter. Mon. Wea. Rev., 130, 103-114.
30. Silverman, B. W., 1986: Density Estimation for Statistics and Data Analysis. Chapman and Hall, 175 pp.
31. Smith, K. W., 2007: Cluster ensemble Kalman filter. Tellus, 59A, 749-757.
32. Snyder, C., and F. Zhang, 2003: Assimilation of simulated Doppler radar observations with an ensemble Kalman filter. Mon. Wea. Rev., 131, 1663-1677.
33. van Leeuwen, P. J., 2003: A variance-minimizing filter for largescale applications. Mon. Wea. Rev., 131, 2071-2084.
34. Whitaker, J. S., G. P. Compo, X. Wei, and T. M. Hamill, 2004: Reanalysis without radiosondes using ensemble data assimilation. Mon. Wea. Rev., 132, 1190-1200.
35. Xiong, X., I. M. Navon, and B. Uzunoglu, 2006: A note on the particle filter with posterior Gaussian resampling. Tellus, 58A, 456-460.
36. Zhou, Y., D. McLaughlin, and D. Entekhabi, 2006: Assessing the performance of the ensemble Kalman filter for land surface data assimilation. Mon. Wea. Rev., 134, 2128-2142.