Quantifying Bayesian filter performance for turbulent dynamical systems through information theory

Communications in Mathematical Sciences

Volumn 12, Issue 5, 2014, Pages 901-978

  Branicki, M ^a   Majda, A J ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

Information theory; Kalman filter; Turbulent dynamical systems; Uncertainty quantification

Indexed keywords

EID: 84897875247     PISSN: 15396746     EISSN: 19450796     Source Type: Journal    
DOI: 10.4310/CMS.2014.v12.n5.a6     Document Type: Article

References (72)

1. R.V. Abramov and A.J. Majda, Blended response algorithms for linear fluctuation-dissipation for complex nonlinear dynamical systems, Nonlin., 20(12), 2793-2821, 2007.
2. B. Anderson and J. Moore, Optimal Filtering, Prentice-Hall, Englewood Cliffs, NJ, 1979.
3. J.L. Anderson, An ensemble adjustment Kalman filter for data assimilation, Mon. Wea. Rev., 129, 2884-2903, 2001.
4. J.L. Anderson, A local least squares framework for ensemble filtering, Mon. Wea. Rev., 131(4), 634-642, 2003.
5. A. Bain and D. Crisan, Fundamentals of Stochastic Filtering, Stochastic Modelling and Applied Probability, Springer, 60, 2009.
6. T. Bengtsson, P. Bickel, and B. Li, Essays in Honor of David A. Freedman, IMS Lecture Notes-Monograph Series in Probability and Statistics, Curse of dimensionailty revisited: Collapse of the particle filter in very large scale systems, Inst. Math. Sci., 2, 316-334, 2008.
7. T. Berger, Rate Distortion Theory, Englewood Clifs, NJ: Prentice Hall, 1971.
8. L.M. Berliner, R.F. Milliff, and C.K. Wikle, Bayesian hierarchical modeling of air-sea interaction, J. Geophys. Res., 108(C4), 3104, 2003.
9. P. Bickel, B. Li, and T. Bengtsson, Essays in Honor of J. K. Gosh, IMS Lecture Notes-Monograph Series, ch. Sharp Failure Rates for the Bootstrap Filter in High Dimensions, Inst. Math. Sci., 318-329, 2008.
10. M. Branicki, N. Chen, and A.J. Majda, Non-Gaussian Test Models for Prediction and State Estimation with Model Errors, Chinese Ann. Math., 34B(1), 29-64, 2013.
11. M. Branicki, B. Gershgorin, and A.J. Majda, Filtering skill for turbulent signals for a suite of nonlinear and linear Kalman filters, J. Comput. Phys., 231, 1462-1498, 2012.
12. M. Branicki and A.J. Majda, Dynamic stochastic superresolution of sparsely observed dynamical systems, J. Comput. Phys., 241, 333-363, 2013.
13. M. Branicki and A.J. Majda, Quantifying uncertainty for statistical predicions with model errors in non-Guassian models with intermittency, Nonlin., 25, 2543-2578, 2012.
14. R.S. Bucy and P.D. Joseph, Filtering for Stochastic Processes with Applications to Guidance, New York: Wiley, 1968.
15. E. Castronovo, J. Harlim, and A. Majda, Mathematical criteria for filtering complex systems: Plentiful observations, J. Comput. Phys., 227, 3678-3714, 2008.
16. A.J. Chorin and P. Krause, Dimensional reduction for a bayesian filter, Proc. Natl. Acad. Sci. USA, 101, 15013-15017, 2004.
17. C. Chui and G. Chen, Kalman Filtering with Real-time Applications, Springer, New York, 1999.
18. S. Cohn and D. Dee, Observability of discretized partial differential equations, SIAM J. Numer. Anal., 25(3), 586-617, 1988.
19. S.L. Cotter, M. Dashti, and A.M. Stuart, Approximation of Bayesian inverse problems, SIAM J. Numer. Anal., 48, 322-345, 2010.
20. T.A. Cover and J.A. Thomas, Elements of Information Theory, Second Edition, Wiley-Interscience, Hoboken, 2006.
21. D.P. Dee and A.M. Da Silva, Data assimilation in the presence of forecast bias, Quart. J. Roy. Meteor. Soc., 124, 269-295, 1998.
22. D.P. Dee and R. Todling, Data assimilation in the presence of forecast bias: The GEOS moisture analysis, Mon. Wea. Rev., 128(9), 3268-3282, 2000.
23. T. DelSole, Stochastic Models of Quasigeostrophic Turbulence, Surveys in Geophys., 25(2), 107-149, 2004.
24. T.E. Duncan, On the calculation of mutual information, SIAM J. Appl. Math., 19(1), 215-220, 1970.
25. B. Farrell and P. Ioannou, Distributed forcing of forecast and assimilation error systems, J. Atmos. Sci., 62, 460-475, 2005.
26. B. Friedland, Treatment of bias in recursive filtering, IEEE Trans. Automatic Control, AC-14, 359-367, 1969.
27. B. Friedland, Estimating sudden changes of biases in linear dynamical systems, IEEE Trans. Automatic Control, AC-27, 237-240, 1982.
28. J.I. Galdos and D.E. Gustafson, Information and distortion in reduced-order filter design, IEEE Trans. Informat. Theory, 23(2), 183-194, 1977.
29. R.G. Gallager, Information Theory and Reliable Communication, Wiley, New York, 1968.
30. B. Gershgorin, J. Harlim, and A.J. Majda, Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation, J. Comput. Phys., 229, 32-57, 2010.
31. B. Gershgorin, J. Harlim, and A.J. Majda, Test models for improving filtering with model errors through stochastic parameter estimation, J. Comput. Phys., 229, 1-31, 2010.
32. B. Gershgorin and A.J. Majda, A nonlinear test model for filtering slow-fast systems, Comm. Math. Sci., 6, 611-649, 2008.
33. B. Gershgorin and A.J. Majda, Filtering a nonlinear slow-fast system with strong fast forcing, Comm. Math. Sci., 8, 67-92, 2009.
34. B. Gershgorin and A.J. Majda, Filtering a statistically exactly solvable test model for turbulent tracers from partial observations, J. Comput. Phys., 230, 1602-1638, 2011.
35. A.L. Gibbs and F.E. Su, On choosing and bounding probability metrics, Intern. Stat. Rev., 70(3), 419-435, 2002.
36. J. Harlim and A.J. Majda, Test models for filtering and prediction of moisture-coupled tropical waves, Q.J.R. Meteorol. Soc., 139, 119-136, 2013.
37. J. Harlim and A.J. Majda, Mathematical strategies for filtering complex systems: Mathematical strategies for filtering complex systems: Regularly spaced sparse observations, J. Comput. Phys., 227, 5304-5341, 2008.
38. J. Harlim and A.J. Majda, Filtering turbulent sparsely observed geophysical flows, Mon. Wea. Rev., 138(4), 1050-1083, 2010.
39. J. Harlim and A.J. Majda, Test models for filtering with superparameterization, Multiscale Model. Simul., 11(1), 282-308, 2013.
40. K. Haven, A.J. Majda, and R. Abramov, Quantifying predictability through information theory: Small sample estimation in a non-gaussian framework, J. Comput. Phys., 334-362, 2005.
41. A.H. Jazwinski, Stochastic Processes and Filtering Theory, Academic Press, New York, 1970.
42. T. Kailath, A general likelyhood ratio formula for random signals in gaussian noise, IEEE Trans. Informat. Theory, 15, 350-361, 1969.
43. R.E. Kalman, A new approach to linear filtering and prediction problems, J. Basic Eng. (Series D), 82, 35-45, 1960.
44. R.E. Kalman and R.S. Bucy, New results in linear filtering and prediction problems, J. Basic Eng. (Series D), 83, 95-107, 1961.
45. S.R. Keating, A.J. Majda, and K.S. Smith, New methods for estimating poleward eddy heat transport using satellite altimetry, Mon. Wea. Rev., 140, 1703-1722, 2012.
46. R. Kleeman, Measuring dynamical prediction utility using relative entropy, J. Atmos. Sci., 59, 13, 2057-2072, 2002.
47. R. Kleeman, A.J. Majda, and I. Timofeyev, Quantifying predictability in a model with statistical features of the atmosphere, Proc. Natl. Acad. Sci. USA, 99, 15291-15296, 2002.
48. S. Kravtsov, D. Kondrashov, and M. Ghil, Multilevel regression modeling of nonlinear processes: Derivation and applications to climatic variability, J. Climate, 18, 21, 4404-4424, 2005.
49. R.S. Liptser and A.N. Shiryaev, Statistics of Random Process, Second, Springer-Verlag, New York, 2001.
50. A.J. Majda, Challenges in climate science and contemporary applied mathematics, Comm. Pure Appl. Math., 65(7), 920-948, 2012.
51. A.J. Majda, R. Abramov, and B. Gershgorin, High skill in low frequency climate response through fluctuation dissipation theorems despite structural instability, Proc. Natl. Acad. Sci. USA, 107(2), 581-586, 2010.
52. A.J. Majda, R.V. Abramov, and M.J. Grote, Information Theory and Stochastics for Multiscale Nonlinear Systems, CRM Monograph Series, Americal Mathematical Society, Providence, 25, 2005.
53. A.J. Majda and B. Gershgorin, Quantifying uncertainty in climage change science through empirical information theory, Proc. Natl. Acad. Sci., 107(34), 14958-14963, 2010.
54. A. J. Majda and J. Harlim, Physics constrained nonlinear regression models for time series, Nonlin., 26(1), 201-217, 2013.
55. A.J. Majda, R. Kleeman, and D. Cai, A mathematical framework for predictability through relative entropy, Meth. Appl. Anal., 9(3), 425-444, 2002.
56. A.J. Majda and X. Wang, Linear response theory for statistical ensembles in complex systems with time-periodic forcing, Comm. Math. Sci., 8(1), 145-172, 2010.
57. A.J. Majda and Y. Yuan, Fundamental limitations of ad hoc linear and quadratic multi-level regression models for physical systems, Disc. Cont. Dyn. Sys., 4, 1333-1363, 2012.
58. A.J. Majda and M. Branicki, Lessons in uncertainty quantification for turbulent dynamical systems, Disc. Cont. Dyn. Sys., 32(9), 3133-3231, 2012.
59. A.J.Majda andM.J. Grote, Explicit off-line criteria for stable accurate time filtering of strongly unstable spatially extended systems, Proc. Natl. Acad. Sci. USA, 104, 1124-1129, 2007.
60. A.J. Majda and J. Harlim, Filtering Complex Turbulent Systems, Cambridge University Press, 2012.
61. A.J. Majda, J. Harlim, and B. Gershgorin, Mathematical strategies for filtering turbulent dynamical systems, Disc. Cont. Dyn. Sys., 27, 441-486, 2010.
62. R. Miller, E. Carter, and S. Blue, Data assimilation into nonlinear stochastic models, Tellus A, 51, 167-194, 1999.
63. S.K. Mitter and N.J. Newton, Information and entropy flow in the Kalman-Bucy filter, J. Stat. Phys., 18, 145-176, 2005.
64. N. Newton, Dual nonlinear filters and entropy production, SIAM J. Control Optim., 46(5), 1637-1663, 2007.
65. N. Newton, Interactive statistical mechanics and nonlinear filtering, J. Stat. Phys., 133, 711- 737, 2008.
66. E. Ott, B.R. Hunt, I. Szunyogh, A.V. Zimin, E.J. Kostelich, M. Corazza, E. Kalnay, D.J. Patil, and J.A. Yorke, Estimating the state of large spatio-temporally chaotic systems, Phys. Lett. A, 330, 365-370, 2004.
67. R.D. Reiss, Approximate Distributions of Order Statistics, New York, Springer-Verlag, 1989.
68. C.E. Shannon and W. Weaver, The Mathematical Theory of Communication, U. Illinois Press, 1949.
69. Ch. Snyder, T. Bengtsson, P. Bickel, and J. Anderson, Obstacles to high-dimensional particle filtering, Mon. Wea. Rev., 136, 4629-4640, 2008.
70. Y. Tomita, S. Omatu, and T. Soeda, An application of the information theory to filtering problem, Inf. Sci., 11, 13-27, 1976.
71. H.L.Widemann and E.B. Stear, Entropy analysis of estimating systems, IEEE Trans. Informat. Theory, 16, 264-270, 1970.
72. M. Zakai and J. Ziv, Lower and upper bounds on the optimal filtering error of certain diffusion processe, IEEE Trans. Informat. Theory, 18(3), 325-331, 1972.