Filtering complex turbulent systems

Filtering Complex Turbulent Systems

Volumn , Issue , 2012, Pages 1-357

  Majda, Andrew J ^a   Harlim, John ^b  

^a NEW YORK UNIVERSITY   (United States)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84926093390     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9781139061308     Document Type: Book

References (134)

1. Abramov, R. 2006. A practical computational framework for the multidimensional moment-constrained maximum entropy principle. J. Comput. Phys., 211(1), 198-209.
2. Abramov, R. V. 2007. An improved algorithm for the multidimensional moment-constrained maximum entropy problem. J. Comput. Phys., 226(1), 621-644.
3. Abramov, R. V. 2009. The multidimensional moment-constrained maximum entropy problem: A BFGS algorithm with constraint scaling. J. Comput. Phys., 228(1), 96-108.
4. Abramov, R. V. 2010. The multidimensional maximum entropy moment problem: A review on numerical methods. Comm. Math. Sci., 8(2), 377-392.
5. Abramov, R. and Majda, A. J. 2004. Quantifying uncertainty for non-Gaussian ensembles in complex systems. SIAM J. Sci. Comput., 25(2), 411-447.
6. Abramov, R. V. and Majda, A. J. 2007. Blended response algorithm for linear fluctuation-dissipation for complex nonlinear dynamical systems. Nonlinearity, 20, 2793-2821.
7. Anderson, B.D. and Moore, J. B. 1979. Optimal Filtering. Prentice-Hall, Englewood Cliffs, NJ.
8. Anderson, J. L. 2001. An ensemble adjustment Kalman filter for data assimilation. Monthly Weather Rev., 129, 2884-2903.
9. Anderson, J.L. 2003. A local least squares framework for ensemble filtering. Monthly Weather Rev., 131(4), 634-642.
10. Anderson, J. L. 2007. An adaptive covariance inflation error correction algorithm for ensemble filters. TellusA, 59, 210-224.
11. Anderson, J. L. 2010. A non-Gaussian ensemble filter update for data assimilation. Monthly Weather Rev., 138(11), 4186-4198.
12. Anderson, J. L. and Anderson, S. L. 1999. A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts. Monthly Weather Rev., 127, 2741-2758.
13. Baek, S.-J., Hunt, B. R., Kalnay, E., Ott, E. and Szunyogh, 1.2006. Local ensemble Kalman filtering in the presence of model bias. Tellus A, 58(3), 293-306.
14. Bain, A. and Crisan, D. 2009. Fundamentals of Stochastic Filtering. Springer, New York.
15. Bengtsson, T., Bickel, P. and Li, B. 2008. Curse of dimensionality revisited: Collapse of the particle filter in very large scale systems. In: Nolan, D. and Speed, T. (eds), IMS Lecture Notes - Monograph Series in Probability and Statistics: Essays in Honor of David A. Freedman, vol. 2. Institute of Mathematical Sciences, pp. 316-334.
16. Bensoussan, A. 2004. Stochastic Control of Partially Observable Systems. Cambridge University Press, Cambridge.
17. Berliner, L.M., Milliff, R.F. and Wikle, C. K. 2003. Bayesian hierarchical modeling of air-sea interaction. J. Geophys. Res., 108, 3104-3120.
18. Bickel, P., Li, B. and Bengtsson, T. 2008. Sharp failure rates for the bootstrap filter in high dimensions. In: IMS Lecture Notes - Monograph Series: Essays in Honor of J. K. Gosh, vol. 3. Institute of Mathematical Sciences, pp. 318-329.
19. Bishop, C. H., Etherton, B. and Majumdar, S. J. 2001. Adaptive sampling with the ensemble transform Kalman filter. Part I: The theoretical aspects. Monthly Weather Rev., 129, 420-436.
20. Bourlioux, A. and Majda, A.J. 2002. Elementary models with probability distribution function intermittency for passive scalars with a mean gradient. Phys. Fluids, 14(2), 881-897.
21. Branicki, M., Gershgorin, B. and Majda, A.J. 2011. Filtering skill for turbulent signals for a suite of nonlinear and linear extended Kalman filters. J. Comput. Phys. doi:10.1016/j.jcp.2011.10.029.
22. Burgers, G., van Leeuwen, P.J. and Evensen, G. 1998. On the analysis scheme in the ensemble Kalman filter. Monthly Weather Rev., 126, 1719-1724.
23. Cane, M. A., Kaplan, A., Miller, R. N., Tang, B., Hackert, E. C. and Busalacchi, A. J. 1996. Mapping tropical Pacific sea level: Data assimilation via a reduced state space Kalman filter. J. Geophys. Res., 101, 22599-22617.
24. Castronovo, E., Harlim, J. and Majda, A.J. 2008. Mathematical test criteria for filtering complex systems: Plentiful observations. J. Comput. Phys., 227(7), 3678-3714.
25. Chorin, A. J. and Krause, P. 2004. Dimensional reduction for a Bayesian filter. Proc. Natl Acad. Sci., 101(42), 15013-15017.
26. Chorin, A.J. and Tu, X. 2009. Implicit sampling for particle filters. Proc. Nat. Acad. Sci., 106, 17249-17254.
27. Chui, C. and Chen, G. 1999. Kalman Filtering: With Real-Time Applications. Springer, New York.
28. Cohn, S. E. and Dee, D. P. 1988. Observability of discretized partial differential equations. SIAMJ. Numer. Anal., 25(3), 586-617.
29. Cohn, S.E. and Parrish, S.F. 1991. The behavior of forecast error covariances for a Kalman filter in two dimensions. Monthly Weather Rev., 119, 1757-1785.
30. Constantin, P., Foias, C., Nicolaenko, B. and Temam, R. 1988. Integral Manifolds and Inertial Manifolds for Dissipative Partial Differential Equations. Applied Mathematical Sciences, vol. 70. Springer, Berlin.
31. Courtier, P., Andersson, E., Heckley, W., Pailleux, J., Vasiljevic, D., Hamrud, M., Hollingsworth, A., Rabier, F. and Fisher, M. 1998. The ECMWF implementation of three-dimensional variational assimilation (3D-Var). I: Formulation. Quart. J. Roy. Meteorol. Soc., 124, 1783-1807.
32. Daley, R. 1991. Atmospheric Data Analysis. Cambridge University Press, New York.
33. Dee, D. P. and Da Silva, A. M. 1998. Data assimilation in the presence of forecast bias. Quart. J. Roy. Meteorol. Soc., 124, 269-295.
34. Dee, D. P. and Todling, R. 2000. Data assimilation in the presence of forecast bias: the GEOS moisture analysis. Monthly Weather Rev., 128(9), 3268-3282.
35. Del Moral, P. 1996. Nonlinear filtering: Interacting particle solutions. Markov Proce. Related Fields, 2(44), 555-580.
36. Del Moral, P. and Jacod, J. 2001. Interacting particle filtering with discrete observation. In: Doucet, A., de Freitas, N., and Gordon, N. (eds), Sequential Monte Carlo Methods in Practice, Springer, Berlin, pp. 43-75.
37. Delsole, T. 2004. Stochastic model of quasigeostrophic turbulence. Surv. Geophys., 25(2), 107-149.
38. Doucet, A., De Freitas, N. and Gordon, H. J. 2008. Sequential Monte Carlo Methods in Practice. Springer, Berlin.
39. Embid, P. F. and Majda, A. J. 1998. Low Froude number limiting dynamics for stably stratified flow with small or fixed Rossby Number. Geophys. Astrophys. Fluid Dyn., 87, 1-50.
40. Evensen, G. 1994. Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res., 99, 10143-10162.
41. Evensen, G. 2003. The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dynam., 53, 343-367.
42. Farrell, B. F. and Ioannou, P. J. 2001. State estimation using a reduced-order Kalman filter. J. Atmos. Sci., 58(23), 3666-3680.
43. Farrell, B. F. and Ioannou, P. J. 2005. Distributed forcing of forecast and assimilation error systems. J. Atmos. Sci., 62(2), 460-475.
44. Folland, G. B. 1999. Real Analysis. Wiley-Interscience, New York.
45. Franzke, C. and Majda, A. J. 2006. Low-order stochastic mode reduction for a prototype atmospheric gCm. J. Atmos. Sci., 63, 457-479.
46. Friedland, B. 1969. Treatment of bias in recursive filtering. IEEE Trans. Automat. Contr., AC-14, 359-367.
47. Friedland, B. 1982. Estimating sudden changes of biases in linear dynamical systems. IEEE Trans. Automat. Contr., AC-27, 237-240.
48. Gandin, L. S. 1965. Objective analysis of meteorological fields. Gidrometerologich-eskoe Izdatelstvo (English translation by Israeli Program for Scientific Translations, Jerusalem, 1963).
49. Gardiner, C. W. 1997. Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences. Springer-Verlag, New York.
50. Gaspari, G. and Cohn, S. E. 1999. Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteorol. Soc., 125, 723-758.
51. Gershgorin, B. and Majda, A. J. 2008. A nonlinear test model for filtering slow-fast systems. Comm. Math. Sci., 6(3), 611-649.
52. Gershgorin, B. and Majda, A. J. 2010. Filtering a nonlinear slow-fast system with strong fast forcing. Comm. Math. Sci., 8(1), 67-92.
53. Gershgorin, B. and Majda, A. J. 2011. Filtering a statistically exactly solvable test model for turbulent tracers from partial observations. J. Comput. Phys., 230(4), 1602-1638.
54. Gershgorin, B., Harlim, J. and Majda, A. J. 2010a. Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation. J. Comput. Phys., 229(1), 32-57.
55. Gershgorin, B., Harlim, J. and Majda, A. J. 2010b. Test models for improving filtering with model errors through stochastic parameter estimation. J. Comput. Phys., 229(1), 1-31.
56. Ghil, M. and Malanotte-Rizolli, P. 1991. Data assimilation in meteorology and oceanography. Adv. Geophys., 33, 141-266.
57. Grote, M. J. and Majda, A. J. 2006. Stable time filtering of strongly unstable spatially extended systems. Proce. Nat. Acad. Sci., 103, 7548-7553.
58. Haken, H. 1975. Analogy between higher instabilities in fluids and lasers. Phys. Lett. A, 53(1), 77-78.
59. Hamill, T. M., Whitaker, J. S. and Snyder, C. 2001. Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Monthly Weather Rev., 129, 2776-2790.
60. Harlim, J. 2006. Errors in the initial conditions for numerical weather prediction: A study of error growth patterns and error reduction with ensemble filtering. Ph.D. thesis, University of Maryland, College Park, MD.
61. Harlim, J. 2011. Interpolating irregularly spaced observations for filtering turbulent complex systems. SIAM J. Sci. Comp., 33(5), 2620-2640.
62. Harlim, J. and Hunt, B.R. 2007a. A non-Gaussian ensemble filter for assimilating infrequent noisy observations. Tellus A, 59(2), 225-237.
63. Harlim, J. and Hunt, B.R. 2007b. Four-dimensional local ensemble transform Kalman filter: numerical experiments with a global circulation model. Tellus A, 59(5), 731-748.
64. Harlim, J. and Majda, A. J. 2008a. Filtering nonlinear dynamical systems with linear stochastic models. Nonlinearity, 21(6), 1281-1306.
65. Harlim, J. and Majda, A. J. 2008b. Mathematical strategies for filtering complex systems: Regularly spaced sparse observations. J. Comput. Phys., 227(10), 5304-5341.
66. Harlim, J. and Majda, A.J. 2010a. Catastrophic filter divergence in filtering nonlinear dissipative systems. Comm. Math. Sci., 8(1), 27-43.
67. Harlim, J. and Majda, A. J. 2010b. Filtering turbulent sparsely observed geophysical flows. Monthly Weather Rev., 138(4), 1050-1083.
68. Haven, K., Majda, A. J. and Abramov, R. 2005. Quantifying predictability through information theory: small sample estimation in a non-Gaussian framework. J. Comput. Phys., 206, 334-362.
69. Houtekamer, P. L. and Mitchell, H.L. 2001. A sequential ensemble Kalman filter for atmospheric data assimilation. Monthly Weather Rev., 129, 123-137.
70. Hunt, B.R., Kostelich, E.J. and Szunyogh, I. 2007. Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Physica D, 230, 112-126.
71. Isaacson, E. and Keller, H. B. 1966. Analysis of Numerical Methods. John Wiley, New York.
72. Jazwinski, A. H. 1970. Stochastic Processes and Filtering Theory. Academic Press, San Diego, CA.
73. Junk, M. 2004. Maximum entropy moment problems and extended Euler equations. In: Proceedings of IMA Workshop “Simulation of Transport in Transition Regimes”, Springer, Berlin, pp. 189-198.
74. Kaipio, J. P. and Somersalo, E. 2005. Statistical and Computational Inverse Problems. Springer, New York.
75. Kallianpur, G. 1980. Stochastic Filtering Theory. Springer-Verlag, Berlin.
76. Kalman, R.E. and Bucy, R. 1961. New results in linear prediction and filtering theory. Trans. AMSE J. Basic Eng., 83D, 95-108.
77. Kalnay, E. 2003. Atmospheric Modeling, Data Assimilation, and Predictability. Cambridge University Press, Cambridge.
78. Kalnay, E., Li, H., Miyoshi, T., Yang, S.-C. and Ballabrera-Poy, J. 2007. 4D-Var or ensemble Kalman filter? Tellus A, 59A, 758-773.
79. Kang, E. L. and Harlim, J. 2011. Filtering nonlinear spatio-temporal chaos with an autoregressive linear stochastic model. Submitted to Physica D.
80. Keating, S.R., Majda, A.J. and Smith, K. S. 2011. New methods for estimating poleward eddy heat transport using satellite altimetry. Submitted to Monthly Weather Rev.
81. Keppenne, C.L. 2000. Data assimilation into a primitive-equation model with a parallel ensemble Kalman filter. Monthly Weather Rev., 128, 1971-1981.
82. Khouider, B. and Majda, A. J. 2006. A simple multicloud parametrization for convectively coupled tropical waves. Part I: Linear Analysis. J. Atmos. Sci., 63, 1308-1323.
83. Khouider, B. and Majda, A. J. 2007. A simple multicloud parametrization for convectively coupled tropical waves. Part II: Nonlinear simulations. J. Atmos. Sci., 64, 381-400.
84. Kleeman, R. and Majda, A. J. 2005. Predictabiliy in a model of geophysical turbulence. J. Atmos. Sci., 62, 2864-2879.
85. Lawler, G. 1995. Introduction to Stochastic Processes. Chapman & Hall/CRC, Bora Raton.
86. Lee, E. B. and Markus, L. 1967. Foundations of Optimal Control Theory. John Wiley, New York.
87. Lorenc, A. C. 1986. Analysis methods for numerical weather prediction. Quart. J. Roy. Meteorol. Soc., 112, 1177-1194.
88. Lorenc, A. C. 2003. The potential of the ensemble Kalman filter for NWP-a comparison with 4D-Var. Quart. J. Roy. Meteorol. Soc., 129, 3183-3203.
89. Lorenz, E. N. 1963. Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130-141.
90. Lorenz, E. N. 1965. A study of the predictability of a 28-variable atmospheric model. Tellus, 17, 321-333.
91. Lorenz, E. N. 1996. Predictability - a problem partly solved. In: Proceedings on Predictability, ECMWF, 4-8 September 1995, pp. 1-18.
92. Lorenz, E.N. and Emanuel, K. 1998. Optimal sites for supplementary weather observations: Simulations with a small model. J. Atmos. Sci., 55, 399-414.
93. Majda, A. J. 2000. Real world turbulence and modern applied mathematics. In: Arnold, V. I., Atiyah, M., Lax, P., and Mazur, B. (eds), Mathematics: Frontiers and Perspectives 2000. American Mathematical Society, Providence, RI, pp. 137-151.
94. Majda, A. J. 2003. Introduction to PDEs and Waves for the Atmosphere and Ocean. Courant Lecture Notes in Mathematics, vol. 9. American Mathematical Society, Providence, RI.
95. Majda, A. J. and Gershgorin, B. 2011. Elementary models for turbulent diffusion with complex physical features: eddy diffusivity, spectrum, and intermittency. Phil. Trans. Roy. Soc., series B (in press).
96. Majda, A. J. and Grote, M. J. 2007. Explicit off-line criteria for stable accurate time filtering of strongly unstable spatially extended systems. Proc. Nat. Acad. Sci., 104, 1124-1129.
97. Majda, A. J. and Kramer, P. 1999. Simplified models for turbulent diffusion: theory, numerical modeling, and physical phenomena. Phys. Rep., 214, 238-574.
98. Majda, A. J. and Timofeyev, I. 2002. Statistical mechanics for truncations of the BurgersHopf equation: A model for intrinsic behavior with scaling. Milan J. Math., 70, 39-96.
99. Majda, A. J. and Timofeyev, I. 2004. Low dimensional chaotic dynamics versus intrinsic stochastic chaos: A paradigm model. Physica D, 199, 339-368.
100. Majda, A. J. and Wang, X. 2006. Nonlinear Dynamics and Statistical Theories for Basic Geophysical Flows. Cambridge University Press, Cambridge.
101. Majda, A. J., Abramov, R. V. and Grote, M. J. 2005. Information Theory and Stochastics for Multiscale Nonlinear Systems. CRM Monograph Series vol. 25, American Mathematical Society, Providence, RI.
102. Majda, A. J., Franzke, C. and Khouider, B. 2008. An applied mathematics perspective on stochastic modelling for climate. Phil. Trans. A Math. Phys. Eng Sci., 366(1875), 2429-2455.
103. Majda, A.J., Gershgorin, B. and Yuan, Y. 2009. Low-frequency climate response and fluctuation-dissipation theorems: Theory and practice. J. Atmos. Sci. 67, 1186-1201.
104. Majda, A.J., Timofeyev, I. and Vanden-Eijnden, E. 1999. Models for stochastic climate prediction. Proc. Natl. Acad. Sci., 96, 15687-15691.
105. Majda, A. J., Timofeyev, I. and Vanden-Eijnden, E. 2001. A mathematical framework for stochastic climate models. Comm. Pure Appl. Math., 54, 891-974.
106. Majda, A. J., Timofeyev, I. and Vanden-Eijnden, E. 2003. Systematic strategies for stochastic mode reduction in climate. J. Atmos. Sci., 60, 1705-1722.
107. Majda, A. J., Timofeyev, I. and Vanden-Eijnden, E. 2006. Stochastic models for selected slow variables in large deterministic systems. Nonlinearity, 19, 769-794.
108. Miller, R. N., Carter, E. F. and Blue, S. T. 1999. Data assimilation into nonlinear stochastic models. Tellus A, 51(2), 167-194.
109. Mitchell, H.L., Houtekamer, P. L. and Pellerin, G. 2002. Ensemble size and model-error representation in an ensemble Kalman filter. Monthly Weather Rev., 130, 2791-2808.
110. Neelin, J.D., Lintner, B. R., Tian, B., Li, Q., Zhang, L., Patra, P. K., Chahine, M. T. and Stechmann, S. N. 2011. Long tails in deep columns of natural and anthropogenic tropospheric tracers. Geophys. Res. Lett., 37, L05804.
111. O’Kane, T. J. and Frederiksen, J. S. 2008. Comparison of statistical dynamical, square root and ensemble Kalman filters. Entropy, 10(4), 684-721.
112. Oksendal, B. 2003. Stochastic Differential Equations: An Introduction with Applications. Universitext. Springer, Berlin.
113. Ott, E., Hunt, B.R., Szunyogh, I., Zimin, A. V., Kostelich, E.J., Corrazza, M., Kalnay, E. and Yorke, J. A. 2004. A local ensemble Kalman filter for atmospheric data assimilation. Tellus A, 56, 415-428.
114. Parrish, D. F. and Derber, J. C. 1992. The National Meteorological Centers spectral statistical-interpolation analysis system. Monthly Weather Rev., 120(8), 1747-1763.
115. Pedlosky, J. 1979. Geophysical Fluid Dynamics. Springer, New York.
116. Pitcher, E. J. 1977. Application of stochastic dynamic prediction to real data. J. Atmos. Sci., 34(1), 3-21.
117. Richtmeyer, R. D. and Morton, K. W. 1967. Difference Methods for Initial Value Problems. John Wiley, New York.
118. Rossi, V. and Vila, J.-P. 2006. Nonlinear filtering in discrete time: A particle convolution approach. Ann. Inst. Stat. Univ Paris, 3, 71-102.
119. Salmon, R. 1998. Lectures on Geophysical Fluid Dynamics. Oxford University Press, Oxford.
120. Smith, S. K., Boccaletti, G., Henning, C. C., Marinov, I. N., Tam, C. Y., Held, I. M. and Val-lis, G. K. 2002. Turbulent diffusion in the geostropic inverse cascade. J. Fluid Mech., 469, 13-48.
121. Snyder, C., Bengtsson, T., Bickel, P. and Anderson, J.L. 2008. Obstacles to highdimensional particle filtering. Monthly Weather Rev. 136, 4629-4640.
122. Solari, H. G., Natiello, M. A. and Mindlin, G. B. 1996. Nonlinear Dynamics. Institute of Physics, Bristol.
123. Sparrow, C. 1982. The Lorenz Equations: Bifurcation, Chaos, and Strange Attractors. Springer, New York.
124. Strikwerda, J. 2004. Finite Difference Schemes and Partial Differential Equations. (2nd ed.) SIAM, Philadelphia, PA.
125. Szunyogh, I., Kostelich, E.J., Gyarmati, G., Patil, D.J., Hunt, B.R., Kalnay, E., Ott, E. and Yorke, J. A. 2005. Assessing a local ensemble Kalman filter: perfect model experiments with the NCEP global model. Tellus A, 57, 528-545.
126. Thomas, S. J., Hacker, J. P. and Anderson, J. L. 2007. A robust formulation of the ensemble Kalman filter. Quart. J. Roy. Meteorol. Soc., 135, 507-521.
127. Todling, R. and Ghil, M. 1994. Tracking atmospheric instabilities with the Kalman filter. Part I: Methodology and one-layer results. Monthly Weather Rev., 122(1), 183-204.
128. Tribbia, J. J. and Baumhefner, D. P 1988. The reliability of improvements in deterministic short-range forecasts in the presence of initial state and modeling deficiencies. Monthly Weather Rev., 116, 2276-2288.
129. Vallis, G. K. 2006. Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-Scale Circulation. Cambridge University Press, Cambridge.
130. van Leeuwen, P. J. 2009. Particle filtering in geosciences. Monthly Weather Rev., 137, 4089-4114.
131. Wang, X., Bishop, C.H. and Julier, S. J. 2004. Which is better, an ensemble of positivenegative pairs or a centered spherical simplex ensemble? Monthly Weather Rev., 132(7), 1590-1605.
132. Whitaker, J. S. and Hamill, T.M. 2002. Ensemble data assimilation without perturbed observations. Monthly Weather Rev., 130, 1913-1924.
133. Yang, S.-C., Baker, D., Cordes, K., Huff, M., Nagpal, F., Okereke, E., Villafane, J., Kalnay, E. and Duane, G. S. 2006. Data assimilation as synchronization of truth and model: experiments with the three-variable Lorenz system. J. Atmos. Sci., 63(9), 2340-2354.
134. Young, L.-S. 2002. What are SRB measures, and which dynamical systems have them? J. Statist. Phys, 108(5), 733-754.