Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems

Journal of Computational Physics

Volumn 228, Issue 6, 2009, Pages 1862-1902

  Marzouk, Youssef M ^a   Najm, Habib N ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

Bayesian inference; Dimensionality reduction; Galerkin projection; Gaussian processes; Inverse problems; Karhunen Lo ve expansion; Markov chain Monte Carlo; Polynomial chaos; RKHS

Indexed keywords

BAYESIAN NETWORKS; CHAINS; DIFFERENTIAL EQUATIONS; GALERKIN METHODS; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); INFERENCE ENGINES; MARKOV PROCESSES; MONTE CARLO METHODS; POLYNOMIALS; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

BAYESIAN INFERENCE; DIMENSIONALITY REDUCTION; FORWARD MODELING; GALERKIN PROJECTIONS; GAUSSIAN PROCESSES; KARHUNEN-LOÈVE EXPANSION; MARKOV CHAIN MONTE CARLO; MARKOV CHAIN MONTE-CARLO; POLYNOMIAL CHAOS; RKHS;

INVERSE PROBLEMS;

EID: 59149084494     PISSN: 00219991     EISSN: 10902716     Source Type: Journal    
DOI: 10.1016/j.jcp.2008.11.024     Document Type: Article

References (80)

1. Vogel C.R. Computational Methods for Inverse Problems (2002), SIAM
2. Evans S.N., and Stark P.B. Inverse problems as statistics. Inverse Problems 18 (2002) R55-R97
3. Tenorio L. Statistical regularization of inverse problems. SIAM Rev. 43 2 (2001) 347-366
4. Kaipio J., and Somersalo E. Statistical and Computational Inverse Problems (2005), Springer
5. Tarantola A. Inverse Problem Theory and Methods for Model Parameter Estimation (2005), SIAM
6. Marzouk Y.M., Najm H.N., and Rahn L.A. Stochastic spectral methods for efficient Bayesian solution of inverse problems. J. Comput. Phys. 224 2 (2007) 560-586
7. Wiener N. The homogeneous chaos. Am. J. Math. 60 (1938) 897-936
8. Cameron R., and Martin W. The orthogonal development of nonlinear functionals in series of Fourier-Hermite functionals. Ann. Math. 48 (1947) 385-392
9. Ghanem R., and Spanos P. Stochastic Finite Elements: A Spectral Approach (1991), Springer-Verlag, New York
10. Xiu D., and Karniadakis G. The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM J. Sci. Comput. 24 2 (2002) 619-644
11. Debusschere B., Najm H., Pébay P., Knio O., Ghanem R., and Le Maître O. Numerical challenges in the use of polynomial chaos representations for stochastic processes. SIAM J. Sci. Comput. 26 2 (2004) 698-719
12. Wan X., and Karniadakis G.E. An adaptive multi-element generalized polynomial chaos method for stochastic differential equations. J. Comput. Phys. 209 (2005) 617-642
13. Ghanem R. Probabilistic characterization of transport in heterogeneous media. Comput. Methods Appl. Mech. Eng. 158 (1998) 199-220
14. R. Ghanem, J. Red-Horse, A. Sarkar, Modal properties of a space-frame with localized system uncertainties, in: A. Kareem, A. Haldar, B.S., Jr., E. Johnson (Eds.), 8th ASCE Specialty Conference of Probabilistic Mechanics and Structural Reliability, PMC200-269, ASCE, 2000.
15. Le Maître O., Knio O., Najm H., and Ghanem R. A stochastic projection method for fluid flow I. Basic formulation. J. Comput. Phys. 173 (2001) 481-511
16. Le Maître O., Reagan M., Najm H., Ghanem R., and Knio O. A stochastic projection method for fluid flow II. Random process. J. Comput. Phys. 181 (2002) 9-44
17. Xiu D., Lucor D., Su C.-H., and Karniadakis G. Stochastic modeling of flow-structure interactions using generalized polynomial chaos. ASME J. Fluids Eng. 124 (2002) 51-59
18. Debusschere B., Najm H., Matta A., Knio O., Ghanem R., and Le Maître O. Protein labeling reactions in electrochemical microchannel flow: numerical simulation and uncertainty propagation. Phys. Fluids 15 8 (2003) 2238-2250
19. Reagan M., Najm H., Debusschere B., Le Maître O., Knio O., and Ghanem R. Spectral stochastic uncertainty quantification in chemical systems. Comb. Theory Model. 8 (2004) 607-632
20. Wang J., and Zabaras N. Using Bayesian statistics in the estimation of heat source in radiation. Int. J. Heat Mass Transfer 48 (2005) 15-29
21. Berkooz G., Holmes P., and Lumley J.L. The proper orthogonal decomposition in the analysis of turbulent flows. Annu. Rev. Fluid Mech. 25 (1993) 539-575
22. Balakrishnan S., Roy A., Ierapetritou M.G., Flach G.P., and Georgopoulos P.G. Uncertainty reduction and characterization for complex environmental fate and transport models: an empirical Bayesian framework incorporating the stochastic response surface method. Water Resour. Res. 39 12 (2003) 1350
23. Kennedy M.C., and O'Hagan A. Bayesian calibration of computer models. J. Royal Stat. Soc.: Series B 63 3 (2001) 425-464
24. Christen J.A., and Fox C. MCMC using an approximation. J. Comput. Graph. Stat. 14 4 (2005) 795-810
25. Higdon D., Lee H., and Holloman C. Markov chain Monte Carlo-based approaches for inference in computationally intensive inverse problems. Bayesian Stat. 7 (2003) 181-197
26. Geyer C.J. Markov chain Monte Carlo maximum likelihood. In: Keramidas E.M. (Ed). Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface vol. 23 (1991), Interface Foundation of North America 156-163
27. Efendiev Y., Hou T.Y., and Luo W. Preconditioning Markov chain Monte Carlo simulations using coarse-scale models. SIAM J. Sci. Comput. 28 (2006) 776-803
28. Lee H., Higdon D., Bi Z., Ferreira M., and West M. Markov random field models for high-dimensional parameters in simulations of fluid flow in porous media. Technometrics 44 3 (2002) 230-241
29. Wang J., and Zabaras N. A Markov random field model of contamination source identification in porous media flow. Int. J. Heat Mass Transfer 49 (2006) 939-950
30. Kaipio J., and Somersalo E. Statistical inverse problems: discretization, model reduction, and inverse crimes. J. Comput. Appl. Math. 198 2 (2007) 493-504
31. Li W., and Cirpka O.A. Efficient geostatistical inverse methods for structured and unstructured grids. Water Resour. Res. 42 (2006) W06402
32. Xiu D., and Karniadakis G. Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Comput. Methods Appl. Mech. Eng. 191 (2002) 4927-4948
33. Le Maître O., Reagan M., Debusschere B., Najm H., Ghanem R., and Knio O. Natural convection in a closed cavity under stochastic, non-Boussinesq conditions. SIAM J. Sci. Comput. 26 2 (2004) 375-394
34. Frauenfelder P., Schwab C., and Todor R. Finite elements for elliptic problems with stochastic coefficients. Comput. Methods Appl. Mech. Eng. 194 (2005) 205-228
35. Matthies H.G., and Keese A. Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations. Comput. Methods Appl. Mech. Eng. 194 (2005) 1295-1331
36. Grigoriu M. Stochastic Calculus (2002), Birkhäuser
37. Hoel P.G., Port S.C., and Stone C.J. Introduction to Stochastic Processes (1987), Waveland Press
38. Seeger M. Gaussian processes for machine learning. Int. J. Neural Syst. 14 2 (2004) 1-38
39. Williams C.K.I., and Rasmussen C.E. Gaussian processes for regression. In: Teuretzky D.S., Mozer M.C., and Hasselmo M.E. (Eds). Advances in Neural Information Processing Systems vol. 8 (1996), MIT Press
40. Cressie N.A.C. Statistics for Spatial Data (1993), Wiley
41. Rasmussen C.E., and Williams C.K.I. Gaussian Processes for Machine Learning (2006), MIT Press
42. Genton M.G. Classes of kernels for machine learning: a statistics perspective. J. Mach. Learning Res. 2 (2001) 299-312
43. Loève M. Probability Theory II. fourth ed. (1978), Springer
44. Adler R.J., and Taylor J.E. Random Fields and Geometry (2007), Springer
45. P. Abrahamsen, A review of Gaussian random fields and correlation functions, Technical Report 917, Norwegian Computing Center, Oslo, Norway, 1997.
46. Courant R., and Hilbert D. Methods of Mathematical Physics vol. 1 (1953), Wiley-Interscience
47. Sirovich L. Turbulence and the dynamics of coherent structures. Part 1: coherent structures. Quart. Appl. Math. 45 3 (1987) 561-571
48. A. Mohammad-Djafari, Bayesian inference for inverse problems, in: Bayesian inference and Maximum Entropy Methods in Science and Engineering, vol. 21, 2002, pp. 477-496.
49. Aster R., Borchers B., and Thurber C. Parameter Estimation and Inverse Problems (2004), Academic Press
50. Gouveia W.P., and Scales J.A. Resolution of seismic waveform inversion: Bayes versus Occam. Inverse Problems 13 (1997) 323-349
51. Malinverno A. Parsimonious Bayesian Markov chain Monte Carlo inversion in a nonlinear geophysical problem. Geophys. J. Int. 151 (2002) 675-688
52. Jackson C., Sen M.K., and Stoffa P.L. An efficient stochastic Bayesian approach to optimal parameter and uncertainty estimation for climate model predictions. J. Clim. 17 (2004) 2828-2841
53. Wang J., and Zabaras N. Hierarchical Bayesian models for inverse problems in heat conduction. Inverse Prob. 21 (2005) 183-206
54. MacKay D.J.C. Comparison of approximate methods for handling hyperparameters. Neural Comput. 11 5 (1999) 1035-1068
55. Gilks W.R., Richardson S., and Spiegelhalter D.J. Markov Chain Monte Carlo in Practice (1996), Chapman & Hall, London
56. Kakutani S. Spectral analysis of stationary Gaussian processes. In: Neyman J. (Ed). Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability vol. 2 (1961), University of California Press 239-247
57. Schoutens W. Stochastic Processes and Orthogonal Polynomials (2000), Springer
58. Reagan M., Najm H., Ghanem R., and Knio O. Uncertainty quantification in reacting flow simulations through non-intrusive spectral projection. Combustion and Flame 132 (2003) 545-555
59. Wahba G. Spline Models for Observational Data (1990), SIAM
60. Kimeldorf G.S., and Wahba G. A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Stat. 41 2 (1970) 495-502
61. Oliver D.S., Cunha L., and Reynolds A.C. Markov chain Monte Carlo methods for conditioning a permeability field to pressure data. Math. Geol. 29 1 (1997) 61-91
62. Sommeijer B., Shampine L., and Verwer J. RKC: an explicit solver for parabolic PDEs. J. Comput. Appl. Math. 88 (1997) 315-326
63. J.G. Verwer, B.P. Sommeijer, W. Hundsdorfer, RKC time-stepping for advection-diffusion-reaction problems, CWI Report MAS-E0405, CWI, Amsterdam, The Netherlands, 2004.
64. Gelman A., Carlin J.B., Stern H.S., and Rubin D.B. Bayesian Data Analysis. second ed. (2003), Chapman & Hall, CRC
65. Gelman A. Prior distributions for variance parameters in hierarchical models. Bayesian Anal. 1 3 (2006) 515-534
66. H.K.H. Lee, C.H. Holloman, C.A. Calder, D.M. Higdon, Flexible Gaussian processes via convolution, Discussion paper 2002-09, Duke University, Department of Statistical Science, 2002.
67. Kaipio J.P., Kolehmainen V., Vauhkonen M., and Somersalo E. Inverse problems with structural prior information. Inverse Prob. 15 (1999) 713-729
68. Calvetti D., and Somersalo E. A Gaussian hypermodel to recover blocky objects. Inverse Prob. 23 (2007) 733-754
69. Press W., Teukolsky S.A., Vetterling W., and Flannery B. Numerical Recipes in C: The Art of Scientific Computing. second ed. (1992), Cambridge University Press, Cambridge, UK
70. Robert C.P., and Casella G. Monte Carlo Statistical Methods. second ed. (2004), Springer
71. Ghanem R.G. The nonlinear Gaussian spectrum of log-normal stochastic processes and variables. ASME J. Appl. Mech. 66 4 (1999) 964-973
72. W. Luo, Wiener chaos expansion and numerical solutions of stochastic partial differential equations, Ph.D. thesis, California Institute of Technology, May 2006.
73. Dunkley J., Bucher M., Ferreira P.G., and Moodley K. Fast and reliable Markov chain Monte Carlo technique for cosmological parameter estimation. Mon. Notices Roy. Astronom. Soc. 356 3 (2005) 925-936
74. Lorenz E.N. Deterministic nonperiodic flow. J. Atmos. Sci. 20 2 (1963) 130-141
75. Hou T.Y., Luo W., Rozovskii B., and Zhou H.-M. Wiener Chaos expansions and numerical solutions of randomly-forced equations of fluid mechanics. J. Comput. Phys. 216 (2006) 687-706
76. Doostan A., Ghanem R.G., and Red-Horse J. Stochastic model reduction for chaos representations. Comput. Methods Appl. Mech. Eng. 196 (2007) 3951-3966
77. Babuška I., Nobile F., and Tempone R. A stochastic collocation method for elliptic partial differential equations with random input data. SIAM J. Numer. Anal. 45 3 (2007) 1005-1034
78. Xiu D., and Hesthaven J.S. High-order collocation methods for differential equations with random inputs. SIAM J. Sci. Comput. 27 3 (2005) 1118-1139
79. Kraut S., Anderson R.H., and Krolik J.L. A generalized Karhunen-Loève basis for efficient estimation of tropospheric refractivity using radar clutter. IEEE Trans. Signal Process. 52 1 (2004) 48-60
80. Aronszajn N. Theory of reproducing kernels. Trans. Am. Math. Soc. 68 3 (1950) 337-404