Spatio-temporal smoothing and EM estimation for massive remote-sensing data sets

Journal of Time Series Analysis

Volumn 32, Issue 4, 2011, Pages 430-446

  Katzfuss, Matthias ^a   Cressie, Noel ^a  

^a OHIO STATE UNIVERSITY   (United States)

Author keywords

AIRS instrument; EM algorithm; Fixed rank smoothing; Global CO2; Maximum likelihood estimation; Mixed effects models; Spatio temporal statistics

Indexed keywords

EID: 79958198727     PISSN: 01439782     EISSN: 14679892     Source Type: Journal    
DOI: 10.1111/j.1467-9892.2011.00732.x     Document Type: Article

References (55)

1. Banerjee, S., Gelfand, A. E., Finley, A. O. and Sang, H. (2008) Gaussian predictive process models for large spatial data sets. Journal of the Royal Statistical Society, Series B 70, 825-48.
2. Berliner, L. M. (1996) Hierarchical Bayesian time series models. In Maximum Entropy and Bayesian Methods (eds. K. M. Hanson and R. N. Silver ). Dordrecht: Kluwer, Academic Publishers, pp. 15-22.
3. Berliner, L. M., Wikle, C. K. and Cressie, N. (2000). Long-lead prediction of Pacific SSTs via Bayesian dynamic modeling, Journal of Climate 13, 3953-68.
4. Bréon, F.-M. and Ciais, P. (2010). Spaceborne remote sensing of greenhouse gas concentrations. Comptes Rendus Geoscience 342, 412-424.
5. Chahine, M., Pwaagepeagano, T., Aumann, H., Atlas, R., Barnet, C., Blaisdell, J., Chen, L., Divakarla, M., Fetzer, E., Goldberg, M., Gautier, C., Granger, S., Hannon, S., Irion, F. W., Kakar, R., Kalnay, E., Lambrigtsen, B. H., Lee, S.-Y., Marshall, J. L., McMillian, W. W., McMillin, L., Olsen, E. T., Revercomb, H., Rosenkranz, P., Smith, W. L., Staelin, D., Strow, L. L., Susskind, J., Tobin, D., Wolf, W., Zhou, L. (2006). AIRS: Improving weather forecasting and providing new data on greenhouse gases. Bulletin of the American Meteorological Society 87, 911-26.
6. Christensen, O. and Waagepetersen, R. (2002). Bayesian prediction of spatial count data using generalized linear mixed models. Biometrics 58, 280-6.
7. Cressie, N., Huang, H.-C. and Gabrosek, J. (2002) Fast, resolution-consistent spatial prediction of global processes from satellite data. Journal of Computational and Graphical Statistics 11, 63-88.
8. Cressie, N. and Johannesson, G. (2008) Fixed rank kriging for very large spatial data sets. Journal of the Royal Statistical Society, Series B 70, 209-26.
9. Cressie, N. and Kornak, J. (2003) Spatial statistics in the presence of location error with an application to remote sensing of the environment. Statistical Science 18, 436-56.
10. Cressie, N., Shi, T. and Kang, E. L. (2010) Fixed rank filtering for spatio-temporal data. Journal of Computational and Graphical Statistics 19, 724-45.
11. Cressie, N. and Wikle, C. K. (2002) Space-time Kalman filter. In Encyclopedia of Environmetrics, Vol. 4 (eds. A. H. El-Shaarawi and W. W. Piegorsch ). Chichester: John Wiley & Sons, pp. 2045-9.
12. Cressie, N. and Wikle, C. K. (2011) Statistics for Spatio-Temporal Data. Hoboken, NJ: Wiley.
13. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39, 1-38.
14. Farrell, B. and Ioannou, P. (2001) State estimation using a reduced-order Kalman filter. Journal of the Atmospheric Sciences 58, 3666-80.
15. Fassò, A. and Cameletti, M. (2009a) A unified statistical approach for simulation, modeling, analysis and mapping of environmental data. Simulation 86, 139-53.
16. Fassò, A. and Cameletti, M. (2009b) The EM algorithm in a distributed computing environment for modelling environmental space-time data. Environmental Modelling & Software 24, 1027-35.
17. Furrer, R., Sain, S., Nychka, D. and Meehl, G. (2007) Multivariate Bayesian analysis of atmosphere-ocean general circulation models. Environmental and Ecological Statistics 14, 249-66.
18. Hamilton, J. (1994) Time Series Analysis. Princeton: Princeton University Press.
19. Hannan, E. and Deistler, M. (1988) The Statistical Theory of Linear Systems. Wiley.
20. Henderson, H. and Searle, S. (1981) On deriving the inverse of a sum of matrices. Siam Review 23, 53-60.
21. Johannesson, G. and Cressie, N. (2004) Variance-covariance modeling and estimation for multiresolution spatial models. In GeoENV IV - Geostatistics for Environmental Applications Kluwer (eds. X. Sanchez-Vila, J. Carrera and J. Gomez-Hernandez ). Dordrecht: Springer, pp. 319-30.
22. Johannesson, G., Cressie, N. and Huang, H.-C. (2007) Dynamic multi-resolution spatial models. Environmental and Ecological Statistics 14, 5-25.
23. Kalman, R. (1960) A new approach to linear filtering and prediction problems. Journal of Basic Engineering 82, 35-45.
24. Kang, E. L. and Cressie, N. (2011) Bayesian inference for the spatial random effects model. Journal of the American Statistical Association, forthcoming.
25. Kang, E. L., Cressie, N. and Shi, T. (2010) Using temporal variability to improve spatial mapping with application to satellite data. Canadian Journal of Statistics 38, 271-89.
26. Kaplan, A., Cane, M., Kushnir, Y., Clement, A., Blumenthal, M. and Rajagopalan, B. (1998) Analyses of global sea surface temperature 1856-1991. Journal of Geophysical Research 103, 18567-89.
27. Katzfuss, M. and Cressie, N. (2009) Maximum likelihood estimation of covariance parameters in the spatial-random-effects model. In Proceedings of the Joint Statistical Meetings Alexandria: American Statistical Association, pp. 3378-90.
28. Katzfuss, M. and Cressie, N. (2011) Bayesian Hierarchical Spatio-Temporal Smoothing for Massive Datasets. Technical Report 853, Department of Statistics, The Ohio State University.
29. Landgrebe, D. A. (2003) Signal Theory Methods in Multispectral Remote Sensing. Hoboken: Wiley.
30. Lopes, H., Salazar, E. and Gamerman, D. (2008) Spatial dynamic factor analysis. Bayesian Analysis 3, 759-92.
31. Mardia, K., Goodall, C., Redfern, E. and Alonso, F. (1998) The kriged Kalman filter. Test 7, 217-82.
32. McLachlan, G. and Krishnan, T. (2008) The EM Algorithm and Extensions. New York: Wiley-Interscience.
33. Nychka, D., Wikle, C. K. and Royle, J. (2002) Multiresolution models for nonstationary spatial covariance functions. Statistical Modelling 2, 315-31.
34. Ray, S. (2010) AIRS Carbon Dioxide Data. Available at:
35. Sahr, K. (2003) DGGRID Software. Available at:, version 4.3b.
36. Shi, T. and Cressie, N. (2007) Global statistical analysis of MISR aerosol data: a massive data product from NASA's Terra satellite. Environmetrics 18, 665-80.
37. Shumway, R. (2000) Dynamic mixed models for irregularly observed time series. Resenhas-Reviews of the Institute of Mathematics and Statistics 4, 433-56.
38. Shumway, R. and Stoffer, D. (1982) An approach to time series smoothing and forecasting using the EM algorithm. Journal of Time Series Analysis 3, 253-64.
39. Shumway, R. and Stoffer, D. (2006) Time Series Analysis and its Applications. With R Examples. New York: Springer.
40. Smith, T., Reynolds, R., Livezey, R. and Stokes, D. (1996) Reconstruction of historical sea surface temperatures using empirical orthogonal functions. Journal of Climate 9, 1403-20.
41. Stein, M. L. (2008) A modeling approach for large spatial datasets. Journal of the Korean Statistical Society 37, 3-10.
42. Stein, M. L. and Jun, M. (2008) Nonstationary covariance models for global data. Annals of Applied Statistics 2, 1271-89.
43. Stroud, J., Mueller, P. and Sansó, B. (2001) Dynamic models for spatiotemporal data. Journal of the Royal Statistical Society, Series B 63, 673-89.
44. Tzeng, S., Huang, H.-C. and Cressie, N. (2005) A fast, optimal spatial-prediction method for massive datasets. Journal of the American Statistical Association 100, 1343-57.
45. Voutilainen, A., Pyhälahti, T., Kallio, K., Pulliainen, J., Haario, H. and Kaipio, J. (2007) A filtering approach for estimating lake water quality from remote sensing data. International Journal of Applied Earth Observation and Geoinformation 9, 50-64.
46. Wikle, C. K. (2003) Hierarchical Bayesian models for predicting the spread of ecological processes. Ecology 84, 1382-94.
47. Wikle, C. K. (2010) Low-rank representations for spatial processes. InHandbook of Spatial Statistics (eds A. E. Gelfand, M. Fuentes, P.Guttorp and P. Diggle Boca Raton: Chapman and Hall/CRC, pp. 107-18.
48. Wikle, C. K. and Cressie, N. (1999) A dimension-reduced approach to space-time Kalman filtering. Biometrika 86, 815-29.
49. Wikle, C. K. and Hooten, M. (2006) Hierarchical Bayesian spatio-temporal models for population spread. In Applications of Computational Statistics in the Environmental Sciences: Hierarchical Bayes and MCMC Methods (eds. J. S. Clark, A. E. Gelfand ), Oxford University Press, pp. 145-69.
50. Wikle, C. K., Milliff, R., Nychka, D. and Berliner, L. (2001) Spatiotemporal hierarchical Bayesian modeling: tropical ocean surface winds. Journal of the American Statistical Association 96, 382-97.
51. Wu, C. F. J. (1983) On the convergence properties of the EM algorithm. Annals of Statistics 11, 95-103.
52. Xu, K. and Wikle, C. K. (2007) Estimation of parameterized spatio-temporal dynamic models. Journal of Statistical Planning and Inference 137, 567-88.
53. Zhang, H. (2002) On estimation and prediction for spatial generalized linear mixed models. Biometrics 58, 129-36.
54. Zhao, Y., Staudenmayer, J., Coull, B. A. and Wand, M. P. (2006) General design Bayesian generalized linear mixed models. Statistical Science 21, 35-51.
55. Zhu, H., Gu, M. and Peterson, B. (2007) Maximum likelihood from spatial random effects models via the stochastic approximation expectation maximization algorithm. Statistics and Computing 17, 163-77.