On the use of a Huber norm for observation quality control in the ECMWF 4D-Var

Quarterly Journal of the Royal Meteorological Society

Volumn 141, Issue 690, 2015, Pages 1514-1527

  Tavolato, Christina ^a,b   Isaksen, Lars ^a  

^a EUROPEAN CENTRE FOR MEDIUM RANGE WEATHER FORECASTS   (United Kingdom)

^b UNIVERSITY OF VIENNA   (Austria)

Author keywords

Huber norm; Observation quality control; Robust estimation; Variational data assimilation

Indexed keywords

ERROR STATISTICS; ERRORS; QUALITY ASSURANCE; SPECIFICATIONS; STATISTICS; STORMS;

ASSIMILATION SYSTEM; DATA ASSIMILATION SYSTEMS; HUBER NORMS; IN-SITU OBSERVATIONS; OBSERVATION ERRORS; OBSERVATION QUALITIES; ROBUST ESTIMATION; VARIATIONAL DATA ASSIMILATION;

QUALITY CONTROL;

CYCLONE; DATA ASSIMILATION; EXTREME EVENT; IN SITU MEASUREMENT; QUALITY CONTROL; STATISTICAL DATA; WEATHER FORECASTING;

EID: 84939441307     PISSN: 00359009     EISSN: 1477870X     Source Type: Journal    
DOI: 10.1002/qj.2440     Document Type: Article

References (27)

1. Andersson E, Järvinen H. 1999. Variational quality control. Q. J. R. Meteorol. Soc. 125: 697-722.
2. Andersson E, Bauer P, Beljaars ACM, Chevallier F, Hólm EV, Janisková M, Kållberg P, Kelly G, Lopez P, McNally AP, Moreau E, Simmons AJ, Thépaut J-N, Tompkins AM. 2005. Assimilation and modeling of the atmospheric hydrological cycle in the ECMWF forecasting system. Bull. Am. Meteorol. Soc. 86: 387-403.
3. Bonavita M, Isaksen L, Hólm EV. 2012. On the use of EDA background error variances in the ECMWF 4D-Var. Q. J. R. Meteorol. Soc. 138: 1540-1559, doi: 10.1002/qj.1899.
4. Courtier P, Thépaut J-N, Hollingsworth A. 1994. A strategy for operational implementation of 4D-Var, using an incremental approach. Q. J. R. Meteorol. Soc. 120: 1367-1388.
5. Courtier P, Andersson E, Heckley W, Vasiljevic D, Hamrud M, Hollingsworth A, Rabier F, Fisher M, Pailleux J. 1998. The ECMWF implementation of three-dimensional variational assimilation (3D-Var). I: Formulation. Q. J. R. Meteorol. Soc. 124: 1783-1807.
6. Dee DP, Rukhovets L, Todling R, da Silva AM, Larson JW. 2001. An adaptive buddy check for observational quality control. Q. J. R. Meteorol. Soc. 127: 2451-2471.
7. Dee DP, Uppala SM, Simmons AJ, Berrisford P, Poli P, Kobayashi S, Andrae U, Balmaseda MA, Balsamo G, Bauer P, Bechtold P, Beljaars ACM, van de Berg L, Bidlot J, Bormann N, Delsol C, Dragani R, Fuentes M, Geer AJ, Haimberger L, Healy SB, Hersbach H, Hólm EV, Isaksen L, Kållberg P, Köhler M, Matricardi M, McNally AP, Monge-Sanz BM, Morcrette J-J, Park B-K, Peubey C, de Rosnay P, Tavolato C, Thépaut J-N, Vitart F. 2011. The ERA-Interim: Configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137: 553-597.
8. Desroziers G, Berre L, Chapnik B, Poli P. 2005. Diagnosis of observation-, background- and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc. 131: 3385-3396.
9. Dharssi I, Lorenc AC, Ingleby NB. 1992. Treatment of gross errors using probability theory. Q. J. R. Meteorol. Soc. 118: 1017-1036.
10. Fisher M, Nocedal J, Trémolet Y, Wright SJ. 2009. Data assimilation in weather forecasting: a case study in PDE-constrained optimization. Optim. Eng. 10: 409-426.
11. Guitton A, Symes WW. 2003. Robust inversion of siesmic data using the Huber norm. Geophysics 68: 1310-1319.
12. Hollingsworth A, Shaw DB, Lönnberg P, Illari L, Arpe K, Simmons AJ. 1986. Monitoring of observations and analysis quality by a data assimilation system. Mon. Weather Rev. 114: 861-879.
13. Hólm EV, Andersson E, Beljaars ACM, Lopez P, Mahfouf J-F, Simmons AJ, Thépaut J-N. 2002. Assimilation and Modelling of the Hydrological Cycle: ECMWF's Status and Plans, ECMWF Technical Memorandum 383. ECMWF: Reading, UK.
14. Huber PJ. 1964. Robust estimates of a location parameter. Ann. Math. Stat. 35: 73-101.
15. Huber PJ. 1972. Robust statistics: A review. Ann. Math. Stat. 43: 1041-1067.
16. Huber PJ. 2002. John W. Tukey's contribution to robust statistics. Ann. Math. Stat. 30: 1640-1648.
17. Ingleby NB, Lorenc AC. 1993. Bayesian quality control using multivariate normal distributions. Q. J. R. Meteorol. Soc. 119: 1195-1225.
18. Isaksen L, Bonavita M, Buizza R, Fisher M, Haseler J, Leutbecher M, Raynaud L. 2010. Ensemble of Data Assimilations at ECMWF ECMWF Technical Memorandum. 636. ECMWF: Reading, UK.
19. Järvinen H, Undén P. 1997. Observation Screening and Background Quality Control in the ECMWF3D-VarData Assimilation System ECMWF Technical Memorandum 236. ECMWF: Reading, UK.
20. Järvinen H, Andersson E, Bouttier F. 1999. Variational assimilation of time sequences of surface observations with serially correlated errors. Tellus 51A: 469-488.
21. Lorenc AC, 'Analysis methods for the quality control of observations', 1984. Reading, UK, 6-9 November 1984, 397-428.
22. Lorenc AC. 1986. Analysis methods for numerical weather prediction. Q. J. R. Meteorol. Soc. 112: 1177-1194.
23. Lorenc AC, Hammon O. 1988. Objective quality control of observations using Bayesian methods-Theory, and practical implementation. Q. J. R. Meteorol. Soc. 114: 515-543.
24. Tavolato C, Isaksen L. 2010. Huber norm quality control in the IFS. ECMWF Newsletter 122: 27-31.
25. Tukey JW. 1960. A survey of sampling from contaminated distributions. In Contributions to Probability and Statistics. Olkin, I Ghurye, S, Hoeffding, W, (eds.). 448-485. Stanford University Press: Stanford, CA.
26. Ulbrich U, Fink AH, Klawa M, Pinto JG. 2001. Three extreme storms over Europe in December 1999. Weather 56: 70-80.
27. Vasiljevic D, Andersson E, Isaksen L, Garcia-Mendez A. 2006. Surface pressure bias correction in data assimilation. ECMWF Newsletter 108: 20-27.