Beyond the basics: Evaluating model-based precipitation forecasts using traditional, spatial, and object-based methods

Weather and Forecasting

Volumn 29, Issue 6, 2014, Pages 1451-1472

  Wolff, Jamie K ^a   Harrold, Michelle ^a   Fowler, Tressa ^a   Gotway, John Halley ^a   Nance, Louisa ^a   Brown, Barbara G ^a  

^a NATIONAL CENTER FOR ATMOSPHERIC RESEARCH   (United States)

Author keywords

Forecast verification skill; Model comparison; Model evaluation performance

Indexed keywords

WEATHER FORECASTING;

COARSE-RESOLUTION MODEL; FORECAST VERIFICATION/SKILL; GLOBAL FORECAST SYSTEMS; MODEL COMPARISON; MODEL EVALUATION/PERFORMANCE; PRECIPITATION FORECAST; QUANTITATIVE PRECIPITATION FORECAST; VERIFICATION TECHNIQUES;

NUMERICAL METHODS;

FORECASTING METHOD; MESOSCALE METEOROLOGY; NUMERICAL MODEL; PRECIPITATION ASSESSMENT; QUANTITATIVE ANALYSIS;

EID: 84919643113     PISSN: 08828156     EISSN: 15200434     Source Type: Journal    
DOI: 10.1175/WAF-D-13-00135.1     Document Type: Article

References (39)

1. Accadia, C., S. Mariani, M. Casaioli, and A. Lavagnini, 2003: Sensitivity of precipitation forecast skill scores to bilinear interpolation and a simple nearest-neighbor average method on high-resolution verification grids. Wea. Forecasting, 18, 918-932, doi:10.1175/1520-0434(2003)018,0918:SOPFSS.2.0.CO;2.
2. Ahijevych, D., E. Gilleland, B. G. Brown, and E. Ebert, 2009: Application of spatial verification methods to idealized and NWP-gridded precipitation forecasts. Wea. Forecasting, 24, 1485-1497, doi:10.1175/2009WAF2222298.1.
3. Baldwin, M. E., cited 2012: Quantitative precipitation forecast verification documentation. NOAA/NCEP/Environmental Modeling Center. [Available online at http://www.emc.ncep.noaa.gov/ mmb/ylin/pcpverif/scores/docs/mbdoc/pptmethod.html.]
4. Baldwin, M. E., and J. S. Kain, 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21, 636-648, doi:10.1175/ WAF933.1.
5. Clark, A. J., W. A. Gallus Jr., and T.-C. Chen, 2007: Comparison of the diurnal precipitation cycle in convection-resolving and non-convection-resolving mesoscale models. Mon. Wea. Rev., 135, 3456-3473, doi:10.1175/MWR3467.1.
6. Clark, A. J., W. A. Gallus Jr., and M. L. Weisman, 2010: Neighborhood-based verification of precipitation forecasts from convectionallowing NCAR WRF Model simulations and the operational NAM. Wea. Forecasting, 25, 1495-1509, doi:10.1175/ 2010WAF2222404.1.
7. Clark, A. J., R. G. Bullock, T. L. Jensen, M. Xue, and F. Kong, 2014: Application of object-based time-domain diagnostics for tracking precipitation systems in convection-allowing models. Wea. Forecasting, 29, 517-542, doi:10.1175/ WAF-D-13-00098.1.
8. Davis, C. A., K.W.Manning, R. E. Carbone, S. B. Trier, and J. D. Tuttle, 2003: Coherence of warm-season continental rainfall in numerical weather prediction models. Mon. Wea. Rev., 131, 2667-2679, doi:10.1175/1520-0493(2003)131,2667: COWCRI.2.0.CO;2.
9. Davis, C. A., B. Brown, and R. Bullock, 2006: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 1772-1784, doi:10.1175/MWR3145.1.
10. Davis, C. A., B. Brown, R. Bullock, and J. Halley Gotway, 2009: The Method for Object-Based Diagnostic Evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC Spring Program. Wea. Forecasting, 24, 1252-1267, doi:10.1175/ 2009WAF2222241.1.
11. Done, J., C. A. Davis, and M. Weisman, 2004: The next generation of NWP: Explicit forecasts of convection using the Weather Research and Forecasting (WRF) Model. Atmos. Sci. Lett., 5, 110-117, doi:10.1002/asl.72.
12. Duda, J. D., and W. A. Gallus, 2013: The impact of large-scale forcing on skill of simulated convective initiation and upscale evolution with convection-allowing grid spacings in the WRF. Wea. Forecasting, 28, 994-1018, doi:10.1175/ WAF-D-13-00005.1.
13. Ebert, E. E., 2008: Fuzzy verification of high-resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15, 51-64, doi:10.1002/met.25.
14. Ebert, E. E., 2009: Neighborhood verification: A strategy for rewarding close forecasts. Wea. Forecasting, 24, 1498-1510, doi:10.1175/ 2009WAF2222251.1.
15. EMC, 2003: The GFS atmospheric model. NCEP Office Note 442, Global Climate and Weather Modeling Branch, Environmental Modeling Center, Camp Springs, MD, 14 pp. [Available online at http://www.emc.ncep.noaa.gov/officenotes/ newernotes/on442.pdf.]
16. Fowler, T. L., T. Jensen, E. I. Tollerud, J. Halley Gotway, P. Oldenburg, and R. Bullock, 2010: New Model Evaluation Tools (MET) software capabilities for QPF verification. Preprints, Third Int. Conf. on QPE, QPF and Hydrology, Nanjing, China, WMO/World Weather Research Programme. [Code and documentation available online at http://www.dtcenter. org/met/users/metoverview/index.php.]
17. Fulton, R. A., J. P. Breidenbach, D.-J. Seo, D. A. Miller, and T. O'Bannon, 1998: The WSR-88D rainfall algorithm. Wea. Forecasting, 13, 377-395, doi:10.1175/1520-0434(1998)013,0377: TWRA.2.0.CO;2.
18. Gilleland, E., 2010: Confidence intervals for forecast verification. NCAR Tech. Note NCAR/TN-4791STR, 71 pp., doi:10.5065/ D6WD3XJM.
19. Gilleland, E., D. Ahijevych, and B. G. Brown, 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 1416-1430, doi:10.1175/2009WAF2222269.1.
20. Gilleland, E., D. Ahijevych, B. G. Brown, and E. Ebert, 2010: Verifying forecasts spatially. Bull. Amer. Meteor. Soc., 91, 1365-1373, doi:10.1175/ 2010BAMS2819.1.
21. Grams, J. S., W. A. Gallus Jr., S. E. Koch, L. S. Wharton, A. Loughe, and E. E. Ebert, 2006: The use of a modified Ebert-McBride technique to evaluate mesoscale model QPF as a function of convective system morphology during IHOP 2002. Wea. Forecasting, 21, 288-306, doi:10.1175/WAF918.1.
22. Hamill, T. M., 1999: Hypothesis tests for evaluating numerical precipitation forecasts. Wea. Forecasting, 14, 155-167, doi:10.1175/1520-0434(1999)014,0155:HTFENP.2.0.CO;2.
23. Hunter, S. M., 1996: WSR-88D radar rainfall estimation: Capabilities, limitations and potential improvements. Natl. Wea. Dig., 20, 26-41.
24. Janjić, Z. I., 2003: A nonhydrostatic model based on a new approach. Meteor. Atmos. Phys., 82, 271-285, doi:10.1007/ s00703-001-0587-6.
25. Janjić Z.I., 2004: The NCEP WRF core. Preprints, 20th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., 12.7. [Available online at http://ams.confex.com/ams/ pdfpapers/70036.pdf.]
26. Johnson, A., X. Wang, and M. Xue, 2013: Object-based evaluation of the impact of horizontal grid spacing on convectionallowing forecasts. Mon. Wea. Rev., 141, 3413-3425, doi:10.1175/MWR-D-13-00027.1.
27. Jolliffe, I. T., and D. B. Stephenson, 2011: Forecast Verification. A Practitioner's Guide in Atmospheric Science. John Wiley and Sons, 240 pp.
28. Kain, J. S., S. J. Weiss, J. J. Levit, M. E. Baldwin, and D. R. Bright, 2006: Examination of convection-allowing configurations of the WRF Model for the prediction of severe convective weather: The SPC/NSSL Spring Program 2004. Wea. Forecasting, 21, 167-181, doi:10.1175/WAF906.1.
29. Lin, Y., and K. E. Mitchell, 2005: The NCEP stage II/IV hourly precipitation analyses: Development and applications. Preprints, 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 1.2. [Available online at https://ams.confex.com/ ams/pdfpapers/83847.pdf.]
30. Maddox, R. A., J. Zhang, J. J. Gourley, and K. W. Howard, 2002: Weather radar coverage over the contiguousUnited States. Wea. Forecasting, 17, 927-934, doi:10.1175/1520-0434(2002)017,0927: WRCOTC.2.0.CO;2.
31. Mass, C. F., D. Ovens, K. Westrick, and B. A. Colle, 2002: Does increasing horizontal resolution produce more skillful forecasts? Bull. Amer. Meteor. Soc., 83, 407-430, doi:10.1175/ 1520-0477(2002)083,0407:DIHRPM.2.3.CO;2.
32. Mittermaier, M., and N. Roberts, 2010: Intercomparison of spatial forecast verification methods: Identifying skillful spatial scales using the fractions skill score. Wea. Forecasting, 25, 343-354, doi:10.1175/2009WAF2222260.1.
33. Mittermaier, M., N. Roberts, and S. A. Thompson, 2013: A long-term assessment of precipitation forecast skill using the Fractions skill score. Meteor. Appl., 20, 176-186, doi:10.1002/met.296.
34. R Development Core Team, cited 2013: R: A language and environment for statistical computing.RFoundation for Statistical Computing, Vienna, Austria. [Available online at http://www. R-project.org.]
35. Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations fromhigh-resolution forecasts of convective events.Mon. Wea. Rev., 136, 78-97, doi:10.1175/2007MWR2123.1.
36. Schwartz, C. S., and Coauthors, 2009: Next-day convectionallowing WRF Model guidance: A second look at 2-km versus 4-km grid spacing. Mon. Wea. Rev., 137, 3351-3372, doi:10.1175/2009MWR2924.1.
37. Weisman, M. L., C. Davis, W. Wang, K. W. Manning, and J. B. Klemp, 2008: Experiences with 0-36-h explicit convective forecasts with the WRF-ARW Model. Wea. Forecasting, 23, 407-437, doi:10.1175/2007WAF2007005.1.
38. Westrick, K. J., C. F. Mass, and B. A. Colle, 1999: The limitations of the WSR-88D radar network for quantitative precipitation measurement over the coastal western United States. Bull. Amer.Meteor. Soc., 80, 2289-2298, doi:10.1175/ 1520-0477(1999)080,2289:TLOTWR.2.0.CO;2.
39. Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 2nd ed. Elsevier, 704 pp.