Reliability diagrams for non-parametric density forecasts of continuous variables: Accounting for serial correlation

Quarterly Journal of the Royal Meteorological Society

Volumn 136, Issue 646, 2010, Pages 77-90

  Pinson, Pierre ^a   McSharry, Patrick ^b,c   Madsen, Henrik ^a  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Calibration; Consistency resampling; Probabilistic forecasting; Surrogate; Verification; Wind power

Indexed keywords

COMBINED EFFECT; CONTINUOUS VARIABLES; COUNTING STATISTICS; DENSITY FORECAST; DIAGNOSTIC TOOLS; NON-PARAMETRIC; PROBABILISTIC FORECASTING; PROBABILISTIC FORECASTS; RELIABILITY ASSESSMENTS; RELIABILITY EVALUATION; RESAMPLING; RESAMPLING METHOD; SAMPLING EFFECTS; SERIAL CORRELATION; SURROGATE; SYSTEMATIC BIAS;

CALIBRATION; CLUSTER ANALYSIS; DISCRIMINANT ANALYSIS; GRAPHIC METHODS; REGRESSION ANALYSIS; RELIABILITY ANALYSIS; WIND POWER;

ELECTRIC LOAD FORECASTING;

CALIBRATION; CORRELATION; DIAGRAM; FORECASTING METHOD; MODEL VALIDATION; PROBABILITY DENSITY FUNCTION; RELIABILITY ANALYSIS; SAMPLING BIAS; SURROGATE METHOD; WIND POWER;

EID: 77249106476     PISSN: 00359009     EISSN: 1477870X     Source Type: Journal    
DOI: 10.1002/qj.559     Document Type: Article

References (48)

1. Abramson B, Clemen R. 1995. Probability forecasting. Int. J. Forecasting 11: 1-4.
2. AhnH,ChenJJ.1995.Generationofover-dispersedandunder-dispersed Binomial variates. J. Comput. Graph. Stat. 4: 55-64.
3. Atger F. 1998. The skill of ensemble prediction systems. Mon. Weather Rev. 127: 1941-1953.
4. Atger F. 2004. Estimation of the reliability of ensemble probabilistic forecasts. Q. J. R. Meteorol. Soc. 130: 1509-1523.
5. Berkowitz J. 2001. Testing density forecasts, with applications to risk management. J. Business Econ. Statistics 19: 465-474.
6. Bremnes JB. 2006. A comparison of a few statistical models for making quantile wind power forecasts. Wind Energy 9: 3-11.
7. Bröcker J. 2009. On reliability analysis of multi-categorical forecasts. Nonlinear Processes in Geophys. 15: 661-673.
8. Bröcker J, Smith LA. 2007a. Increasing the reliability of reliability diagrams. Weather and Forecasting 22: 651-661.
9. Bröcker J, Smith LA. 2007b. Scoring probabilistic forecasts: The importance of being proper. Weather and Forecasting 22: 382-388.
10. Christoffersen PF. 1998. Evaluating interval forecasts. Int. Econ. Rev. 39: 841-862.
11. Costa A, Crespo A, Navarro J, Lizcano G, Madsen H, Feitosa E. 2008. A review on the young history of the wind power short-term prediction. Renewable and Sustainable Energy Reviews 12: 1725-1744.
12. Davison AC, Hinkley DV. 1997. Bootstrap Methods and their Application, Cambridge Series in Statistical and Probabilistic Methods. Cambridge University Press: Cambridge.
13. De Oliveira V. 2003. A note on the correlation structure of transformed Gaussian random fields. Aust. N. Z. J. Stat. 45: 353-366.
14. Diebold FX, Gunther TA, Tay AS. 1998. Evaluating density forecasts with applications to financial risk management. Int. Econ. Rev. 39: 863-883.
15. Giebel G, Kariniotakis G, Brownsword R. 2003. 'The state of the art in short-term prediction of wind power - A literature overview'. Technical Report, ANEMOS EU project, deliverable report D1.1. Available online at http://www.anemos-project.eu.
16. Gneiting T. 2008a. Editorial: Probabilistic forecasting. J. R. Stat. Soc. A 171: 319-321.
17. Gneiting T. 2008b. 'Quantiles as optimal point predictors'. Technical Report No. 538. University of Washington, Department of Statistics: Seattle, USA.
18. Gneiting T, Larson K, Westrick K, Genton MG, Aldrich E. 2006. Calibrated probabilistic forecasting at the stateline wind energy center - The regime-switching space-time method. J. Amer. Stat. Assoc. 101: 968-979.
19. Gneiting T, Balabdaoui F, Raftery AE. 2007. Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. B 69: 243-268.
20. Hall P, Rieck A. 2001. Improving coverage accuracy of nonparametric prediction intervals. J. R. Stat. Soc. B 63: 717-725.
21. Hamill T. 2001. Interpretationof rank histograms for verifyingensemble forecasts. Mon. Weather Rev. 129: 550-560.
22. Jolliffe IT. 2007. Uncertainty and inference for verification measures. Weather and Forecasting 22: 137-150.
23. Jolliffe IT, Stephenson DB. 2003. Forecast Verification: A Practitioner's Guide in Atmospheric Science. Wiley: New York.
24. Lange M. 2005. On the uncertainty of wind power predictions - Analysis of the forecast accuracy and statistical distribution of errors. J. Sol. Energy - Trans. ASME 127: 177-184.
25. Lahiri SN. 2003. Resampling Methods for Dependent Data, Springer Series in Statistics. Springer: Berlin.
26. Leutbecher M, Palmer TN. 2008. Ensemble forecasting. J. Comput. Phys. 227: 3515-3539.
27. Madsen H. 2007. Time Series Analysis. Chapman & Hall/CRC: London.
28. Matos M, Bessa R. 2009. 'Decision support tools for power balance and reserve management'. Technical Report, ANEMOS.plus EU project, deliverable report D3.3. INESC Porto: Porto, Portugal.
29. Møller JK, Nielsen HAa, Madsen H. 2008. Time-adaptive quantile regression. Comput. Stat. Data Analysis 52: 1292-1303.
30. Murphy AH. 1993. What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather and Forecasting 8: 281-293.
31. Nielsen TS, Madsen H, Nielsen HA. 2002. 'Prediction of wind power using time-varying coefficient functions'. In ProceedingsofIFAC 2002, 15th World Congress on Automatic Control, Barcelona, Spain. Elsevier: Amsterdam, The Netherlands.
32. Nielsen HA, Nielsen TS, Madsen H, Badger J, Giebel G, Landberg L, Sattler K, Voulund L, Tøfting J. 2006. 'From wind ensembles to probabilistic information about future wind power production - Results from an actual application'. In Proceedings of IEEE PMAPS Conference, Probabilistic Methods Applied to Power Systems, Stockholm, Sweden.
33. Pinson P. 2006. 'Estimation of the uncertainty in wind power forecasting'. PhD Thesis. Ecole des Mines de Paris: Paris, France. Available onlineat http://pastel.paristech.org.
34. Pinson P, Madsen H. 2009a. Ensemble-based probabilistic forecasting at Horns Rev. Wind Energy 12: 137-155.
35. Pinson P, Madsen H. 2009b. 'Adaptive modeling and forecasting of wind power fluctuations with Markov-switching autoregressive models'. Technical Report. Technical University of Denmark: DTU Informatics, Denmark.
36. Pinson P, Chevallier C, Kariniotakis G. 2007a. Trading wind generation with short-term probabilistic forecasts of wind power. IEEE Trans. Power Systems 22: 1148-1156.
37. Pinson P, Nielsen HAa, Møller JK, Madsen H, Kariniotakis G. 2007b. Nonparametric probabilistic forecasts of wind power: required properties and evaluation. Wind Energy 10: 497-516.
38. Schreiber T, Schmitz A. 2000. Surrogate time series. Physica D 142: 346-382.
39. Smith LA. 1997. The maintenance of uncertainty. Proc. International School of Physics Enrico Fermi 133: 177-246.
40. Tay AS, Wallis KF. 2000. Density forecasting: a survey. J. Forecasting 19: 235-254.
41. Taylor JW.1999.Evaluating volatility and interval forecasts. J. Forecasting 18: 111-128.
42. Taylor JW, Buizza R. 2006. Density forecasting for weather derivative pricing. Int. J. Forecasting 22: 29-42.
43. Theiler J, Eubank S, Longtin A, Galdrikan B, Farmer JD. 1992. Testing for nonlinearity in time-series: the method of surrogate data. Physica D 58: 77-94.
44. Timmermann A. 2000. Density forecasting in economics and finance. J. Forecasting 19: 231-234.
45. Tsai CA, Hsueh HM, Chen JJ. 2003. Estimation of false discovery rates in multiple testing: Application to microarray data. Biometrics 59: 1071-1081.
46. Upton G, Cook I. 2002. A Dictionary of Statistics. Oxford University Press: Oxford.
47. Vincent CL, Giebel G, Pinson P, Madsen H. 2009. Resolving non-stationary spectral signals in wind speed time-series using the Hilbert-Huang transform. J. Appl. Meteorol. Clim. In press. DOI:10.1175/2009JAMC2058.1.
48. Wilks DS. 1995. Statistical Methods in the Atmospheric Sciences, 2nd edn, International Geophysics Series Vol. 59. Academic Press: New York.