Evaluating probability forecasts for GDP declines using alternative methodologies

International Journal of Forecasting

Volumn 29, Issue 1, 2013, Pages 175-190

  Lahiri, Kajal ^a   Wang, J George ^b  

^a STATE UNIVERSITY OF NEW YORK   (United States)

^b CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

Binary prediction; Calibration; Odds ratio; Rare events; Recession; Relative operating characteristics; Resolution; Skill score; Subjective probability; Survey of professional forecasters

Indexed keywords

EID: 84868641858     PISSN: 01692070     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ijforecast.2012.07.004     Document Type: Article

References (60)

1. Agresti A. An introduction to categorical analysis 2007, John Wiley and Sons, New Jersey. 2nd ed.
2. Agresti A., Coull B.A. Approximation is better than exact for interval estimation of binomial proportions. The American Statistician 1998, 52:1-7.
3. Berge T., Jordà Ò. Evaluating the classification of economic activity into recessions and expansions. American Economic Journal: Macroeconomics 2011, 3:246-277.
4. Blöchlinger A., Leippold M. A new goodness-of-fit test for event forecasting and its application to credit defaults. Management Science 2011, 57:487-505.
5. Braun P., Yaniv I. A case study of expert judgment: economists' probabilities versus base rate model forecasts. Journal of Behavioral Decision Making 1992, 5:217-231.
6. Clements M.P. Consensus and uncertainty: using forecast probabilities of output declines. International Journal of Forecasting 2008, 24:76-86.
7. Clements M.P. Internal consistency of survey respondents' forecasts: evidence based on the survey of professional forecasters. The methodology and practice of econometrics. A festschrift in honour of David F. Hendry 2009, 206-226. Oxford University Press. J.L. Castle, N. Shephard (Eds.).
8. Clements M.P. An empirical investigation of the effects of rounding on the SPF probabilities of decline and output growth histograms. Journal of Money, Credit and Banking 2011, 43:207-220.
9. Clements M.P., Harvey D.I. Forecasting encompassing tests and probability forecasts. Journal of Applied Econometrics 2010, 25:1028-1062.
10. Clemen R.T., Winkler R.L. Aggregating probability distributions. Advances in decision analysis: from foundations to applications 2007, 154-176. Cambridge University Press, Cambridge. E. Ward, R.F. Miles, D. von Winterfeldt (Eds.).
11. Cramer J. Predictive performance of the binary logit model in unbalanced samples. Journal of the Royal Statistical Society, Series D (The Statistician) 1999, 48:85-94.
12. Dawid A.P. Statistical theory: a prequential approach. Journal of the Royal Statistical Society 1984, 147:279-297.
13. Dawid A.P. Probability forecasting. Encyclopedia of statistical sciences 1986, Vol. 7:210-218. Wiley-Interscience, New York. S. Kotz, N.L. Johnson, C.B. Mead (Eds.).
14. DeGroot M.H., Fienberg S.E. A comparison and evaluation of forecasters. The Statistician 1983, 32:12-22.
15. Diebold F.X., Mariano R.S. Comparing predictive accuracy. Journal of Business and Economic Statistics 1995, 13(6):253-263.
16. Diebold F.X., Rudebusch G.D. Turning point prediction with the composite leading index: an ex ante analysis. Leading economic indicators: new approaches and forecasting approaches 1991, Cambridge University Press, Cambridge. K. Lahiri, G.H. Moore (Eds.).
17. Doswell C.A., Davies-Jones R., Keller D.L. On summary measures of skill in rare event forecasting based on contingency tables. Weather and Forecasting 1990, 5:576-585.
18. Elliott G., Timmermann A. Economic forecasting. Journal of Economic Literature 2008, 46(1):3-56.
19. Erev I., Wallsten T.S., Budescu D. Simultaneous over- and under confidence: the role of error in judgment process. Psychological Review 1994, 101:519-527.
20. Fawcett T. An introduction to ROC analysis. Pattern Recognition Letters 2006, 27:861-874.
21. Fildes R., Stekler H.O. The state of macroeconomic forecasting. Journal of Macroeconomics 2002, 24(4):435-468.
22. Galbraith J.W., van Norden S. Kernel-based calibration diagnostics for recession and inflation probability forecasts. International Journal of Forecasting 2011, 27:1041-1057.
23. Graham H.R. Is a group of forecasters better than one? than none?. Journal of Business 1996, 69(2):193-232.
24. Granger C.W.J. Can we improve the perceived quality of economic forecasts?. Journal of Applied Econometrics 1996, 11:455-473.
25. Granger C.W.J., Pesaran M.H. Economic and statistical measures of forecast accuracy. Journal of Forecasting 2000, 19:537-560.
26. Hall P., Hyndman R.J., Fan Y. Nonparametric confidence intervals for receiver operating characteristic curves. Biometrica 2004, 91(3):743-750.
27. Hanley J.A., McNeil B. The meaning and use of the area under the receiver operating characteristics (ROC) curve. Diagnostic Imaging 1982, 29:307-335.
28. Hastie T., Tibshirani R., Friedman J. The elements of statistical learning: data mining, inference, and prediction 2001, Springer.
29. Hogg R., Craig A. Introduction to mathematical statistics 1994, Prentice Hall.
30. Isiklar G., Lahiri K. How far ahead can we forecast? Evidence from cross-country surveys. International Journal of Forecasting 2007, 23:167-187.
31. Forecasting verification: a practitioner's guide in atmospheric science 2003, John Wiley & Son Limited. I.T. Jolliffe, D.B. Stephenson (Eds.).
32. Jolliffe I.T., Stephenson D.B. Proper scores for probability forecasts can never be equitable. Monthly Weather Review 2008, 136:1505-1510.
33. Lahiri K., Wang J.G. Predicting cyclical turning points with leading index in a Markov switching model. Journal of Forecasting 1994, 13(3):245-263.
34. Lahiri K., Wang J.G. Subjective probability forecasts for recessions: guidelines for use. Business Economics 2006, 41(2):26-37.
35. Lopez J.A. Evaluating the predictive accuracy of volatility models. Journal of Forecasting 2001, 20:87-109.
36. Ma G., Hall W.J. Confidence bands for the receiver operating characteristics curves. Medical Decision Making 1993, 13:191-197.
37. Manski C., Molinari F. Rounding of probabilistic expectations in surveys. Journal of Business and Economic Statistics 2010, 28:219-231.
38. Manzato A. A note on the maximum Peirce score. Weather and Forecasting 2007, 22:1148-1154.
39. Murphy A. Scalar and vector partitions of the probability score: Part I. Two-state situation. Journal of Applied Meteorology 1972, 11:273-282.
40. Murphy A.H. Skill scores based on the mean square error and their relationships to the correlation coefficient. Monthly Weather Review 1988, 116:2417-2424.
41. Murphy A.H. Probabilities, odds, and forecasters of rare events. Weather and Forecasting 1991, 6:302-306.
42. Murphy A.H., Wilks D.S. A case study of the use of statistical models in forecast verification: precipitation probability forecasts. Weather and Forecasting 1998, 13:795-810.
43. Murphy A.H., Winkler R.L. Diagnostic verification of probability forecasts. International Journal of Forecasting 1992, 7:435-455.
44. Peirce C.S. The numerical measure of the success of predictions. Science 1884, 4:453-454.
45. Pesaran M.H., Timmermann A. A simple nonparametric test of predictive performance. Journal of Business and Economic Statistics 1992, 10:461-465.
46. Pesaran M.H., Timmermann A. Testing dependence among serially correlated multicategory variables. Journal of the American Statistical Association 2009, 104:325-337.
47. Ranjan R., Gneiting T. Combining probability forecasts. Journal of the Royal Statistical Society, Series B 2010, 72(1):71-91.
48. Roebber P.J., Bosart L.F. The complex relationship between forecast skill and forecast value: a real-world analysis. Weather and Forecasting 2006, 11:544-558.
49. Rudebusch G.D., Williams J.C. Forecasting recessions: the puzzle of the enduring power of the yield curve. Journal of Business and Economic Statistics 2009, 27:492-503.
50. Seillier-Moiseiwitsch F., Dawid A.P. On testing the validity of sequential probability forecasts. Journal of the American Statistical Association 1993, 88(421):355-359.
51. Stein R.M. The relationship between default prediction and lending profits: integrating ROC analysis and loan pricing. Journal of Banking and Finance 2005, 29:1213-1236.
52. Stephenson D.B. Use of the "odds ratio" for diagnosing forecast skill. Weather and Forecasting 2000, 15:221-232.
53. Stephenson D.B., Coelho C.A.S., Jolliffe I.T. Two extra components in the Brier score decomposition. Weather and Forecasting 2008, 23:752-757.
54. Ungemach C., Chater N., Stewart N. Are probabilities overweighted or underweighted when rare outcomes are experienced (rarely)?. Psychological Science 2009, 20:473-479.
55. Wilks D.S. Sampling distributions of the Brier score and Brier skill score under serial dependence. Quarterly Journal of the Royal Meteorological Society, Part B 2010, 136:2109-2118.
56. Wright G., Ayton P. Judgmental probability forecasting in the intermediate and medium term. Organizational Behavior and Human Decision Processes 1992, 51:344-363.
57. Yates J.F. External correspondence: decompositions of the mean probability score. Organizational Behavior and Human Performance 1982, 30:132-156.
58. Yates J.F. Subjective probability accuracy analysis. Subjective probability 1994, 381-410. John Wiley, Chichester, UK. G. Wright, P. Ayton (Eds.).
59. Yates J.F., Curley S.P. Conditional distribution analysis of probabilistic forecasts. Journal of Forecasting 1985, 4:61-73.
60. Zellner A., Hong C., Min C.-K. Forecasting turning points in international growth rates using Bayesian exponentially weighted autoregression, time varying parameter, and pooling techniques. Journal of Econometrics 1991, 49:275-304.