Improving predictions using ensemble bayesian model averaging

Political Analysis

Volumn 20, Issue 3, 2012, Pages 271-291

  Montgomery, Jacob M ^a   Hollenbach, Florian M ^b   Ward, Michael D ^b  

^a WASHINGTON UNIVERSITY   (United States)

^b DUKE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84864516626     PISSN: 10471987     EISSN: 14764989     Source Type: Journal    
DOI: 10.1093/pan/mps002     Document Type: Article

References (115)

1. Abramowitz, A. I. 2008. Forecasting the 2008 presidential election with the time-for-change model. PS: Political Science & Politics 41:691-5.
2. Andriole, S. J., and R. A. Young. 1977. Toward the development of an integrated crisis warning system. International Studies Quarterly 21:107-50.
3. Armstrong, J. S. 2001. Combining forecasts. In Principles of forecasting: A handbook for researchers and practitioners, ed. J. S. Amstrong. Norwell, MA: Kluwer Academic.
4. Ascher, W. 1978. Forecasting: An appraisal for policy-makers and planners. Baltimore: Johns Hopkins University Press.
5. Bartels, L. M. 1997. Specification uncertainty and model averaging. American Journal of Political Science 41:641-74.
6. Bartels, L. M., and J. Zaller. 2001. Presidential vote models: A recount. PS: Political Science and Politics 34:9-20.
7. Bates, J., and C. Granger. 1969. The combination of forecasts. Operations Research 20:451-68.
8. Bennett, D. S., and A. C. Stam. 2009. Revisiting predictions of war duration. Conflict Management and Peace Science 26:256-67.
9. Berg, J. E., F. D. Nelson, and T. A. Rietz. 2008. Prediction market accuracy in the long run. International Journal of Forecasting 24:285-300.
10. Berrocal, V. J., A. E. Raftery, T. Gneiting, and R. C. Steed. 2010. Probabilistic weather forecasting for winter road maintenance. Journal of the American Statistical Association 105:522-37.
11. Billio, M., R. Casarin, F. Ravazzolo, and H. K. Van Dijk. 2010. Combining predictive densities using Bayesian filtering with applications to U.S. economics data. Norges Bank Working Paper. http://ssrn.com/abstract51735421 (accessed June 1, 2011).
12. Billio, M., R. Casarin, F. Ravazzolo, and H. K. Van Dijk. 2011. Bayesian combinations of stock price predictions with an application to the Amsterdam exchange index. Tinbergen Institute Discussion Paper No. 2011-082/4. http://www.tinbergen.nl/discussionpapers/11082.pdf (accessed June 1, 2011).
13. Brandt, P. T., M. Colaresi, and J. R. Freeman. 2008. The dynamics of reciprocity, accountability, and credibility. Journal of Conflict Resolution 52:343-74.
14. Brandt, P. T., J. R. Freeman, and P. A. Schrodt. 2011a. Racing horses: Constructing and evaluating forecasts in political science. Paper prepared for the 28th Annual Summer Meeting of the Society for Political Methodology. http://polmeth.wustl.edu/media/Paper/RHMethods20110721small_1.pdf (accessed August 20, 2011).
15. Brandt, P. T., J. R. Freeman, and P. A. Schrodt. 2011b. Real-time, time-series forecasting of inter- and intra-state political conflict. Conflict Management and Peace Science 28:41-64.
16. Brier, G. W. 1950. Verification of forecasts expressed in terms of probability. Monthly Weather Review 78:1-3.
17. Brock, W. A., S. N. Durlauf, and K. D. West. 2007. Model uncertainty and policy evaluation: Some theory and empirics. Journal of Econometrics 136:629-64.
18. Brown, L. B., and H. W. Chappell. 1999. Forecasting presidential elections using history and polls. International Journal of Forecasting 15:127-35.
19. Bueno de Mesquita, B. 2002. Predicting politics. Columbus: Ohio State University Press.
20. Bueno de Mesquita, B. 2011. A new model for predicting policy choices: Preliminary tests. Conflict Management and Peace Science 28:65-85.
21. Campbell, J. E. 1992. Forecasting the presidential vote in the states. American Journal of Political Science 36:386-407.
22. Campbell, J. E. 2008. The trial-heat forecast of the 2008 presidential vote: Performance and value considerations in an open-seat election. PS: Political Science & Politics 41:697-701.
23. Campbell, J. E., and K. A. Wink. 1990. Trial-heat forecasts of the presidential vote. American Politics Research 18:251-69.
24. Chmielecki, R. M., and A. E. Raftery. 2010. Probabilistic visibility forecasting using Bayesian model averaging. Monthly Weather Review 139:1626-36.
25. Choucri, N., and T. W. Robinson, eds. 1978. Forecasting in international relations: Theory, methods, problems, prospects. San Francisco, CA: W. H. Freeman.
26. Clyde, M. 2003. Model averaging. In Subjective and objective Bayesian statistics: Principles, models, and applications, ed. S. J. Press, 320-35. Hoboken, NJ: Wiley-Interscience.
27. Clyde, M., and E. I. George. 2004. Model uncertainty. Statistical Science 19:81-94.
28. Cuzàn, A. G., and C. M. Bundrick. 2008. Forecasting the 2008 presidential election: A challenge for the fiscal model. PS: Political Science & Politics 41:717-22.
29. Davies, J. L., and T. R. Gurr. 1998. Preventive measures: Building risk assessment and crisis early warning systems. Lanham, MD: Rowman & Littlefield.
30. Dawid, A. P. 1982. The well-calibrated Bayesian (with discussion). Journal of the American Statistical Association 77:605-13.
31. Dawid, A. P. 1984. Present position and potential developments: Some personal views. Statistical theory: The prequential approach (with discussion). Journal of the Royal Statistical Society Series A (Statistics in Society) 147:278-92.
32. de Marchi, S., C. Gelpi, and J. D. Grynaviski. 2004. Untangling neural nets. American Political Science Review 98:371-8.
33. de Sola Pool, I., R. P. Abelson, and S. L. Popkin. 1964. Candidates, issues, and strategies: A computer simulation of the 1960 and 1964 presidential elections. Cambridge, MA: MIT Press.
34. Draper, D. 1995. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society Series B (Methodological) 57:45-97.
35. Enders, W., and T. M. Sandler. 2005. After 9/11: Is it all different now? Journal of Conflict Resolution 49:259-77.
36. Erikson, R. S., and C. Wlezien. 2008. Leading economic indicators, the polls, and the presidential vote. PS: Political Science & Politics 41:703-7.
37. Fair, R. C. 1978. The effect of economic events on votes for president. Review of Economics and Statistics 60:159-73.
38. Fair, R. C. 2010. Presidential and congressional vote-share equations: November 2010 update.Working Paper, Yale University. http://fairmodel.econ.yale.edu/RAYFAIR/PDF/2010C.pdf (accessed June 7, 2011).
39. Fearon, J. D., and D. D. Laitin. 2003. Ethnicity, insurgency, and civil war. American Political Science Review 97:75-90.
40. Feder, S. A. 2002. Forecasting for policy-making in the post-Cold War period. Annual Review of Political Science 5:111-25.
41. Forthcoming Forecast combination and Bayesian model averaging: A prior sensitivity analysis
42. Fraley, C., A. E. Raftery, and T. Gneiting. 2010. Calibrating multimodel forecast ensembles with exchangeable and missing members using Bayesian model averaging. Monthly Weather Review 138:190-202.
43. Fraley, C., A. E. Raftery, T. Gneiting, J. M. Sloughter, and V. J. Berrocal. 2011. Probabilistic weather forecasting in R. R Journal 3:55-63.
44. Fraley, C., A. E. Raftery, J. M. Sloughter, and T. Gneiting. 2010. EnsembleBMA: Probabilistic forecasting using ensembles and Bayesian model averaging. R package version 4.5. http://CRAN.R-project.org/package=ensembleBMA.
45. Freeman, J. R., and B. L. Job. 1979. Scientific forecasts in international relations: Problems of definition and epistemology. International Studies Quarterly 23:113-43.
46. Geer, J., and R. R. Lau. 2006. Filling in the blanks:A new method for estimating campaign effects. British Journal of Political Science 36:269-90.
47. Gill, J. 2004. Introduction to the special issue. Political Analysis 12:647-74.
48. Gleditsch, K. S., and M. D.Ward. 2010. Contentious issues and forecasting interstate disputes. Presented at the 2010 Annual Meeting of the International Studies Association, New Orleans, LA.
49. Gneiting, T., and A. E. Raftery. 2007. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102:359-78.
50. Gneiting, T., and T. L. Thorarinsdottir. 2010. Predicting inflation: Professional experts versus no-change forecasts. Working Paper. http://arxiv.org/abs/1010.2318v1http://arxiv.org/abs/1010.2318v1 (accessed June 15, 2011).
51. Graefe, A., A. G. Cuzan, R. J. Jones, and J. S. Armstrong. 2010. Combining forecasts for U.S. presidential elections: The PollyVote. Working Paper. http://dl.dropbox.com/u/3662406/Articles/Graefe_et_al_Combining.pdf (accessed May 15, 2011).
52. Greenhill, B. D., M. D. Ward, and A. Sacks. 2011. The separation plot: A new visual method for evaluating the fit of binary data. American Journal of Political Science 55:990-1002.
53. Gurr, T. R., and M. I. Lichbach. 1986. Forecasting internal conflict: A competitive evaluation of empirical theories. Comparative Political Studies 19:3-38.
54. Hamill, T. S., J. S. Whitaker, and X. Wei. 2004. Ensemble reforecasting: Improving medium-range forecast skill using retrospective forecasts. Monthly Weather Review 132:1434-47.
55. Hastie, T., R. Tibshirani, and J. Friedman. 2009. The elements of statistical learning: Data mining, inference, and prediction. New York: Springer.
56. Hausegger, L., and L. Baum. 1999. Inviting congressional action: A study of Supreme Court motivations in statutory interpretation. American Journal of Political Science 43:162-85.
57. Hibbs, D. A. 2000. Bread and peace voting in U.S. presidential elections. Public Choice 104:149-80.
58. Hildebrand, D. K., J. D. Laing, and H. Rosenthal. 1976. Prediction analysis in political research. American Political Science Review 70:509-35.
59. Hoeting, J. A., D. Madigan, A. E. Raftery, and C. T. Volinsky. 1999. Bayesian model averaging: A tutorial. Statistical Science 14:382-417.
60. Holbrook, T. M. 2008. Incumbency, national conditions, and the 2008 presidential election. PS: Political Science & Politics 41:709-12.
61. Huisman, J., L. Breuer, H. Bormann, A. Bronstert, B. Croke, H.-G. Frede, T. Gräff, L. Hubrechts, A. Jakeman, G. Kite, et al. 2009. Assessing the impact of land-use change on hydrology by ensemble modeling (LUCHEM) II: Ensemble combinations and predictions. Advances in Water Resources 32:147-58.
62. Imai, K., and G. King. 2004. Did illegal overseas absentee ballots decide the 2000 U.S. presidential election? Perspectives on Politics 2:537-49.
63. Jerome, B., V. Jerome, and M. S. Lewis-Beck. 1999. Polls fail in France: Forecasts of the 1997 legislative election. International Journal of Forecasting 15:163-74.
64. King, G., and L. Zeng. 2001. Improving forecasts of state failure. World Politics 53:623-58.
65. Klein, D. E., and R. J. Hume. 2003. Fear of reversal as an explanation of lower court compliance. Law & Society Review 37:579-606.
66. Koop, G., and D. Korobilis. 2009. Forecasting inflation using dynamic model averaging.Working Paper. http://personal.strath.ac.uk/gary.koop/koop_korobilis_forecasting_inflation_using_DMA.pdf (accessed May 25, 2011).
67. Krause, G. A. 1997. Voters, information heterogeneity, and the dynamics of aggregate economic expectations. American Journal of Political Science 41:1170-200.
68. Leblang, D., and S. Satyanath. 2006. Institutions, expectations, and currency crises. International Organization 60:245-62.
69. Lewis-Beck, M. S. 2005. Election forecasting: Principles and practice. British Journal of Politics&International Relations 7:145-64.
70. Lewis-Beck, M. S., and C. Tien. 2008. The job of president and the jobs model forecast: Obama for '08? PS: Political Science & Politics 41:687-90.
71. Lock, K., and A. Gelman. 2010. Bayesian combination of state polls and election forecasts. Political Analysis 18:337-48.
72. Lockerbie, B. 2008. Election forecasting: The future of the presidency and the house. PS: Political Science & Politics 41:713-6.
73. Madigan, D., and A. E. Raftery. 1994. Model selection and accounting for model uncertainty in graphical models using Occam's window. Journal of the American Statistical Association 89:1535-46.
74. Marshall, M. G., K. Jaggers, and T. R. Gurr. 2009. Polity IV project: Political regime characteristics and transition 1800-2007. College Park, MD: CIDCM, University of Maryland.
75. Martin, A. D., K. M. Quinn, T.W. Ruger, and P. T. Kim. 2004. Competing approaches to predicting Supreme Court decision-making. Perspectives on Politics 2:761-7.
76. McCandless, T. C., S. E. Haupt, and G. S. Young. 2011. The effects of imputing missing data on ensemble temperature forecasts. Journal of Computers 6:162-71.
77. McCormick, T. H., A. E. Raftery, D. Madigan, and R. S. Burd. 2011. Dynamic logistic regression and dynamic model averaging for binary classification. Working Paper. http://www.stat.columbia.edu/madigan/PAPERS/ldbma27.pdf (accessed March 26, 2011).
78. Min, S.-K., and A. Hense. 2006. A Bayesian approach to climate model evaluation and multi-model averaging with an application to global mean surface temperatures from IPCC AR4 coupled climate models. Geophysical Research Letters 33:L08708.
79. Min, S.-K., D. Simonis, and A. Hense. 2007. Probabilistic climate change predictions applying Bayesian model averaging. Philosophical Transactions of the Royal Society A: Mathematical, Physical, and Engineering Sciences 365:2103-16.
80. Montgomery, J. M., F. Hollenbach, and M. D. Ward. 2012. Replication data for: Improving predictions using ensemble Bayesian model averaging. IQSS Dataverse Network. http://hdl.handle.net/1902.1/17286.
81. Montgomery, J. M., and B. Nyhan. 2010. Bayesian model averaging: Theoretical developments and practical applications. Political Analysis 18:245-70.
82. Muhlbaier, M. D., and R. Polikar. 2007. An ensemble approach for incremental learning in nonstationary environments. Multiple Classifier Systems 4472:490-500.
83. Norpoth, H. 2008. On the razor's edge: The forecast of the primary model. PS: Political Science & Politics 41:683-6.
84. O'Brien, S. P. 2002. Anticipating the good, the bad, and the ugly: An early warning approach to conflict and instability analysis. Journal of Conflict Resolution 46:791-811.
85. O'Brien, S. P. 2010. Crisis early warning and decision support: Contemporary approaches and thoughts on future research. International Studies Review 12:87-104.
86. Page, S. E. 2008. Uncertainty, difficulty, and complexity. Journal of Theoretical Politics 20:115-49.
87. Page, S. E. 2011. Diversity and complexity. Princeton, NJ: Princeton University Press.
88. Page, S. E., L. M. Sander, and C. M. Schneider-Mizell. 2007. Conformity and dissonance in generalized voter models. Journal of Statistical Physics 128:1279-87.
89. Pevehouse, J. C., and J. S. Goldstein. 1999. Serbian compliance or defiance in Kosovo? Statistical analysis and real-time predictions. Journal of Conflict Resolution 43:538-46.
90. Raftery, A. E. 1995. Bayesian model selection in social research. Sociological Methodology 25:111-63.
91. Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski. 2005. Using Bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review 133:1155-74.
92. Raftery, A. E., M. Kárný, and P. Ettler. 2010. Online prediction under model uncertainty via dynamic model averaging: Application to a cold rolling mill. Technometrics 52:52-66.
93. Raftery, A. E., and Y. Zheng. 2003. Long-run performance of Bayesian model averaging. Journal of the American Statistical Association 98:931-8.
94. Richards, M. J., and H. M. Kritzer. 2002. Jurisprudential regimes in Supreme Court decision-making. American Political Science Review 96:305-20.
95. Rosenstone, S. J. 1983. Forecasting presidential elections. New Haven, CT: Yale University Press.
96. Ruger, T. W., P. T. Kim, A. D. Martin, and K. M. Quinn. 2004. The Supreme Court Forecasting Project: Legal and political science approaches to predicting Supreme Court decision-making. Columbia Law Review 104:1150-210.
97. Schneider, G., N. P. Gleditsch, and S. Carey. 2011. Forecasting in international relations: One quest, three approaches. Conflict Management and Peace Science 28:5-14.
98. Schrodt, P. A., and D. J. Gerner. 2000. Using cluster analysis to derive early warning indicators for political change in the Middle East, 1979-1996. American Political Science Review 94:803-18.
99. Segal, J. A., and A. D. Cover. 1989. Ideological values and the votes of U.S. Supreme Court Justices. American Political Science Review 83:557-65.
100. Singer, J. D., and M. D. Wallace. 1979. To augur well: Early warning indicators in world politics. Beverly Hills, CA: Sage.
101. Sloughter, J. M., T. Gneiting, and A. E. Raftery. 2010. Probabilistic wind-speed forecasting using ensembles and Bayesian model averaging. Journal of the American Statistical Association 105:25-35.
102. Sloughter, J. M., A. E. Raftery, T. Gneiting, and C. Fraley. 2007. Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Monthly Weather Review 135:3209-20.
103. Smith, R. L., C. Tebaldi, D. Nychka, and L. O. Mearns. 2009. Bayesian modeling of uncertainty in ensembles of climate models. Journal of the American Statistical Association 104:97-116.
104. Songer, D. R., J. A. Segal, and C. M. Cameron. 1994. The hierarchy of justice: Testing a principal-agent model of Supreme Courtcircuit court interactions. American Journal of Political Science 38:673-96.
105. Spirtes, P., C. N. Glymour, and R. Scheines. 2000. Causation, prediction, and search. Vol. 81. Cambridge, MA: MIT Press.
106. Tomas, A. 2011. A dynamic logistic multiple classifier system for online classification. Working Paper. http://www.stats.ox.ac.uk/tomas/html_links/T2011.pdf (accessed June 1, 2011).
107. Vincent, J. E. 1980. Scientific prediction versus crystal ball gazing: Can the unknown be known? International Studies Quarterly 24:450-4.
108. Vrugt, J. A., M. P. Clark, C. G. Diks, Q. Duan, and B. A. Robinson. 2006. Multi-objective calibration of forecast ensembles using Bayesian model averaging. Geophysical Research Letters 33:L19817.
109. Vrugt, J. A., C. G. Diks, and M. P. Clark. 2008. Ensemble Bayesian model averaging using Markov chain Monte Carlo sampling. Environmental Fluid Mechanics 8:579-95.
110. Ward, M. D., B. D. Greenhill, and K. M. Bakke. 2010. The perils of policy by p-value: Predicting civil conflict. Journal of Peace Research 47:363-75.
111. Ward, M. D., R. M. Siverson, and X. Cao. 2007. Disputes, democracies, and dependencies: A re-examination of the Kantian peace. American Journal of Political Science 51:583-601.
112. Whiteley, P. F. 2005. Forecasting seats from votes in British general elections. British Journal of Politics & International Relations 7:165-73.
113. Wright, J. H. 2008. Bayesian model averaging and exchange rate forecasts. Journal of Econometrics 146:329-41.
114. Wright, J. H. 2009. Forecasting U.S. inflation by Bayesian model averaging. Journal of Forecasting 28:131-44.
115. Zhang, X., R. Srinivasan, and D. Bosch. 2009. Calibration and uncertainty analysis of the SWAT model using genetic algorithms and Bayesian model averaging. Journal of Hydrology 374:307-17.