Golden rule of forecasting: Be conservative

Journal of Business Research

Volumn 68, Issue 8, 2015, Pages 1717-1731

  Armstrong, J Scott ^a,b   Green, Kesten C ^b   Graefe, Andreas ^c  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

^c UNIVERSITY OF MUNICH   (Germany)

Author keywords

Analytics; Bias; Big data; Checklists; Combining; Uncertainty

Indexed keywords

EID: 84930276395     PISSN: 01482963     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbusres.2015.03.031     Document Type: Review

References (125)

1. Allen P.G. Economic forecasting in agriculture. International Journal of Forecasting 1994, 10(1):81-135.
2. Armstrong J.S. An application of econometric models to international marketing. Journal of Marketing Research 1970, 7(2):190-198.
3. Armstrong J.S. The seer-sucker theory: The value of experts in forecasting. Technology Review 1980, June/July, 83:18-24.
4. Armstrong J.S. Long-range forecasting: From crystal ball to computer 1985, Wiley, New York.
5. Armstrong J.S. Judgmental bootstrapping: Inferring experts' rules for forecasting. Principles of forecasting: A handbook for researchers and practitioners 2001, 171-192. Springer, New York. J.S. Armstrong (Ed.).
6. Armstrong J.S. Combining forecasts. Principles of forecasting: A handbook for researchers and practitioners 2001, 417-439. Springer, New York. J.S. Armstrong (Ed.).
7. Armstrong J.S. Evaluating forecasting methods. Principles of forecasting: A handbook for researchers and practitioners 2001, 443-472. Springer, New York. J.S. Armstrong (Ed.).
8. Armstrong J.S. Principles of forecasting: A handbook for researchers and practitioners 2001, Springer, New York.
9. Armstrong J.S. Findings from evidence-based forecasting: Methods for reducing forecast error. International Journal of Forecasting 2006, 22(3):583-598.
10. Armstrong J.S. How to make better forecasts and decisions: Avoid face-to-face meetings. Foresight: The International Journal of Applied Forecasting 2006, 5:3-8.
11. Armstrong J.S. Persuasive advertising 2010, Palgrave MacMillan, New York.
12. Armstrong J.S. Illusions in regression analysis. International Journal of Forecasting 2012, 28(3):689-694.
13. Armstrong J.S. Predicting job performance: The moneyball factor. Foresight: The International Journal of Applied Forecasting 2012, 25:31-34.
14. Armstrong J.S., Adya M., Collopy F. Rule-based forecasting: Using judgment in time-series extrapolation. Principles of forecasting: A handbook for researchers and practitioners 2001, 259-282. Springer, New York. J.S. Armstrong (Ed.).
15. Armstrong J.S., Andress J.G. Exploratory analysis of marketing data: Trees vs. regression. Journal of Marketing Research 1970, 7:487-492.
16. Armstrong J.S., Collopy F. Error measures for generalizing about forecasting methods: Empirical comparisons. International Journal of Forecasting 1992, 8:69-80.
17. Armstrong J.S., Collopy F. Causal forces: Structuring knowledge for time-series extrapolation. Journal of Forecasting 1993, 12(2):103-115.
18. Armstrong J.S., Collopy F. Integration of statistical methods and judgment for time series forecasting: Principles from empirical research. Forecasting with judgment 1998, 263-393. Wiley, Chichester. G. Wright, P. Goodwin (Eds.).
19. Armstrong J.S., Collopy F., Yokum J.T. Decomposition by causal forces: A procedure for forecasting complex time series. International Journal of Forecasting 2005, 21(1):25-36.
20. Armstrong J.S., Du R., Green K.C., Graefe A. Predictive validity of evidence-based advertising principles. Working paper 2014, (Available at https://marketing.wharton.upenn.edu/files/?whdmsaction=public:main.file&fileID=6794).
21. Armstrong J.S., Graefe A. Predicting elections from biographical information about candidates: A test of the index method. Journal of Business Research 2011, 64(7):699-706.
22. Armstrong J.S., Green K.C. Effects of corporate social responsibility and irresponsibility policies: Conclusions from evidence-based research. Journal of Business Research 2013, 66:1922-1927.
23. Armstrong J.S., Green K.C., Soon W. Polar bear population forecasts: A public-policy forecasting audit. Interfaces 2008, 38(5):382-405.
24. Ascher W. Forecasting: An appraisal for policy-makers and planners 1978, The Johns Hopkins University Press, Baltimore.
25. Boorman D. Today's electronic checklists reduce likelihood of crew errors and help prevent mishaps. International Civil Aviation Organization Journal 2001, 1:17-36.
26. Booth H. Demographic forecasting: 1980 to 2005 in review. International Journal of Forecasting 2006, 22(3):547-581.
27. Bunn D.W., Vassilopoulos A.I. Comparison of seasonal estimation methods in multi-item short-term forecasting. International Journal of Forecasting 1999, 15(4):431-443.
28. Carson R.T., Cenesizoglu T., Parker R. Forecasting (aggregate) demand for US commercial air travel. International Journal of Forecasting 2011, 27:923-941.
29. Chamberlin T.C. The method of multiple working hypotheses. Science 1890, 1965, 148:754-759. (Reprint of an 1890 paper).
30. Chen H., Boylan J.E. Empirical evidence on individual, group and shrinkage indices. International Journal of Forecasting 2008, 24:525-543.
31. Clemen R.T. Combining forecasts: A review and annotated bibliography. International Journal of Forecasting 1989, 5(4):559-583.
32. Collopy F., Armstrong J.S. Rule-based forecasting: Development and validation of an expert systems approach to combining time series extrapolations. Management Science 1992, 38(10):1394-1414.
33. Dana J., Dawes R.M. The superiority of simple alternatives to regression for social science predictions. Journal of Educational and Behavioral Statistics 2004, 29(3):317-331.
34. Dangerfield B.J., Morris J.S. Top-down or bottom-up: Aggregate versus disaggregate extrapolations. International Journal of Forecasting 1992, 8(2):233-241.
35. Dorn H.F. Pitfalls in population forecasts and projections. Journal of the American Statistical Association 1950, 45:311-334.
36. Erikson R.S., Wlezien C. The timeline of presidential elections: How campaigns do (and do not) matter 2012, University of Chicago Press, Chicago.
37. Fildes R., Goodwin P. Against your better judgment? How organizations can improve their use of management judgment in forecasting. Interfaces 2007, 37(6):570-576.
38. Fildes R., Goodwin P., Lawrence M., Nikolopoulos K. Effective forecasting and judgmental adjustments: An empirical evaluation and strategies for improvement in supply-chain planning. International Journal of Forecasting 2009, 25(1):3-23.
39. Fildes R., Hastings R. The organization and improvement of market forecasting. The Journal of the Operational Research Society 1994, 45(1):1-16.
40. Fildes R., Hibon M., Makridakis S., Meade N. Generalizing about univariate forecasting methods: Further empirical evidence. International Journal of Forecasting 1998, 14:339-358.
41. Fildes R., Makridakis S. The impact of empirical accuracy studies on time series analysis and forecasting. International Statistical Review 1995, 63(3):289-308.
42. Fildes R., Petropoulos F. Simple versus complex selection rules for forecasting many time series. Journal of Business Research 2015, 68(8):1692-1701. (in this issue).
43. Flores B.E., Whybark C.D. A comparison of focus forecasting with averaging and exponential smoothing. Production and Inventory Management 1986, 27(3):96-103.
44. Flyvbjerg B. Quality control and due diligence in project management: Getting decisions right by taking the outside view. International Journal of Project Management 2013, 31(5):760-774.
45. Flyvbjerg B., Skamris Holm M.K., Buhl S.L. How (in)accurate are demand forecasts in public works projects?: The case of transportation. Journal of the American Planning Association 2005, 71:131-146.
46. Franses P.H., Legerstee R. Do experts' adjustments on model-based SKU-level forecasts improve forecast quality?. Journal of Forecasting 2010, 29(3):331-340.
47. Freedman D.A. Statistical models and shoe leather. Sociological Methodology 1991, 21(1):201-313.
48. Gardner E.S. The strange case of the lagging forecasts. Interfaces 1984, 14(3):47-50.
49. Gardner E.S. Further notes on lagging forecasts. Interfaces 1985, 15(5):63.
50. Gardner E.S. Exponential smoothing: The state of the art. Journal of Forecasting 1990, 4:1-28.
51. Gardner E.S. Exponential smoothing: The state of the art-Part II. International Journal of Forecasting 2006, 22:637-666.
52. Gardner E.S., Anderson E.A. Focus forecasting reconsidered. International Journal of Forecasting 1997, 13(4):501-508.
53. Gardner E.S., Anderson-Fletcher E.A., Wickes A.M. Further results on focus forecasting vs. exponential smoothing. International Journal of Forecasting 2001, 17(2):287-293.
54. Gardner E.S., McKenzie E. Forecasting trends in time series. Management Science 1985, 31:1237-1246.
55. Gawande A. The Checklist Manifesto: How to get things right 2010, Metropolitan Books, New York.
56. Goodwin P. Improving the voluntary integration of statistical forecasts and judgment. International Journal of Forecasting 2000, 16(1):85-99.
57. Goodwin P. When simple alternatives to Bayes formula work well: Reducing the cognitive load when updating probability forecasts. Journal of Business Research 2015, 68(8):1686-1691. (in this issue).
58. Goodwin P., Fildes R. Judgmental forecasts of time series affected by special events: Does providing a statistical forecast improve accuracy?. Journal of Behavioral Decision Making 1999, 12(1):37-53.
59. Goodwin P., Meeran S. Robust testing of the utility-based high-technology product sales forecasting methods proposed by Decker and Gnibba-Yukawa (2010). Journal of Product Innovation Management 2012, 29(S1):211-218.
60. Gorr W., Olligschlaeger A., Thompson Y. Short-term forecasting of crime. International Journal of Forecasting 2003, 19(4):579-594.
61. Graefe A. Accuracy of vote expectation surveys in forecasting elections. Public Opinion Quarterly 2014, 78(S1):204-232.
62. Graefe A. Improving forecasts using equally weighted predictors. Journal of Business Research 2015, 68(8):1792-1799. (in this issue).
63. Graefe A., Armstrong J.S. Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task. International Journal of Forecasting 2011, 27(1):183-195.
64. Graefe A., Armstrong J.S. Forecasting elections from voters' perceptions of candidates' ability to handle issues. Journal of Behavioral Decision Making 2013, 26(3):295-303.
65. Graefe A., Armstrong J.S., Jones R.J., Cuzán A.G. Combining forecasts: An application to elections. International Journal of Forecasting 2014, 30(1):43-54.
66. Graefe A., Küchenhoff H., Stierle V., Riedl B. Limitations of ensemble Bayesian model averaging for forecasting social science problems. International Journal of Forecasting 2014, (forthcoming, available at http://ssrn.com/abstract=2266307).
67. Green K.C. Game theory, simulated interaction, and unaided judgement for forecasting decisions in conflicts: Further evidence. International Journal of Forecasting 2005, 21:463-472.
68. Green K.C., Armstrong J.S. Global warming: Forecasts by scientists versus scientific forecasts. Energy & Environment 2007, 18(7-8):997-1021.
69. Green K.C., Armstrong J.S. Structured analogies for forecasting. International Journal of Forecasting 2007, 23(3):365-376.
70. Green K.C., Armstrong J.S. Forecasting global climate change. Climate change: The facts 2014, 170-186. IPA, Melbourne. A. Moran (Ed.).
71. Green K.C., Armstrong J.S. Simple versus complex forecasting: The evidence. Journal of Business Research 2015, 68(8):1792-1799. (in this issue).
72. Groff G.K. Empirical Comparison of Models for Short Range Forecasting. Management Science 1973, 20(1):22-31. http://www.jstor.org/stable/2629066.
73. Hales B.M., Pronovost P.J. The checklist-A tool for error management and performance improvement. Journal of Critical Care 2006, 21:231-235.
74. Harvey N. Why are judgments less consistent in lss predictable task situations. Organizational Behavior and Human Decision Processes 1995, 63:247-263.
75. Hauser P.M. Social statistics in use 1975, Russell Sage, New York.
76. Haynes A.B., Weiser T.G., Berry W.R., Lipsitz S.R., Breizat A.H.S., Dellinger E.P., et al. A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine 2009, 360(5):491-499.
77. Hoch S.J. Counterfactual reasoning and accuracy in predicting personal events. Journal of Experimental Psychology: Learning, Memory, and Cognition 1985, 11(4):719-731.
78. Jørgensen M. Top-down and bottom-up expert estimation of software development effort. Information and Software Technology 2004, 46(1):3-16.
79. Kabat G.C. Hyping health risks 2008, Columbia University Press, New York.
80. Keilman N. European demographic forecasts have not become more accurate over the past 25years. Population and Development Review 2008, 34(1):137-153.
81. Keogh E., Kasetty S. On the need for time series data mining benchmarks: A survey and empirical demonstration. Data Mining and Knowledge Discovery 2003, 7(4):349-371.
82. Kim M., Hunter J.E. Relationships among attitudes, behavioral intentions, and behavior: A meta-analysis of past research, part 2. Communication Research 1993, 20(3):331-364.
83. Kinney W.R. Predicting earnings: Entity versus subentity data. Journal of Accounting Research 1971, 9:127-136.
84. Koriat A., Lichtenstein S., Fischhoff B. Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory 1980, 6(2):107-118.
85. Kort F. Predicting Supreme Court decisions mathematically: A quantitative analysis of "right to counsel" cases. The Americam Political Science Review 1957, 51:1-12.
86. Larrick R.P., Soll J.B. Intuitions about combining opinions: Misappreciation of the averaging principle. Management Science 2006, 52(1):111-127.
87. Legerstee R., Franses P.H. Do experts' SKU forecasts improve after feedback?. Journal of Forecasting 2014, 33:66-79.
88. MacGregor D. Decomposition for judgmental forecasting and estimation. Principles of forecasting: A handbook for researchers and practitioners 2001, 107-123. Springer, New York. J.S. Armstrong (Ed.).
89. Makridakis S., Andersen A., Carbone R., Fildes R., Hibon M., Lewandowski R., et al. The accuracy of extrapolation (time series) methods: Results of a forecasting competition. Journal of Forecasting 1982, 1(2):111-153.
90. Makridakis S., Hibon M. The M-3 competition: Results, conclusions and implications. International Journal of Forecasting 2000, 16:451-476.
91. McCarthy T.M., Davis D.F., Golicic S.L., Mentzer J.T. The evolutions of sales forecasting management: A 20-year longitudinal study of forecasting practices. Journal of Forecasting 2006, 25:303-324.
92. McNown R., Rogers A., Little J. Simplicity and complexity in extrapolation population forecasting models. Mathematical Population Studies 1995, 5(3):235-257.
93. Meehl P.E. Clinical versus statistical prediction 1954, University of Minnesota Press, Minneapolis.
94. Miller T., Liberatore M. Seasonal exponential smoothing with damped trends: An application for production planning. International Journal of Forecasting 1993, 9:509-515.
95. Miller D.M., Williams D. Damping seasonal factors: Shrinkage estimators for the X-12-ARIMA program. International Journal of Forecasting 2004, 20(4):529-549. (Published with, commentary, pp. 551-568).
96. Mollick E. Establishing Moore's Law. IEEE Annals of the History of Computing 2006, 28:62-75.
97. Morwitz V.G. Methods for forecasting from intentions data. Principles of forecasting 2001, Kluwer Academic Publishers, Boston. J.S. Armstrong (Ed.).
98. Murphy A.H., Winkler R.L. Probability forecasting in meterology. Journal of the American Statistical Association 1984, 79:489-500.
99. Namboodiri N.K., Lalu N.M. The average of several simple regression estimates as an alternative to the multiple regression estimate in postcensal and intercensal population estimation: A case study. Rural Sociology 1971, 36:187-194.
100. Nelson C.R. The Prediction Performance of the FRB-MIT-PENN Model of the U.S. Economy. The American Economic Review 1972, 62(5):902-917. http://www.jstor.org/stable/1815208.
101. Nikolopoulos K., Litsa A., Petropoulos F., Bougioukosa V., Khammash M. Relative performance of methods for forecasting special events. Journal of Business Research 2015, 68(8):1785-1791. (in this issue).
102. Prasad V., Vandross A., Toomey C., Cheung M., Rho J., Quinn S., et al. A decade of reversal: An analysis of 146 contradicted medical practices 2013, 790-798. MayoClinicProceedings.org, (Available at http://www.senyt.dk/bilag/artiklenframayoclinicproce.pdf).
103. Randall D.A., Wood R.A., Bony S., Colman R., Fichefet T., Fyfe J., et al. Climate models and their evaluation. Climate Change 2007: The physical science basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change 2007, 589-662. Cambridge University Press, Cambridge, UK and New York, USA. S. Solomon, D. Qin, M. Manning, Z. Chen, M. Marquis, K.B. Averyt, M. Tignor, H.L. Miller (Eds.).
104. Rowe G., Wright G. Expert opinions in forecasting: The role of the Delphi technique. Principles of forecasting: A handbook for researchers and practitioners 2001, 125-144. Springer, New York. J.S. Armstrong (Ed.).
105. Runkle D.E. Revisionist history: How data revisions distort economic policy research. Federal Reserve Bank of Minneapolis Quarterly Review 1998, 22(4):3-12.
106. Ryan P., Sessions J. Sessions, Ryan call for halt on taxpayer funding for risky high-speed rail project 2013, U.S. Senate Budget Committee, (Available at http://www.budget.senate.gov/republican/public/index.cfm/2013/3/sessions-ryan-call-for-halt-on-taxpayer-funding-for-risky-high-speed-rail-project).
107. Sanders N.R. Big data driven supply chain management: A framework for implementing analytics and turning analytics into intelligence 2014, Pearson Education, New Jersey.
108. Sanders N.R., Manrodt K.B. Forecasting practices in US corporations: Survey results. Interfaces 1994, 24(2):92-100.
109. Sanders N.R., Ritzman L.P. Judgmental adjustment of statistical forecasts. Principles of forecasting: A handbook for researchers and practitioners 2001, 405-416. Springer, New York. J.S. Armstrong (Ed.).
110. Schnaars S.P. A comparison of extrapolation models on yearly sales forecasts. International Journal of Forecasting 1986, 2:71-85.
111. Schnaars S.P. Megamistakes: Forecasting and the myth of rapid technological change 1989, The Free Press, New York.
112. Shu L.L., Mazar N., Gino F., Ariely D., Bazerman M.H. Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end. Proceedings of the National Academy of Sciences 2012, 109(38):15197-15200.
113. Soyer E., Hogarth R.M. Illusion of predictability: How regression statistics mislead experts. International Journal of Forecasting 2012, 28(3):695-711.
114. Sparks J. The works of Benjamin Franklin 1844, Vol. 8. Charles Tappan Publisher, Boston.
115. Tessier T.H., Armstrong J.S. Decomposition of time-series by level and change. Journal of Business Research 2015, 68(8):1755-1758. (in this issue).
116. Tetlock P.C. Expert political judgment 2005, Princeton University Press, Princeton.
117. Vokurka R.J., Flores B.E., Pearce S.L. Automatic feature identification and graphical support in rule-based forecasting: A comparison. International Journal of Forecasting 1996, 12:495-512.
118. Weimann G. The obsession to forecast: Pre-election polls in the Israeli press. Public Opinion Quarterly 1990, 54:396-408.
119. Winston C. Government failure versus market failure: Microeconomics policy research and government performance 2006, AEI-Brookings Joint Center for Regulatory Studies, Washington, D.C., (Available at http://www.brookings.edu/press/Books/2006/governmentfailurevsmarketfailure.aspx).
120. Withycombe R. Forecasting with combined seasonal indices. International Journal of Forecasting 1989, 5:547-552.
121. Wright M., MacRae M. Bias and variability in purchase intention scales. Journal of the Academy of Marketing Science 2007, 35(4):617-624.
122. Wright M., Stern P. Forecasting new product trial with analogous series. Journal of Business Research 2015, 68(8):1732-1738. (in this issue).
123. Zarnowitz V. An appraisal of short-term economic forecasts. NBER occasional paper 104 1967, National Bureau of Economic Research, New York.
124. Zellner A. Keep it sophisticatedly simple. Simplicity, inference and modelling: Keeping it sophisticatedly simple 2001, 242-262. Cambridge University Press, Cambridge. A. Zellner, H.A. Keuzenkamp, M. McAleer (Eds.).
125. Ziliak S.T., McCloskey D.N. Size matters: The standard error of regressions in the American Economic Review. The Journal of Socio-Economics 2004, 33:527-546.