Statistical and Machine Learning forecasting methods: Concerns and ways forward

PLoS ONE

Volumn 13, Issue 3, 2018, Pages

  Makridakis, Spyros ^a   Spiliotis, Evangelos ^b   Assimakopoulos, Vassilios ^b  

^a UNIVERSITY OF NICOSIA   (Cyprus)

^b NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPETITION; FORECASTING; MACHINE LEARNING; STRESS; TIME SERIES ANALYSIS; ARTIFICIAL NEURAL NETWORK; BAYES THEOREM; NORMAL DISTRIBUTION; STATISTICAL MODEL; SUPPORT VECTOR MACHINE;

BAYES THEOREM; FORECASTING; MACHINE LEARNING; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); NORMAL DISTRIBUTION; SUPPORT VECTOR MACHINE;

EID: 85044501351     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0194889     Document Type: Article

References (81)

1. Makridakis S. The forthcoming Artificial Intelligence (AI) revolution: Its impact on society and firms. Futures. 2017; 90:46-60. https://doi.org/10.1016/j.futures.2017.03.006
2. Zhang G, Eddy Patuwo B, Hu Y M. Forecasting with artificial neural networks:: The state of the art. International Journal of Forecasting. 1998; 14(1):35-62. https://doi.org/10.1016/S0169-2070(97)00044-7
3. Hamzacebi C, Akay D, Kutay F. Comparison of direct and iterative artificial neural network forecast approaches in multi-periodic time series forecasting. Expert Systems with Applications. 2009; 36(2, Part 2):3839-3844. https://doi.org/10.1016/j.eswa.2008.02.042.
4. Deng L. A tutorial survey of architectures, algorithms, and applications for deep learning-ERRATUM. APSIPA Transactions on Signal and Information Processing. 2014; 3. https://doi.org/10.1017/atsip. 2013.9
5. Zhang L, Suganthan PN. A survey of randomized algorithms for training neural networks. Information Sciences. 2016; 364-365(Supplement C):146-155. https://doi.org/10.1016/j.ins.2016.01.039.
6. Salaken SM, Khosravi A, Nguyen T, Nahavandi S. Extreme learning machine based transfer learning algorithms: A survey. Neurocomputing. 2017; 267:516-524. https://doi.org/10.1016/j.neucom.2017.06. 037.
7. Robinson C, Dilkina B, Hubbs J, Zhang W, Guhathakurta S, Brown MA, et al. Machine learning approaches for estimating commercial building energy consumption. Applied Energy. 2017; 208(Supplement C):889-904. https://doi.org/10.1016/j.apenergy.2017.09.060.
8. Voyant C, Notton G, Kalogirou S, Nivet ML, Paoli C, Motte F, et al. Machine learning methods for solar radiation forecasting: A review. Renewable Energy. 2017; 105(Supplement C):569-582. https://doi.org/10.1016/j.renene.2016.12.095.
9. Adya M, Collopy F. How effective are neural networks at forecasting and prediction? A review and evaluation. Journal of Forecasting. 1998; 17(56):481-495. https://doi.org/10.1002/(SICI)1099-131X (1998090)17:5/6%3C481::AID-FOR709%3E3.0.CO;2-Q
10. Chatfield C. Neural networks: Forecasting breakthrough or passing fad? International Journal of Forecasting. 1993; 9(1):1-3. http://dx.doi.org/10.1016/0169-2070(93)90043-M.
11. Sharda R, Patil RB. Connectionist approach to time series prediction: An empirical test. Journal of Intelligent Manufacturing. 1992; 3(1):317-323. https://doi.org/10.1007/BF01577272
12. Crone SF, Hibon M, Nikolopoulos K. Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction. International Journal of Forecasting. 2011; 27 (3):635-660. http://dx.doi.org/10.1016/j.ijforecast.2011.04.001.
13. Keogh E, Kasetty S. On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration. Data Mining and Knowledge Discovery. 2003; 7(4):349-371. https://doi.org/10.1023/A:1024988512476
14. Zhang GP. Avoiding Pitfalls in Neural Network Research. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). 2007; 37. https://doi.org/10.1109/TSMCC.2006. 876059
15. Ahmed NK, Atiya AF, Gayar NE, El-Shishiny H. An Empirical Comparison of Machine Learning Models for Time Series Forecasting. Econometric Reviews. 2010; 29(5-6):594-621. https://doi.org/10.1080/07474938.2010.481556
16. Makridakis S, Hibon M. The M3-Competition: results, conclusions and implications. International Journal of Forecasting. 2000; 16(4):451-476. https://doi.org/10.1016/S0169-2070(00)00057-1
17. Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press; 2016.
18. Wang J, Wang J. Forecasting stochastic neural network based on financial empirical mode decomposition. Neural Networks. 2017; 90:8-20. https://doi.org/10.1016/j.neunet.2017.03.004. PMID: 28364677
19. Zhao L. Neural Networks In Business Time Series Forecasting: Benefits And Problems. Review of Business Information Systems (RBIS). 2009; 13(3):57-62.
20. Assimakopoulos V, Nikolopoulos K. The theta model: a decomposition approach to forecasting. International Journal of Forecasting. 2000; 16(4):521-530. https://doi.org/10.1016/S0169-2070(00)00066-2
21. Ilies I, Jaeger H, Kosuchinas O, Rincon M, Vakänas V, Vaskevicius N. Stepping forward through echoes of the past: Forecasting with Echo State Networks, Technical Report: Jacobs University Bremen; 2007.
22. Gardner ES. Exponential smoothing: The state of the art-Part II. International Journal of Forecasting. 2006; 22(4):637-666. https://doi.org/10.1016/j.ijforecast.2006.03.005
23. Hamid SA, Habib A. Financial Forecasting with Neural Networks. Academy of Accounting and Financial Studies Journal. 2014; 18(4):37-55.
24. Qiu M, Song Y. Predicting the Direction of Stock Market Index Movement Using an Optimized Artificial Neural Network Model. PLOS ONE. 2016; 11(5):1-11. https://doi.org/10.1371/journal.pone.0155133
25. Kock AB, Terä svirta T. Forecasting Macroeconomic Variables Using Neural Network Models and Three Automated Model Selection Techniques. Econometric Reviews. 2016; 35(8-10):1753-1779. https://doi.org/10.1080/07474938.2015.1035163
26. Gabor MR, Dorgo LA. Neural Networks Versus Box-Jenkins Method for Turnover Forecasting: a Case Study on the Romanian Organisation. Transformations in Business and Economics. 2017; 16(1):187-211.
27. Marr B. The Top 10 AI And Machine Learning Use Cases Everyone Should Know About, Forbes; 2016. Available from: https://www.forbes.com/sites/bernardmarr/2016/09/30/what-are-the-top-10-use-casesfor- machine-learning-and-ai/#2c292bf894c9.
28. Zhang GP, Qi M. Neural network forecasting for seasonal and trend time series. European Journal of Operational Research. 2005; 160(2):501-514. https://doi.org/10.1016/j.ejor.2003.08.037
29. Alpaydin E. Machine Learning: Introduction to Machine Learning. The MIT Press; 2004.
30. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: Data mining, Inference, and Prediction, Second Edition. Springer New York; 2009.
31. Hyndman RJ, Koehler AB. Another look at measures of forecast accuracy. International Journal of Forecasting. 2006; 22(4):679-688. https://doi.org/10.1016/j.ijforecast.2006.03.001
32. Goodwin P, Lawton R. On the asymmetry of the symmetric MAPE. International Journal of Forecasting. 1999; 15(4):405-408. http://dx.doi.org/10.1016/S0169-2070(99)00007-2.
33. Hansen LK, Salamon P. Neural Network Ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1990; 12(10):993-1001. https://doi.org/10.1109/34.58871
34. Gardner ES. Exponential smoothing: the state of the art. Journal of Forecasting. 1985; 4(1):1-28. https://doi.org/10.1002/for.3980040103
35. Andrawis RR, Atiya AF, El-Shishiny H. Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition. International Journal of Forecasting. 2011; 27 (3):672-688. https://doi.org/10.1016/j.ijforecast.2010.09.005
36. Hyndman R, Khandakar Y. Automatic time series forecasting: the forecast package for R. Journal of Statistical Software. 2008; 26(3):1-22.
37. Hyndman RJ, Koehler AB, Snyder RD, Grose S. A state space framework for automatic forecasting using exponential smoothing methods. International Journal of Forecasting. 2002; 18(3):439-454. https://doi.org/10.1016/S0169-2070(01)00110-8
38. Connor JT, Martin RD, Atlas LE. Recurrent neural networks and robust time series prediction. IEEE transactions on neural networks. 1994; 5(2):240-54. https://doi.org/10.1109/72.279188 PMID: 18267794
39. Makridakis SG, Wheelwright SC, Hyndman RJ. Forecasting: Methods and applications ( Third Edition). New York: Wiley; 1998.
40. Jain LC, Medsker LR. Recurrent Neural Networks: Design and Applications. 1st ed. Boca Raton, FL, USA: CRC Press, Inc.; 1999.
41. Hochreiter S, Schmidhuber J. Long Short-Term Memory. Neural Computation. 1997; 9(8):1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735 PMID: 9377276
42. Lippmann RP. An Introduction to Computing with Neural Nets. IEEE ASSP Magazine. 1987; 4(2):4-22. https://doi.org/10.1109/MASSP.1987.1165576
43. Młller M. A scaled conjugate gradient algorithm for fast supervised learning. Neural networks. 1993; 6:525-533. https://doi.org/10.1016/S0893-6080(05)80056-5
44. Kourentzes N, Barrow DK, Crone SF. Neural network ensemble operators for time series forecasting. Expert Systems with Applications. 2014; 41(9):4235-4244. https://doi.org/10.1016/j.eswa.2013.12.011
45. Bergmeir C, Benitez JM. Neural Networks in R Using the Stuttgart Neural Network Simulator: RSNNS. Journal of Statistical Software. 2012; 46(7):1-26. https://doi.org/10.18637/jss.v046.i07
46. MacKay DJC. Bayesian Interpolation. Neural Computation. 1992; 4(3):415-447. https://doi.org/10.1162/neco.1992.4.3.415
47. Dan Foresee F, Hagan MT. Gauss-Newton approximation to bayesian learning. In: IEEE International Conference on Neural Networks-Conference Proceedings. vol. 3; 1997. p. 1930-1935.
48. Nguyen D, Widrow B. Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. IJCNN Int Joint Conf Neural Networks. 1990; 13:C21.
49. Rodriguez PP, Gianola D. brnn: Bayesian Regularization for Feed-Forward Neural Networks; 2016. Available from: https://cran.r-project.org/package=brnn.
50. Specht DF. A general regression neural network. IEEE Transactions on Neural Networks. 1991; 2 (6):568-576. https://doi.org/10.1109/72.97934 PMID: 18282872
51. Chasset P -O. GRNN: General regression neural network for the statistical software R; 2013. Available from: http://flow.chasset.net/r-grnn/.
52. Venables WN, Ripley BD. Modern Applied Statistics with S. 4th ed. New York: Springer; 2002. Available from: http://www.stats.ox.ac.uk/pub/MASS4.
53. Breiman L. Classification and Regression Trees. Boca Raton, FL: Chapman & Hall; 1993.
54. Therneau T, Atkinson B, Ripley B. rpart: Recursive Partitioning and Regression Trees; 2015. Available from: https://cran.r-project.org/package=rpart.
55. Schölkopf B, Smola AJ. Learning with kernel: Support Vector Machines, Regularization, Optimization and Beyond. The MIT Press; 2001. Available from: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.5140{&}rep=rep1{&}type=pdf.
56. Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F. e1071: Misc Functions of the Department of Statistics, Probability Theory Group, TU Wien; 2017. Available from: https://cran.r-project.org/package=e1071.
57. Rasmussen CE, Williams C. Gaussian Processes for Machine Learning. The MIT Press; 2006.
58. Karatzoglou A, Smola A, Hornik K, Zeileis A. kernlab-An S4 Package for Kernel Methods in R. Journal of Statistical Software. 2004; 11(9):1-20. https://doi.org/10.18637/jss.v011.i09
59. Elman JL. Finding structure in time. Cognitive Science. 1990; 14(2):179-211. https://doi.org/10.1207/s15516709cog1402-1
60. Chollet F, et al. Keras; 2015. https://github.com/keras-team/keras.
61. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems; 2015. Available from: http://tensorflow.org/.
62. Gorr WL. Research prospective on neural network forecasting. International Journal of Forecasting. 1994; 10(1):1-4. https://doi.org/10.1016/0169-2070(94)90044-2
63. Nelson M, Hill T, Remus B, O'Connor M. Can neural networks applied to time series forecasting learn seasonal patterns: an empirical investigation. System Sciences, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on. 1994; 3:649-655. https://doi.org/10.1109/HICSS.1994. 323316
64. Cottrell M, Girard B, Girard Y, Mangeas M, Muller C. Neural Modeling for Time Series: A Statistical Stepwise Method for Weight Elimination. IEEE Transactions on Neural Networks. 1995; 6(6):1355-1364. https://doi.org/10.1109/72.471372 PMID: 18263428
65. Nelson CR, Plosser CR. Trends and random walks in macroeconmic time series. Some evidence and implications. Journal of Monetary Economics. 1982; 10(2):139-162. https://doi.org/10.1016/0304-3932 (82)90012-5
66. Box GEP, Cox DR. An Analysis of Transformations. Journal of the Royal Statistical Society Series B (Methodological). 1964; 26(2):211-252.
67. Cox DR, Stuart A. Some Quick Sign Tests for Trend in Location and Dispersion. Biometrika. 1955; 42(1-2):80-95. https://doi.org/10.2307/2333424
68. Sakamoto Y, Ishiguro M, Kitagawa G. Akaike Information Criterion Statistics. D. Reidel Publishing Company; 1986.
69. Zaremba W, Sutskever I, Vinyals O. Recurrent Neural Network Regularization. CoRR. 2014.
70. Brown RG. Statistical forecasting for inventory control. New York: McGraw-Hill; 1959.
71. Box G, Jenkins G. Time Series Analysis: Forecasting and Control. San Francisco: Holden-Day; 1970.
72. Pearl J. Causality: Models, Reasoning, and Inference. New York: Cambridge University Press; 2000.
73. Bauwens L, Laurent S, Rombouts JVK. Multivariate GARCH models: a survey. Journal of Applied Econometrics. 2006; 21(1):79-109. https://doi.org/10.1002/jae.842
74. Laurent S, Rombouts JVK, Violante F. On the forecasting accuracy of multivariate GARCH models. Journal of Applied Econometrics. 2012; 27(6):934-955. https://doi.org/10.1002/jae.1248
75. Green KC, Armstrong JS. Simple versus complex forecasting: The evidence. Journal of Business Research. 2015; 68(8):1678-1685. http://dx.doi.org/10.1016/j.jbusres.2015.03.026.
76. Makridakis S, Hibon M. Accuracy of Forecasting: An Empirical Investigation. Journal of the Royal Statistical Society Series A (General). 1979; 142(2):97-145. https://doi.org/10.2307/2345077
77. Spiliotis E, Assimakopoulos V. Are M3 Competition data representative of the reality? Working paper. 2018;.
78. Kang Y, Hyndman RJ, Smith-Miles K. Visualising forecasting algorithm performance using time series instance spaces. International Journal of Forecasting. 2017; 33(2):345-358. https://doi.org/10.1016/j.ijforecast.2016.09.004
79. Petropoulos F, Makridakis S, Assimakopoulos V, Nikolopoulos K. Horses for Courses' in demand forecasting. European Journal of Operational Research. 2014; 237(1):152-163. https://doi.org/10.1016/j.ejor.2014.02.036
80. Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE. A survey of deep neural network architectures and their applications. Neurocomputing. 2017; 234(Supplement C):11-26. https://doi.org/10.1016/j.neucom.2016.12.038.
81. Zeng N, Zhang H, Song B, Liu W, Li Y, Dobaie AM. Facial expression recognition via learning deep sparse autoencoders. Neurocomputing. 2018; 273(Supplement C):643-649. https://doi.org/10.1016/j.neucom.2017.08.043.