A framework for evaluating epidemic forecasts

BMC Infectious Diseases

Volumn 17, Issue 1, 2017, Pages

  Tabataba, Farzaneh Sadat ^a,b   Chakraborty, Prithwish ^a   Ramakrishnan, Naren ^a,b   Venkatramanan, Srinivasan ^b   Chen, Jiangzhuo ^b   Lewis, Bryan ^b   Marathe, Madhav ^a,b  

^a State University   (United States)

^b VIRGINIA BIOINFORMATICS INSTITUTE   (United States)

Author keywords

Epidemic forecasting; Epidemic Features; Error Measure; Performance evaluation; Ranking

Indexed keywords

ALGORITHM; ARTICLE; CONCEPTUAL FRAMEWORK; DIAGNOSTIC VALUE; EPIDEMIC; EPIDEMIOLOGICAL DATA; EPIDEMIOLOGICAL MONITORING; FORECASTING; INFLUENZA; MEASUREMENT ACCURACY; MEASUREMENT ERROR; PREDICTION; STANDARD; UNITED STATES; AGE; HUMAN; INFLUENZA, HUMAN; MARKOV CHAIN; PANDEMIC; THEORETICAL MODEL;

AGE FACTORS; ALGORITHMS; DISEASE OUTBREAKS; FORECASTING; HUMANS; INFLUENZA, HUMAN; MODELS, THEORETICAL; PANDEMICS; STOCHASTIC PROCESSES; UNITED STATES;

EID: 85019258842     PISSN: None     EISSN: 14712334     Source Type: Journal    
DOI: 10.1186/s12879-017-2365-1     Document Type: Article

References (44)

1. Paul MJ, Dredze M, Broniatowski D. Twitter improves influenza forecasting. PLoS Currents. 2014; 6. doi: 10.1371/currents.outbreaks.90b9ed0f59bae4ccaa683a39865d9117.
2. Scott SL, Varian HR. Bayesian Variable Selection for Nowcasting Economic Time Series. In: Economic Analysis of the Digital Economy, NBER Chapters. National Bureau of Economic Research, Inc. and University of Chicago Press: 2015. p. 119-35. https://ideas.repec.org/h/nbr/nberch/12995.html.
3. Stock J. Forecasting Economic Time Series In: Baltagi B, editor. Companion in Theoretical Econometrics. Malden: Basil Blackwell: 2001. p. 562-84.
4. Radzuan NFM, Othman Z, Bakar AA. Uncertain Time Series in Weather Prediction. Procedia Technol. 2013; 11(Iceei):557-64. doi: 10.1016/j.protcy.2013.12.228.
5. Voyant C, Paoli C, Muselli M, Nivet ML. Multi-horizon solar radiation forecasting for Mediterranean locations using time series models. Renew Sust Energ Rev. 2013; 28:44-52. doi: 10.1016/j.rser.2013.07.058.
6. Kumar N, Jha GK. A time series ann approach for weather forecasting. Int J Control Theory Comput Model (IJCTCM). 2013; 3(1):19-25.
7. Chretien JP, George D, Shaman J, Chitale RA, McKenzie FE. Influenza forecasting in human populations: a scoping review. PLoS ONE. 2014; 9(4):94130. doi: 10.1371/journal.pone.0094130.
8. Announcement of Requirements and Registration for the Predict the Influenza Season Challenge. http://www.gpo.gov/fdsys/pkg/FR-2013-11-25/pdf/2013-28198.pdf. Accessed 7 July 2016.
9. RAPIDD Ebola Challenge: Comparison of Disease Forecasting Models. http://www.ebola-challenge.org/. Accessed 7 July 2016.
10. Forecasting the Next Dengue Outbreak. https://www.ncdc.noaa.gov/news/forecasting-next-dengue-outbreak. Accessed 7 July 2016.
11. Dengue Forecasting Project. http://dengueforecasting.noaa.gov/docs/project_description.pdf. Accessed 7 July 2016.
12. DARPA CHIKV Challenge to Address Threat of Chikungunya. http://globalbiodefense.com/2014/08/18/darpa-chikv-challenge-chikungunya/. Accessed 7 July 2016.
13. Makridakis S, Chatfield C, Hibon M, Lawrence M, Mills T, Ord K, Simmons LF. The M2-competition: A real-time judgmentally based forecasting study. Int J Forecast. 1993; 9(1):5-22. doi:10.1016/0169- 2070(93)90044-N.
14. Makridakis S. The M3-Competition: results, conclusions and implications. Int J Forecast. 2000; 16:451-76.
15. Armstrong JS. Evaluating forecasting methods. Boston: Springer; 2001, pp. 443-72. doi: 10.1007/978-0-306-47630-3_2. http://dx.doi.org/10.1007/978-0-306-47630-3_20.
16. Nsoesie EO, Brownstein JS, Ramakrishnan N, Marathe MV. A systematic review of studies on forecasting the dynamics of influenza outbreaks,. Influenza Other Respir Viruses. 2014; 8:309-16. doi: 10.1111/irv.12226.
17. Viboud C, Boëlle PY, Carrat F, Valleron AJ, Flahault A. Prediction of the Spread of Influenza Epidemics by the Method of Analogues. Am J Epidemiol. 2003; 158(10):996-1006. doi: 10.1093/aje/kwg239.
18. Aguirre A, Gonzalez E. The feasibility of forecasting influenza epidemics in Cuba. Memorias do Instituto Oswaldo Cruz. 1992; 87(3):429-32. http://www.ncbi.nlm.nih.gov/pubmed/1343651.
19. Jiang X, Wallstrom G, Cooper GF, Wagner MM. Bayesian prediction of an epidemic curve. J Biomed Inform. 2009; 42(1):90-9. doi:10.1016/j.jbi.2008.05.013.
20. Soebiyanto RP, Adimi F, Kiang RK. Modeling and predicting seasonal influenza transmission in warm regions using climatological parameters. PLoS ONE. 2010; 5(3):1-10. doi:10.1371/journal.pone.0009450.
21. Cha S-H. Comprehensive survey on distance/similarity measures between probability density functions. Intl J Math Models Methods Appl Sci. 2007; 1(4):300-7. doi:10.1007/s00167-009-0884-z.
22. Longini IM, Fine PE, Thacker SB. Predicting the global spread of new infectious agents. Am J Epidemiol. 1986; 123(3):383-91.
23. Chao DL, Matrajt L, Basta NE, Sugimoto JD, Dean B, Bagwell DA, Oiulfstad B, Halloran ME, Longini IM. Planning for the control of pandemic influenza A (H1N1) in Los Angeles County and the United States. Am J Epidemiol. 2011; 173(10):1121-30. doi:10.1093/aje/kwq497.
24. Hall IM, Gani R, Hughes HE, Leach S. Real-time epidemic forecasting for pandemic influenza. Epidemiol Infection. 2007; 135:372-85. doi:10.1017/S0950268806007084.
25. Ong JBS, Chen MI-C, Cook AR, Lee HC, Lee VJ, Lin RTP, Tambyah PA, Goh LG. Real-time epidemic monitoring and forecasting of H1N1-2009 using influenza-like illness from general practice and family doctor clinics in Singapore. PloS ONE. 2010; 5(4):10036. doi:10.1371/journal.pone.0010036.
26. Tizzoni M, Bajardi P, Poletto C, Ramasco JJ, Balcan D, Gonçalves B, Perra N, Colizza V, Vespignani A. Real-time numerical forecast of global epidemic spreading: case study of 2009 A/H1N1pdm. BMC Med. 2012; 10:165. doi:10.1186/1741-7015-10-165.
27. Towers S, Feng Z. Pandemic H1N1 influenza: predicting the course of a pandemic and assessing the efficacy of the planned vaccination programme in the United States. Euro Surveillance: Bulletin Europeen Sur Les Maladies Transmissibles = European Communicable Disease Bull. 2009; 14(41):19358.
28. Shaman J, Karspeck A. Forecasting seasonal outbreaks of influenza. Proc Natl Acad Sci U S A. 2012; 109(3):20425-30. doi: 10.1073/pnas.1208772109.
29. Andersson E, Kühlmann-Berenzon S, Linde A, Schiöler L, Rubinova S, Frisén M. Predictions by early indicators of the time and height of the peaks of yearly influenza outbreaks in Sweden. Scand J Public Health. 2008; 36(5):475-82. doi: 10.1177/1403494808089566.
30. Hyder A, Buckeridge DL, Leung B. Predictive validation of an influenza spread model. PLoS One. 2013;8(6).doi: 10.1371/journal.pone.0065459.
31. Overview of Influenza Surveillance in the United States. http://www.cdc.gov/flu/pdf/weekly/overview.pdf. Accessed 07 July 2016.
32. HHS Region Map. http://www.hhs.gov/about/agencies/iea/regional-offices/index.html. Accessed 7 July 2016.
33. Lekone PE, Finkenstädt BF. Statistical inference in a stochastic epidemic SEIR model with control intervention: Ebola as a case study. Biometrics. 2006; 62(December):1170-7. doi: 10.1111/j.1541-0420.2006.00609.x.
34. Principles of Epidemiology in Public Health Practice, Third Edition An Introduction to Applied Epidemiology and Biostatistics. http://www.cdc.gov/ophss/csels/dsepd/SS1978/Lesson3/Section2.html. Accessed 7 July 2016.
35. 2009 H1N1 Early Outbreak and Disease Characteristics. http://www.cdc.gov/h1n1flu/surveillanceqa.htm%234. Accessed 7 July 2016.
36. Overview of Influenza Surveillance in the United States. http://www.cdc.gov/flu/weekly/overview.htm. Accessed 7 July 2016.
37. U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet). https://public.health.oregon.gov/DiseasesConditions/CommunicableDisease/DiseaseSurveillanceData/Influenza/Documents/ recruitment_cdc_system.pdf. Accessed 7 July 2016.
38. Armstrong BJS, Collopy F. Error Measures For Generalizing About Forecasting Methods: Empirical Comparisons By J. Scott Armstrong and Fred Collopy Reprinted with permission form. Int J Forecast. 1992; 8(1):69-80. doi: 10.1016/0169-2070(92)90008-W.
39. Deza MM, Deza E. Encyclopedia of distances: Springer; 2009, pp. 1-590. doi: 10.1007/978-3-642-00234-2.
40. Abou-Moustafa KT, Ferrie FP. A note on metric properties for some divergence measures: The Gaussian case. J Mach Learn Res. 2012; 25:1-15.
41. Pardo L. Statistical Inference Based on Divergence Measures, Vol. 170: Chapman and Hall/CRC; 2006, p. 497. doi: 10.1201/9781420034813.
42. ARIMA Models for Time Series Forecasting. https://people.duke.edu/%257Ernau/411arim.htm. Accessed 31 Dec 2016.
43. Shcherbakov MV, Brebels A, Shcherbakova NL, Tyukov AP, Janovsky TA, evich Kamaev VA. A survey of forecast error measures. World Appl Sci J. 2013; 24(24):171-6. doi:10.5829/idosi.wasj.2013.24.itmies.80032.
44. Syntetos AA, Boylan JE. On the variance of intermittent demand estimates. Int J Prod Econ. 2010; 128(2):546-55. doi:10.1016/j.ijpe.2010.07.005.