Empirical calibration of time series monitoring methods using receiver operating characteristic curves

International Journal of Forecasting

Volumn 25, Issue 3, 2009, Pages 484-497

  Cohen, Jacqueline ^a   Garman, Samuel ^a   Gorr, Wilpen ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Average run length statistic; Exponential smoothing; Outliers; ROC curve; Step jumps; Structural breaks; Time series monitoring

Indexed keywords

EID: 67649723813     PISSN: 01692070     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ijforecast.2008.11.007     Document Type: Article

References (37)

1. Anderson E.A., and Diaz J. Using process control chart techniques to analyze crime rates in Houston. Texas Journal of the Operational Research Society 47 (1996) 871-881
2. Baltimore CitiStat Reports and Maps (2008). Internet site accessed 2/2/2008, http://www.ci.baltimore.md.us/news/citistat/reports.html
3. Batty M. Monitoring an exponential smoothing forecasting system. Operational Research Quarterly 20 (1969) 319-325
4. Brown R.G. Statistical forecasting for inventory control (1959), McGraw-Hill, New York
5. Brown R.G. Smoothing, forecasting and prediction of discrete time series (1963), Prentice-Hall, Englewood Cliffs, NJ
6. Cohen, J., & Gorr, W. L. (2005). Development of crime forecasting and mapping systems for use by police. Grant 2001-IJ-CX-0018 final report. National Institute of Justice
7. DeNeef P., and Kent D.L. Using treatment-tradeoff preferences to select diagnostic strategies: Linking the ROC curve to threshold analysis. Medical Decision Making 13 (1993) 126-132
8. Farrington D.P., and Tarling R. Criminological prediction: An introduction. In: Farrington D.P., and Tarling R. (Eds). Prediction in criminology (1985), State University of New York Press, Albany
9. Fawcett, T. (2003). ROC graphs: Notes and practical considerations for researchers. Technical report HPL-2003-4. HP Laboratories. Available: http://www.hpl.hp.com/personal/Tom_Fawcett/papers/ROC101.pdf
10. Fiszman M., Chapman W.W., Aronsky D., Evans R.S., and Haug P.J. Automatic detection of acute bacterial pneumonia from chest x-ray reports. Journal of the American Medical Informatics Association 7 (2000) 593-604
11. Gardner E.S. Automatic monitoring of forecast errors. Journal of Forecasting 2 (1983) 1-21
12. Gardner E.S. CUSUM vs smoothed-error forecast monitoring schemes-some simulation results. Journal of the Operational Research Society 36 (1985) 43-47
13. Golder E.R., and Settle J.G. Monitoring schemes in short-term forecasting. Operational Research Quarterly 27 (1976) 489-501
14. Gorr W.L., Olligschlaeger A., and Thompson Y. Short-term forecasting of crime. International Journal of Forecasting 19 (2003) 579-594
15. Gorr W.L., and McKay S.A. Application of time series monitoring methods to detect time series pattern changes in crime mapping systems. In: Wang F. (Ed). Geographic information systems and crime analysis (2005), Idea Group Pub, Hershey, PA 171-182
16. Harrison P.J., and Davies O.L. The use of cumulative sum (CUSUM) techniques for the control of routine forecasts of product demand. Operations Research 12 (1964) 325-333
17. Hart S.D., Webster C.D., and Menzies R.J. A note on portraying the accuracy of violence predictions. Law and Human Behavior 17 (1993) 695-700
18. Kazui S., Naritomi H., Yamamoto H., Sawada T., and Yamaguchi T. Enlargement of spontaneous intracerebral hemorrhage. Stroke 27 (1996) 1783-1787
19. Kleinman K.P., and Abrams A.M. Assessing surveillance using sensitivity, specificity and timeliness. Statistical Methods in Medical Research 15 (2006) 445-464
20. Kupinski M.A., and Anastasio M.A. Multiobjective genetic optimization of diagnostic classifiers with implications for generating receiver operating characteristic curves. IEEE Transactions on Medical Imaging 18 (1999) 675-685
21. McClain J.O. Dominant time series monitoring methods. International Journal of Forecasting 4 (1988) 563-572
22. McKenzie E. Monitoring of exponentially weighted forecasts. Journal of the Operational Research Society 29 (1978) 449-458
23. McNeil B.J., and Hanley J.A. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Medical Decision Making 4 (1984) 137-150
24. Metz C.E. Basic principles of ROC analysis. Seminars in Nuclear Medicine 8 (1978) 283-298
25. Moise A., Clement B., Ducimetiere P., and Bourassa M.G. Comparison of receiver operating curves derived from the same population: A bootstrapping approach. Computers and Biomedical Research 18 (1985) 125-131
26. Monahan, J. (1981). The clinical prediction of violence. DHHS Publication No (ADM) 81-921. Rockville, MD: US Department of Health and Human Services
27. Mossman D. Resampling techniques in the analysis of non-binormal ROC data. Medical Decision Making 15 (1995) 358-366
28. New York City Police Department Crime Statistics (2008). Internet site accessed 2/2/2008, http://www.nyc.gov/html/nypd/html/pct/cspdf.html
29. Obuchowski N.A., and Lieber M.L. Confidence intervals for the receiver operating characteristic area in studies with small samples. Academic Radiology 5 (1998) 561-571
30. Otto R.K. Prediction of dangerous behavior: A review and analysis of second generation research. Forensics Reports 5 (1992) 103-133
31. Pepe M.S. Receiver operating characteristic methodology. Journal of the American Statistical Association 95 (2000) 308-311
32. Peterson W.W., Birdsall T.G., and Fox W.C. The theory of detectability. Transactions of the IRE Professional Group on Information Theory 4 (1954) 171-212
33. Rogerson P.A. Geographic surveillance of crime frequencies in small areas. In: Wang F. (Ed). Geographic information systems and crime analysis (2005), Idea Group Pub, Hershey, PA 153-170
34. Swets J.A. Indices of discrimination or diagnostic accuracy: Their ROCs and implied models. Psychological Bulletin 99 (1986) 100-117
35. Swets J.A. Measuring the accuracy of diagnostic systems. Science 240 (1988) 1285-1293
36. Swets J.A., Dawes R.M., and Monahan J. Better decisions through science. Scientific American 283 (2000) 82-87
37. Trigg D.W. Monitoring a forecasting system. Operational Research Quarterly 15 (1964) 271-274