Pseudo-mathematics and financial charlatanism: The effects of backtest overfitting on out-of-sample performance

Notices of the American Mathematical Society

Volumn 61, Issue 5, 2014, Pages 458-471

  Bailey, David H ^a   Borwein, Jonathan M ^b,c   de Prado, Marcos López ^a   Zhu, Qiji Jim ^d  

^a LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

^b UNIVERSITY OF NEWCASTLE   (Australia)

^c Australian Academy of Science   (Australia)

^d WESTERN MICHIGAN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84898468766     PISSN: 00029920     EISSN: 10889477     Source Type: Journal    
DOI: 10.1090/noti1105     Document Type: Article

References (29)

1. D. Bailey, J. Borwein, M. López de Prado and J. Zhu, The probability of backtest over-fitting, 2013, working paper, available at http://ssrn.com/abstract=2326253
2. D. Bailey and M. López de Prado, The Sharpe ratio efficient frontier, Journal of Risk 15(2) (2012), 3-44. Available at http://ssrn.com/abstract=1821643.
3. Drawdown-based stop-outs and the triple penance rule, 2013, working paper. Available at http://ssrn.com/abstract=2201302.
4. J. Doyle and C. Chen, The wandering weekday effect in major stock markets, Journal of Banking and Finance 33 (2009), 1388-1399.
5. P. Embrechts, C. Klueppelberg and T. Mikosch, Modelling Extremal Events, Springer-Verlag, New York, 2003.
6. R. Feynman, The Character of Physical Law, The MIT Press, 1964.
7. J. Hadar and W. Russell, Rules for ordering uncertain prospects, American Economic Review 59 (1969), 25-34.
8. L. Harris, Trading and Exchanges: Market Mi-crostructure for Practitioners, Oxford University Press, 2003.
9. C. Harvey and Y. Liu, Backtesting, working paper, SSRN, 2013, http://ssrn.com/abstract=2345489.
10. C. Harvey, Y. Liu, and H. Zhu, and The cross-section of expected returns, working paper, SSRN, 2013, http://ssrn.com/abstract=2249314.
11. D. Hawkins, The problem of overfitting, Journal of Chemical Information and Computer Science 44 (2004), 1-12.
12. Y. Hirsch, Don't Sell Stocks on Monday, Penguin Books, 1st edition, 1987.
13. J. Ioannidis, Why most published research findings are false, PLoS Medicine 2(8), August 2005.
14. H. Krumholz Give the data to the people, New York Times, February 2, 2014. Available at http://www.nytimes.com/2014/02/03/opinion/give-the-data-to-the-people.html.
15. D. Leinweber and K. Sisk, Event driven trading and the "new news", Journal of Portfolio Management 38(1) (2011), 110-124.
16. W. Leontief, Academic economics, Science Magazine (July 9, 1982), 104-107.
17. A. Lo, The statistics of Sharpe ratios, Financial Analysts Journal 58 4 (Jul/Aug, 2002). Available at http://ssrn.com/abstract=377260.
18. M. López de Prado and A. Peijan, Measuring the loss potential of hedge fund strategies, Journal of Alternative Investments 7(1), (2004), 7-31. Available at http://ssrn.com/abstract=641702.
19. M. López de Prado and M. Foreman, A mixture of Gaussians approach to mathematical portfolio oversight: The EF3M algorithm, working paper, RCC at Harvard University, 2012. Available at http://ssrn.com/abstract=1931734.
20. J. Mayer, K. Khairy, and J. Howard, Drawing an elephant with four complex parameters, American Journal of Physics 78(6) (2010).
21. C. Reed, The damn'd South Sea, Harvard Magazine (May-June 1999).
22. S. Resnick, Extreme Values, Regular Variation and Point Processes, Springer, 1987.
23. J. Romano and M. Wolf, Stepwise multiple testing as formalized data snooping, Econometrica 73(4) (2005), 1273-1282.
24. F. Schorfheide and K. Wolpin, On the use of holdout samples for model selection, American Economic Review 102(3) (2012), 477-481.
25. V. Stodden, D. Bailey, J. Borwein, R. LeVeque, W. Rider, and W. Stein, Setting the default to reproducible: Reproducibility in computational and experimental mathematics, February 2013. Available at http://www.davidhbailey.com/dhbpapers/icerm-report.pdf.
26. G. Van Belle and K. Kerr, Design and Analysis of Experiments in the Health Sciences, John Wiley & Sons.
27. H. White, A reality check for data snooping, Econometrica 68(5), 1097-1126.
28. S. Weiss and C. Kulikowski, Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems, 1st edition, Morgan Kaufman,1990.
29. A. Wiles, Financial greed threatens the good name of maths, The Times (04 Oct 2013). Available online at http://www.thetimes.co.uk/tto/education/article3886043.ece.