Protecting against evaluation overfitting in empirical reinforcement learning

IEEE SSCI 2011: Symposium Series on Computational Intelligence - ADPRL 2011: 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning

Volumn , Issue , 2011, Pages 120-127

  Whiteson, Shimon ^a   Tanner, Brian ^b   Taylor, Matthew E ^c   Stone, Peter ^d  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b UNIVERSITY OF ALBERTA   (Canada)

^c LAFAYETTE COLLEGE   (United States)

^d University of Texas at Austin   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EMPIRICAL EVALUATIONS; OVERFITTING; PROOF OF CONCEPT; TEST EVALUATION; TILE CODING;

ARTIFICIAL INTELLIGENCE; DYNAMIC PROGRAMMING;

REINFORCEMENT LEARNING;

EID: 79958854955     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ADPRL.2011.5967363     Document Type: Conference Paper

References (34)

1. P. Langley, "Machine learning as an experimental science," Machine Learning, vol. 3, no. 1, pp. 5-8, 1988.
2. P. Cohen and A. Howe, "How evaluation guides AI research," AI Magazine, vol. 9, no. 4, pp. 35-43, 1988.
3. H. Simon, "Artificial intelligence: an empirical science," Artificial Intelligence, vol. 77, no. 1, pp. 95-127, 1995.
4. A. Asuncion and D. Newman, "UCI machine learning repository," 2007. [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
5. R. Sutton, "Generalization in reinforcement learning: Successful examples using sparse coarse coding," in Advances in Neural Information Processing Systems 8, 1996, pp. 1038-1044.
6. C. Drummond and N. Japkowicz, "Warning: Statistical benchmarking is addictive. Kicking the habit in machine learning," Journal of Experimental and Theoretical Artificial Intelligence, 2009.
7. E. Falkenauer, "On method overfitting," Journal of Heuristics, vol. 4, pp. 281-287, 1998. (Pubitemid 128513805)
8. J. Ponce, T. Berg, M. Everingham, D. Forsyth, M. Herbert, S. Lazebnik, M. Marszalek, C. Schmid, B. Russel, A. Torralba, C. Williams, J. Zhang, and A. Zisserman, Dataset Issues in Object Recognition. Springer, 2007.
9. J. Langford, "Clever methods of overfitting," 2005. [Online]. Available: http://hunch.net/?p=22
10. F. Provost and T. Fawcett, "Robust classification for imprecise environments," Machine Learning, vol. 42, no. 3, pp. 203-231, 2001. (Pubitemid 32188799)
11. D. H. Wolpert, "The lack of a priori distinctions between learning algorithms," Neural Computation, vol. 8, no. 7, pp. 1341-1390, 1996. (Pubitemid 126449973)
12. R. E. Bellman, "A Markov decision process," Journal of Mathematical Mechanics, vol. 6, pp. 679-684, 1957.
13. R. Koppejan and S. Whiteson, "Neuroevolutionary reinforcement learning for generalized helicopter control," in GECCO 2009: Proceedings of the Genetic and Evolutionary Computation Conference, 2009, pp. 145-152.
14. S. Whiteson, B. Tanner, and A. White, "The reinforcement learning competitions," AI Magazine, vol. 31, no. 2, pp. 81-94, 2010.
15. A. Wilson, A. Fern, S. Ray, and P. Tadepalli, "Multi-task reinforcement learning: a hierarchical Bayesian approach," in Proceedings of the 24th International Conference on Machine Learning, 2007, pp. 1015-1022.
16. P. Poupart, N. Vlassis, J. Hoey, and K. Regan, "An analytic solution to discrete Bayesian reinforcement learning," in Proceedings of the Twenty- Third International Conference on Machine Learning, 2006.
17. S. Thrun and L. Pratt, Eds., Learning to learn. Norwell, MA, USA: Kluwer, 1998.
18. J. Baxter, "A model of inductive bias learning," J. of AI Research, vol. 12, pp. 149-198, 2000.
19. F. S. Roberts, "Limitations on conclusions using scales of measurement," in Operations Research and The Public Sector, 1994, vol. 6, ch. 18, pp. 621-671.
20. G. Webb, "Multiboosting: A technique for combining boosting and wagging," Machine learning, vol. 40, no. 2, pp. 159-196, 2000.
21. J. Demšar, "Statistical comparisons of classifiers over multiple data sets," The Journal of Machine Learning Research, vol. 7, p. 30, 2006.
22. S. L. Salzberg, "On comparing classifiers: Pitfalls to avoid and a recommended approach," Data Mining and Knowledge Discovery, vol. 1, pp. 317-327, 1997.
23. W. J. Dixon and A. M. Mood, "The statistical sign test," Journal of the American Statistical Association, vol. 41, no. 236, pp. 557-566, 1946.
24. D. Saari and V. Merlin, "The Copeland method," Economic Theory, vol. 8, no. 1, pp. 51-76, 1996.
25. Y. Koren, "The BellKor solution to the Netflix grand prize," August 2009. [Online]. Available: http://www.netflixprize.com/assets/ GrandPrize2009-BPC-BellKor.pdf
26. J. N. Hooker, "Testing heuristics: We have it all wrong," Journal of Heuristics, vol. 1, pp. 33-42, 1995.
27. T. Lane and W. Smart, "Why (PO)MDPs lose for spatial tasks and what to do about it," in Proceedings of the ICML 2005 Workshop on Rich Representations for Reinforcement Learning, 2005.
28. C. R. Gallistel, "The replacement of general-purpose learning models with adaptively specialized learning modules," in The Cognitive Neurosciences. Cambridge, MA: MIT Press, 2000, pp. 1179-1191.
29. G. Konidaris and A. Barto, "Autonomous shaping: Knowledge transfer in reinforcement learning," in Proceedings of the 23rd International Conference on Machine learning. ACM, 2006, pp. 489-496.
30. M. Snel and S. Whiteson, "Multi-task evolutionary shaping without prespecified representations," in GECCO 2010: Proceedings of the Genetic and Evolutionary Computation Conference, July 2010, pp. 1031-1038.
31. M. E. Taylor and P. Stone, "Transfer learning for reinforcement learning domains: A survey," JMLR, vol. 10, no. 1, pp. 1633-1685, 2009.
32. A. Nouri, M. L. Littman, L. Li, R. Parr, C. Painter-Wakefield, and G. Taylor, "A novel benchmark methodology and data repository for real-life reinforcement learning," 2009, poster at the Multidisciplinary Symposium on Reinforcement Learning.
33. B. Tanner and A. White, "RL-Glue : Language-independent software for reinforcement-learning experiments," Journal of Machine Learning Research, vol. 10, pp. 2133-2136, September 2009.
34. D. Cohn, L. Atlas, and R. Ladner, "Improving generalization with active learning," Machine Learning, vol. 15, no. 2, pp. 201-221, 1994.