The Lack of a Priori Distinctions between Learning Algorithms

Neural Computation

Volumn 8, Issue 7, 1996, Pages 1341-1390

  Wolpert, David H ^a  

^a SANTA FE INSTITUTE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0000459353     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/neco.1996.8.7.1341     Document Type: Article

References (33)

1. Anthony M., and Biggs N. 1992. Computational Learning Theory. Cambridge University Press, Cambridge.
2. Berger, J. 1985. Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, Berlin.
3. Berger, J., and Jeffreys, W. 1992. Ockham's razor and Bayesian analysis. Am. Sci. 80, 64-72.
4. Bernardo, J. Smith, A. 1994. Bayesian Theory. John Wiley, New York.
5. Blumer, A., et al. 1987. Occam's razor. Inform. Process. Lett. 24, 377-380.
6. Blumer, A., et al. 1989. Learnability and Vapnik-Chervonenkis dimension. J. ACM 36, 929-965.
7. Bridle, J. 1989. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In Neuro-Computing: Algorithms, Architectures, and Applications, F. Fougelman-Soulie and J. Herault eds., Springer-Verlag, Berlin.
8. Dietterich, T. 1990. Machine learning. Annu. Rev. Comput. Sci. 4, 255-306.
9. Drucker, H. et al. 1993. Improving performance in neural networks using a boosting algorithm. In Neural Information Processing Systems 5, S. Hanson et al. eds. Morgan Kaufmann, San Mateo, CA.
10. Duda, R., and Hart, P. 1973. Pattern Classification and Scene Analysis. John Wiley, New York.
11. Hughes, G. 1968. On the mean accuracy of statistical pattern recognizers. IEEE Transac. Inform. Theory IT-14, 55-63.
12. Kearns, M. J. 1992. Towards efficient agnostic learning. In Proceedings of the 5th Annual Workshop on Computational Learning Theory, ACM Press, New York.
13. Mitchell, T. 1982. Generalization as search. Artif. Intell. 18, 203-226.
14. Mitchell T., and Blum, A. 1994. Course Notes for Machine Learning, CMU.
15. Murphy, P., and Pazzani, M. 1994. Exploring the decision forest: An empirical investigation of Occam's razor in decision tree induction. J. Artif. Intell. Res. 1, 257-275.
16. Natarajan, B. 1991. Machine Learning: A Theoretical Approach. Morgan Kaufmann, San Mateo, CA.
17. Perrone, M. 1993. Improving regression estimation: Averaging methods for variance reduction with extensions to general convex measure optimization. Ph.D. thesis, Brown Univ., Physics Dept.
18. Plutowski, M. 1994. Cross-validation estimates integrated mean squared error. In Advances in Neural Information Processing Systems 6, Cowan et al. eds. Morgan Kaufmann, San Mateo, CA.
19. Schaffer, C. 1993. Overfitting avoidance as bias. Machine Learn. 10, 153-178.
20. Schaffer, C. 1994. A conservation law for generalization performance. In Machine Learning: Proceedings of the Eleventh International Conference, Cohen and Hirsh, eds. Morgan Kaufmann, San Mateo, CA.
21. Schapire, R. 1990. The strength of weak learnability. Machine Learn. 5, 197-227.
22. Vapnik, V. 1982. Estimation of Dependences Based on Empirical Data. Springer-Verlag, Berlin.
23. Vapnik, V., and Bottou, L. 1993. Local algorithms for pattern recognition and dependencies estimation. Neural Comp. 5, 893-909.
24. Waller, W., and Jain, A. 1978. On the monotonicity of the performance of Bayesian classifiers. IEEE Transact. Inform. Theory IT-24, 392-394.
25. Watanabe, S. 1985. Pattern Recognition: Human and Mechanical. John Wiley, New York.
26. Weiss, S. M., and Kulikowski, C. A. 1991. Computer Systems that Learn. Morgan Kaufmann, San Mateo, CA.
27. Wolpert, D. 1992. On the connection between in-sample testing and generalization error. Complex Syst. 6, 47-94.
28. Wolpert, D. 1993. On Overfitting Avoidance as Bias. Tech. Rep. SFI TR 93-03-016.
29. Wolpert, D. 1994a. The relationship between PAC, the Statistical Physics framework, the Bayesian framework, and the VC framework. In The Mathematics of Generalization, D. Wolpert ed., Addison-Wesley, Reading, MA.
30. Wolpert, D. 1994b. Filter likelihoods and exhaustive learning. In Computational Learning Theory and Natural Learning Systems: Volume II, S. Hanson et al. eds. MIT Press, Cambridge, MA.
31. Wolpert, D. 1995. On the Bayesian "Occam factors" argument for Occam's razor. In Computational Learning Theory and Natural Learning Systems: Volume III, T. Petsche et al. eds. MIT Press, Cambridge, MA.
32. Wolpert, D., and Macready, W. 1995. No Free Lunch Theorems for Search. Tech. Rep. SFI TR 95-02-010. Submitted.
33. Wolpert, M., Grossman, T., and Knill, E. 1995. Off-training-set error for the Gibbs and the Bayes optimal generalizers. Submitted.