Bayes Point Machines

Journal of Machine Learning Research

Volumn 1, Issue 4, 2001, Pages 245-279

  Herbrich, Ralf ^a   Graepel, Thore ^b   Campbell, Colin ^c  

^a MICROSOFT RESEARCH   (United Kingdom)

^b TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

^c UNIVERSITY OF BRISTOL   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0000631731     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (58)

1. J. Aitchison. Bayesian tolerance regions (with discussion). Journal of the Royal Statistical Society Series B, 26:161-175, 1964.
2. C. M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995.
3. B. E. BÖser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers. In D. Haussler, editor, Proceedings of the Annual Conference on Computational Learning Theory, pages 144-152, Pittsburgh, PA, July 1992. ACM Press.
4. W. Buntine. A Theory of Learning Classification Rules. PhD thesis, University of Technology, Sydney, Autralia, 1992.
5. C. Cortes. Prediction of Generalization Ability in Learning Machines. PhD thesis, Department of Computer Science, University of Rochester, 1995.
6. R. Cox. Probability, frequency, and reasonable expectations. American Journal of Physics, 14:1-13, 1946.
7. N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, Cambridge, UK, 2000.
8. D. DeCoste and B. Schölkopf. Training invariant support vector machines. Machine Learning, 2002. Accepted for publication. Also: Technical Report JPL-MLTR-00-1, Jet Propulsion Laboratory, Pasadena, CA, 2000.
9. T. G. Dietterich. Approximate statistical test for comparing supervised classification learning algorithms. Neural Computation, 10(7):1895-1924, 1998.
10. W. Feller. An Introduction To Probability Theory and Its Application, volume 2. John Wiley and Sons, New York, 1966.
11. T. Graepel and R. Herbrich. The kernel Gibbs sampler. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 514-520, Cambridge, MA, 2001. MIT Press.
12. T. Graepel, R. Herbrich, and K. Obermayer. Bayesian Transduction. In S. A. Solla, T. K. Leen, and K.-R. Müller, editors, Advances in Neural Information Processing Systems 12, pages 456-462, Cambridge, MA, 2000. MIT Press.
13. D. Haussler. Convolutional kernels on discrete structures. Technical Report UCSC-CRL-99-10, Computer Science Department, University of California at Santa Cruz, 1999.
14. R. Herbrich. Learning Linear Classifiers: Theory and Algorithms. MIT Press, 2001. in press.
15. R. Herbrich and T. Graepel. A PAC-Bayesian margin bound for linear classifiers: Why SVMs work. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 224-230, Cambridge, MA, 2001. MIT Press.
16. R. Herbrich, T. Graepel, and C. Campbell. Bayes point machines: Estimating the Bayes point in kernel space. In Proceedings of IJCAI Workshop Support Vector Machines, pages 23-27, 1999a.
17. R. Herbrich, T. Graepel, and C. Campbell. Bayesian learning in reproducing kernel Hilbert spaces. Technical report, Technical University of Berlin, 1999b. TR 99-11.
18. R. Herbrich, T. Graepel, and C. Campbell. Robust Bayes point machines. In Proceedings of ESANN 2000, pages 49-54, 2000a.
19. R. Herbrich, T. Graepel, and J. Shawe-Taylor. Sparsity vs. large margins for linear classifiers. In Proceedings of the Annual Conference on Computational Learning Theory, pages 304-308, 2000b.
20. T. Joachims. Text categorization with support vector machines: Learning with many relevant features. In Proceedings of the European Conference on Machine Learning, pages 137-142, Berlin, 1998. Springer.
21. T. Joachims. Making large-scale SVM learning practical. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 169-184, Cambridge, MA, 1999. MIT Press.
22. N. Littlestone and M. Warmuth. Relating data compression and learnability. Technical report, University of California Santa Cruz, 1986.
23. D. J. C. MacKay. Bayesian Methods for Adaptive Models. PhD thesis, Computation and Neural Systems, California Institute of Technology, Pasadena, CA, 1991.
24. D. A. McAllester. Some PAC Bayesian theorems. In Proceedings of the Annual Conference on Computational Learning Theory, pages 230-234, Madison, Wisconsin, 1998. ACM Press.
25. D. A. McAllester. PAC-Bayesian model averaging. In Proceedings of the Annual Conference on Computational Learning Theory, pages 164-170, Santa Cruz, USA, 1999.
26. J. Mercer. Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society, London, A 209:415-446, 1909.
27. C. A. Micchelli. Algebraic aspects of interpolation. Proceedings of Symposia in Applied Mathematics, 36:81-102, 1986.
28. T. Minka. Expectation Propagation for approximative Bayesian inference. PhD thesis, MIT Media Labs, Cambridge, USA, 2001.
29. T. M. Mitchell. Version spaces: a candidate elimination approach to rule learning. In Proceedings of the International Joint Conference on Neural Networks, pages 305-310, Cambridge, Massachusetts, 1977. IJCAII.
30. T. M. Mitchell. Generalization as search. Artificial Intelligence, 18(2):202-226, 1982.
31. R. Neal. Bayesian Learning in Neural Networks. Springer, 1996
32. . R. M. Neal. Markov chain Monte Carlo method based on 'slicing' the density function. Technical report, Department of Statistics, University of Toronto, 1997. TR-9722.
33. A. B. J. Novikoff. On convergence proofs on perceptrons. In Proceedings of the Symposium on the Mathematical Theory of Automata, volume 12, pages 615-622. Polytechnic Institute of Brooklyn, 1962.
34. M. Opper and D. Haussler. Generalization performance of Bayes optimal classification algorithms for learning a perceptron. Physical Review Letters, 66:2677, 1991.
35. M. Opper and W. Kinzel. Statistical Mechanics of Generalisation, page 151. Springer, 1995.
36. M. Opper, W. Kinzel, J. Kleinz, and R. Nehl. On the ability of the optimal perceptron to generalize. Journal of Physics A, 23:581-586, 1990.
37. M. Opper and O. Winther. Gaussian processes for classification: Mean field algorithms. Neural Computation, 12(11) =2655-2684, 2000.
38. E. E. Osuna, R. Freund, and F. Girosi. Support vector machines: Training and applications. Technical report, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1997. AI Memo No. 1602.
39. E. J. G. Pitman. The estimation of the location and scale parameters of a continuous population of any given form. Biometrika, 30:391-421, 1939.
40. J. Platt. Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 185-208, Cambridge, MA, 1999. MIT Press.
41. J. Platt. Probabilities for SV machines. In A. J. Smola, P. L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 61-73, Cambridge, MA, 2000. MIT Press.
42. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for adaboost. Machine Learning, 42(3): 287-320, 2001.
43. P. Ruján. Playing billiards in version space. Neural Computation, 9:99-122, 1997.
44. P. Ruján and M. Marchand. Computing the Bayes kernel classifier. In A. J. Smola, P. L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 329-347, Cambridge, MA, 2000. MIT Press.
45. M. Rychetsky, J. Shawe-Taylor, and M. Glesner. Direct Bayes point machines. In Proceedings of the International Conference on Machine Learning, 2000.
46. B. Schölkopf. Support Vector Learning. R. Oldenbourg Verlag, München, 1997. Doktorarbeit, TU Berlin. Download: http://www.kernel-machines.org.
47. B. Schölkopf, R. Herbrich, and A. J. Smola. A generalized representer theorem. In Proceedings of the Annual Conference on Computational Learning Theory, 2001.
48. B. Schölkopf, S. Mika, C. Burges, P. Knirsch, K.-R. Müller, G. Rätsch, and A. Smola. Input space vs. feature space in kernel-based methods. IEEE Transactions on Neural Networks, 10(5):1000-1017, 1999.
49. J. Shawe-Taylor, P. L. Bartlett, R. C. Williamson, and M. Anthony. Structural risk minimization over data-dependent hierarchies. IEEE Transactions on Information Theory, 44(5):1926-1940, 1998.
50. J. Shawe-Taylor and N. Cristianini. Margin distribution and soft margin. In A. J. Smola, P. L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 349-358, Cambridge, MA, 2000. MIT Press.
51. A. J. Smola. Learning with Kernels. PhD thesis, Technische Universität Berlin, 1998. GMD Research Series No. 25.
52. P. Sollich. Probabilistic methods for support vector machines. In S. A. Solla, T. K. Leen, and K.-R. Müller, editors, Advances in Neural Information Processing Systems 12, pages 349-355, Cambridge, MA, 2000. MIT Press.
53. V. Vapnik. The Nature of Statistical Learning Theory. Springer, New York, 1995. ISBN 0-387-94559-8.
54. V. Vapnik. Statistical Learning Theory. John Wiley and Sons, New York, 1998.
55. G. Wahba. Spline Models for Observational Data, volume 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, 1990.
56. T. Watkin. Optimal learning with a neural network. Europhysics Letters, 21:871, 1993.
57. C. K. I. Williams. Prediction with Gaussian processes: From linear regression to linear prediction and beyond. In M. Jordan, editor, Learning and Inference in Graphical Models, pages 599-621. MIT Press, 1999.
58. C. K. I. Williams and D. Barber. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI, 20(12):1342-1351,1998.