Characterizing the function space for bayesian kernel models

Journal of Machine Learning Research

Volumn 8, Issue , 2007, Pages 1769-1797

  Pillai, Natesh S ^a   Wu, Qiang ^a   Liang, Feng ^b   Mukherjee, Sayan ^a   Wolpert, Robert L ^a  

^a Duke University   (United States)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Dirichlet processes; Gaussian processes; Integral operator; L vy processes; Non parametric Bayesian methods; Reproducing kernel Hilbert space

Indexed keywords

GAUSSIAN DISTRIBUTION; IMAGE ANALYSIS; MATHEMATICAL MODELS; MATHEMATICAL OPERATORS; RANDOM PROCESSES;

BAYESIAN KERNEL MODELS; DIRICHLET PROCESSES; GAUSSIAN PROCESSES; INTEGRAL OPERATORS; LEVY PROCESSES; REPRODUCING KERNEL HILBERT SPACES (RKHS);

LEARNING SYSTEMS;

EID: 34547900221     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (60)

1. David Applebaum. Lévy Processes and Stochasitic Calculus. Cambridge Studies in Advanced Mathematics. Cambridge Univ. Press, Cambridge, UK, 2004.
2. Nachman Aronszajn. Theory of reproducing kernels. T. Am. Math. Soc., 686:337-404, 1950.
3. Mikhail Belkin and Partha Niyogi. Semi-supervised learning on Riemannian manifolds. Machine Learning, 56(1-3):209-239, 2004.
4. Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res., 7:2399-2434, 2006.
5. David M. Blei and Michael I. Jordan. Variational inference for Dirichlet process mixtures. Bayesian Anal., 1(1):121-143 (electronic), 2006.
6. Olivier Bousquet and André Elisseeff. Stability and generalization. J. Mach. Learn. Res., 2:499-526, 2002.
7. Sounak Chakraborty, Malay Ghosh, and Bani K. Mallick. Bayesian non-linear regression for large p small n problems. J. Am. Stat. Assoc., 2005. Under revision.
8. Corinna Cortes and Vladimir N. Vapnik. Support-vector networks. Machine Learning, 20(3):273-297, 1995.
9. Felipe Cucker and Stephen Smale. On the mathematical foundations of learning. Bulletin of the American Mathematical Society, 39:1-49, 2001.
10. Carl de Boor and Robert E. Lynch. On splines and their minimum properties. J. Math. Mech., 15:953-969, 1966.
11. Ronald A. DeVore, Ralph Howard, and Charles A. Micchelli. Optimal nonlinear approximation. Manuskripta Mathematika, 1989.
12. Persi Diaconis. Bayesian numerical analysis. In Shanti S. Gupta and James O. Berger, editors, Statistical decision theory and related topics, IV, volume 1, pages 163-175. Springer-Verlag, New York, NY, 1988.
13. Michael D. Escobar and Mike West. Bayesian density estimation and inference using mixtures. J. Am. Stat. Assoc., 90:577-588, 1995.
14. Theodoros Evgeniou, Massimiliano Pontil, and Tomaso Poggio. Regularization networks and support vector machines. Advances in Computational Mathematics, 13:1-50, 2000.
15. Thomas S. Ferguson. Prior distributions on spaces of probability measures. Ann. Stat., 2:615-629, 1974.
16. Thomas S. Ferguson. A Bayesian analysis of some nonparametric problems. Ann. Stat., 1:209-230, 1973.
17. Erik Ivar Fredholm. Sur une nouvelle méthode pour la résolution du problèm de Dirichlet. Euvres complètes: publiées sous les auspices de la Kungliga svenska vetensakademien par l'Institut Mittag-Leffler, pages 61-68, 1900.
18. Subhashis Ghosal and Anindya Roy. Posterior consistency of Gaussian process prior for nonparametric binary regression. Ann. Statist., 34(5):2413-2429, 2006.
19. Jacques Hadamard. Sur les problèmes aux dérivées partielles et leur signification physique. Princeton University Bulletin, pages 49-52, 1902.
20. Jaroslav Hájek. On linear statistical problems in stochastic processes. Czechoslovak Math. J., 12(87):404-444, 1962.
21. Jaroslv Hájek. On a property of normal distributions of any stochastic process. Select. Transl. Math. Statist, and Probability, 1:245-252, 1961.
22. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer-Verlag, 2001.
23. Lancelot F. James, Antonio Lijoa, and Igor Prünster. Conjugacy as a distinctive feature of the Dirichlet process. Scand. J. Stat., 33:105-120, 2005.
24. Iain Johnstone. Function estimation in Gaussian noise: sequence models. Draft of a monograph, 1998.
25. Gopinath Kallianpur. The role of reproducing kernel Hilbert spaces in the study of Gaussian processes. Advances in Probability and Related Topics, 2:49-83, 1970.
26. Hermann König. Eigenvalue distribution of compact operators, volume 16 of Operator Theory: Advances and Applications. Birkhäuser, Basel, CH, 1986.
27. George S. Kimeldorf and Grace Wahba. A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Statist., 41(2):495-502, 1971.
28. Feng Liang, Sayan Mukherjee, and Mike West. Understanding the use of unlabelled data in predictive modeling. Stat. Sci., 2006. To appear.
29. Feng Liang, Ming Liao, Kai Mao, Sayan Mukherjee, and Mike West. Non-parametric Bayesian kernel models. Discussion Paper 2007-10, Duke University ISDS, Durham, NC, 2007. URL {emwww.stat.duke.edu/research/papers/}.
30. Milan N. Lukić and Jay H. Beder. Stochasitic processes with sample paths in reproducing kernel Hilbert spaces. T. Am. Math. Soc., 353(10):3945-3969, 2001.
31. Stephen MacEachern and Peter Müller. Estimating mixture of Dirichlet process models. J. Comput. Graph. Stat., pages 223-238, 1998.
32. Vladimir G. Mazja. Sobolev Spaces. Springer-Verlag, New York, NY, 1985.
33. Peter Müller, Fernando Quintana, and Gary Rosner. A method for combining inference across related nonparametric Bayesian models. J. Am. Stat. Assoc., pages 735-749, 2004.
34. James Mercer. Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society, London A, 209:415-446, 1909.
35. Charles A. Micchelli and Grace Wahba. Design problems for optimal surface interpolation. In Zvi Ziegler, editor, Approximation Theory and Applications, pages 329-348, 1981.
36. Sayan Mukherjee, Pablo Tamayo, Simon Rogers, Ryan M. Rifkin, Anna Engle, Colin Campbell, Todd R. Golub, and Jill P. Mesirov. Estimating dataset size requirements for classifying DNA Microarray data. Journal of Computational Biology, 10:119-143, 2003.
37. Neal, R. M. Bayesian Learning for Neural Networks. Springer, New York, 1996. Lecture Notes in Statistics 118.
38. Emanuel Parzen. Probability density functionals and reproducing kernel Hilbert spaces. In Murray Rosenblatt, editor, Proceedings of the Symposium on Time Series Analysis, pages 155-169, New York, NY, 1963. John Wiley & Sons.
39. Tomaso Poggio and Federico Girosi. Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247:978-982, 1990.
40. Tomaso Poggio, Ryan M. Rifkin, Sayan Mukherjee, and Partha Niyogi. General conditions for predictivity in learning theory. Nature, 428:419-422, 2004.
41. Sridhar Ramaswamy, Pablo Tamayo, Ryan M. Rifkin, Sayan Mukherjee, Chen-Hsiang Yeang, Michael Angelo, Christine Ladd, Michael Reich, Eva Latulippe, Jill P. Mesirov, Tomaso Poggio, William Gerald, Massimo Loda, Eric S. Lander, and Todd R. Golub. Multiclass cancer diagnosis using tumor gene expression signatures. Proc Nat. Aca. Sci., 98:149-54, 2001.
42. Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA, 2006.
43. L. Chris G. Rogers and David Williams. Diffusions, Markov Processes, and Martingales, volume 2. John Wiley & Sons, New York, NY, 1987. ISBN 0-471-91482-7.
44. Bernhard Schölkopf and Alexander J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, 2001.
45. Isaac J. Schoenberg. Positive definite functions on spheres. Duke Mathematics Journal, 9:96-108, 1942.
46. John S. Shawe-Taylor and Nello Cristianini. Kernel Methods for Pattern Analysis. Cambridge Univ. Press, Cambridge, UK, 2004.
47. Peter Sollich. Bayesian methods for support vector machines: Evidence and predictive class probabilities. Machine Learning, 46(1-3):21-52, 2002.
48. Andrei Nikolaevich Tikhonov. Solution of incorrectly formulated problems and the regularization method. Soviet Doklady, 4:1035-1038, 1963.
49. Michael E. Tipping. Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res., 1:211-244, 2001.
50. Chong Tu, Merlise A. Clyde, and Robert L. Wolpert. Lévy adaptive regression kernels. Discussion Paper 2006-08, Duke University ISDS, Durham, NC, 2006. URL http://www.stat.duke.edu/research/papers/.
51. Vladimir N. Vapnik. Statistical Learning Theory. John Wiley & Sons, New York, NY, 1998.
52. Grace Wahba. Splines Models for Observational Data, volume 59 of Series in Applied Mathematics. SIAM, Philadelphia, PA, 1990.
53. Grace Wahba. Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV. In Bernhard Schölkopf, Alexander J. Smola, Christopher J. C. Burges, and Rosanna Soentpiet, editors, Advances in Kernel Methods: Support Vector Learning, pages 69-88. MIT Press, Cambridge, MA, 1999.
54. Larry Wasserman. All of Nonparametric Statistics. Springer-Verlag, 2005.
55. Mike West. Hyperparameter estimation in Dirichlet process mixture models. Discussion Paper 1992-03, Duke University ISDS, Durham, NC, 1992. URL http://www.stat.duke.edu/research/papers/.
56. Robert L. Wolpert and Katja Ickstadt. Reflecting uncertainty in inverse problems: A Bayesian solution using Lévy processes. Inverse Problems, 20(6):1759-1771, 2004.
57. Robert L. Wolpert, Katja Ickstadt, and Martin Bøgsted Hansen. A nonparametric Bayesian approach to inverse problems (with discussion). In José Miguel Bernardo, Maria Jesus Bayarri, James O. Berger, A. Phillip Dawid, David Heckerman, Adrian F. M. Smith, and Mike West, editors, Bayesian Statistics 7, pages 403-418, Oxford, UK, 2003. Oxford Univ. Press. ISBN 0-19-852615-6.
58. st International Conference (ICML 2004), Banff, Canada, New York, NY, 2004. ACM Press. URL http://www.aicml.cs.ualberta.ca/_banff04/icml/pages/accepted.htm.
59. rd International Conference (ICML 2006), Pittsburgh, PA, New York, NY, 2006. ACM Press. URL http://www.icml2006.org/icml2006/technical/accepted.html.
60. Ding-Xuan Zhou. Capacity of reproducing kernel spaces in learning theory. IEEE T. Inform. Theory, 49:1743-1752, 2003.