Variational multinomial logit gaussian process

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 1745-1808

  Chai, Kian Ming A ^a  

^a DSO NATIONAL LABORATORIES   (Singapore)

Author keywords

Gaussian process; Multinomial logistic; Probabilistic classification; Sparse approximation; Variational approximation

Indexed keywords

GAUSSIAN PROCESSES; MULTINOMIALS; PROBABILISTIC CLASSIFICATION; SPARSE APPROXIMATIONS; VARIATIONAL APPROXIMATION;

EXPERIMENTS; GAUSSIAN DISTRIBUTION;

GAUSSIAN NOISE (ELECTRONIC);

EID: 84864936855     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (52)

1. T. Ando. Concavity of certain maps on positive definite matrices and applications to Hadamard products. Linear Algebra and Its Applications, 26:203-241, 1979.
2. D. Angluin and P. Laird. Learning from noisy examples. Machine Learning, 2:343-370, 1988.
3. A. Banerjee. On Bayesian bounds. In International Conference on Machine Learning, 2006.
4. E. V. Bonilla, K. M. A. Chai, and C. K. I. Williams. Multi-task Gaussian process prediction. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems, volume 20, pages 153-160. Curran Associates, Inc., 2008.
5. E. J. Bredensteiner and K. P. Bennett. Multicategory classification by support vector machines. Computational Optimization Applications, 12:53-79, January 1999. (Pubitemid 129574925)
6. J. S. Bridle. Probabilistic interpretation of feedforward classification network outputs with relationships to statistical pattern recognition. In Neuro-computing: Algorithms, Architectures and Applications, NATO ASI Series in Systems and Computer Science. Springer, 1989.
7. E. Challis and D. Barber. Concave Gaussian variational approximations for inference in large-scale Bayesian linear models. In D. Dunson and M. Dudk, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of JMLR: Workshop and Conference Proceedings Series, 2011.
8. K. Crammer and Y. Singer. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2:265-292, 2001.
9. L. Csat́o andM. Opper. Sparse online Gaussian processes. Neural Computation, 14:641-668, 2002.
10. A. Frank and A. Asuncion. UCI machine learning repository, 2010. URL http://archive.ics. uci.edu/ml.
11. J. Geweke. Contemporary Bayesian Econometrics and Statistics. Wiley Series in Probability and Statistics. John Wiley & Sons, Inc., New Jersey, 2005.
12. J. Geweke, M. P. Keane, and D. Runkle. Alternative computational approaches to inference in the multinomial probit model. The Review of Economics and Statistics, 76(4):609-32, 1994.
13. Z. Ghahramani andM. J. Beal. Graphical models and variational methods. In D. Saad andM. Opper, editors, Advanced Mean Field methods -Theory and Practice. MIT Press, 2000a.
14. Z. Ghahramani and M. J. Beal. Variational inference for Bayesian mixtures of factor analysers. In S. A. Solla, T. K. Leen, and K.-R. M̈uller, editors, Advances in Neural Information Processing Systems, volume 12, pages 449-455. MIT Press, Cambridge, MA, 2000b.
15. M. N. Gibbs. Bayesian Gaussian Processes for Regression and Classification. PhD thesis, University of Cambridge, Department of Physics, 1997.
16. M. Girolami and S. Rogers. Variational Bayesian multinomial probit regression with Gaussian process priors. Neural Computation, 18(8):1790-1817, 2006. (Pubitemid 44036395)
17. Y. Guermeur. Combining discriminant models with new multi-class SVMs. Pattern Analysis and Applications, 5(2):168-179, 2002. (Pubitemid 40830130)
18. Y. Guermeur and E. Monfrini. A quadratic loss multi-class SVM for which a radius-margin bound applies. Informatica, 22(1):73-96, 2011.
19. P. Hall. Chi squared approximations to the distribution of a sum of independent random variables. The Annals of Probability, 11(4):1028-1036, 1983.
20. R. Henao and O.Winther. PASS-GP: Predictive active set selection for Gaussian processes. In IEEE International Workshop on Machine Learning for Signal Processing, pages 148-153, 2010.
21. R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, 1985.
22. C.-W. Hsu and C.-J. Lin. A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks, 13(2):415-425, 2002. (Pubitemid 34475042)
23. H.-C. Kim and Z. Ghahramani. Bayesian Gaussian process classification with the EM-EP algorithm. IEEE Transactions on Pattern Analysis Machine Intelligence, 28:1948-1959, 2006. (Pubitemid 46405706)
24. D. A. Knowles and T. P. Minka. Non-conjugate variational message passing for multinomial and binary regression. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24, pages 1701-1709. 2011.
25. S. J. Koopman, N. Shephard, and D. Creal. Testing the assumptions behind importance sampling. Journal of Econometrics, 149(1):2-11, 2009.
26. S. Kotz and S. Nadarajah. Multivariate t Distributions and Their Applications. Cambridge University Press, Cambridge, UK, 2004.
27. F. Lauer and Y. Guermeur. MSVMpack: a multi-class support vector machine package. Journal of Machine Learning Research, 12:2269-2272, 2011. http://www.loria.fr/lauer/MSVMpack.
28. N. D. Lawrence, M. Seeger, and R. Herbrich. Fast sparse Gaussian process methods: The informative vector machine. In Advances in Neural Information Processing Systems, volume 15, pages 609-616, Cambridge, MA, 2003. MIT Press.
29. M. Ĺazaro-Gredilla and A. Figueiras-Vidal. Inter-domain Gaussian processes for sparse inference using inducing features. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 22, pages 1087-1095. Curran Associates, Inc., 2009.
30. Y. Lee, Y. Lin, and G. Wahba. Multicategory support vector machines. Journal of the American Statistical Association, 99(465):67-81, 2004.
31. D. McFadden. Conditional logit analysis of qualitative choice behavior. In P. Zarembka, editor, Frontiers in Econometrics, pages 105-142. Academic Press, 1974.
32. T. P. Minka. Beyond Newton's method, 2002. URL http://research.microsoft. com/en-us/um/people/minka/papers/minka-newton.pdf.
33. R. M. Neal. Bayesian Learning for Neural Networks. Springer-Verlag New York, 1996.
34. R.M. Neal. Regression and classification using Gaussian process priors. In A. P. D. J.M. Bernardo, J. O. Berger and A. F. M. Smith, editors, Bayesian Statistics, volume 6, pages 475-501. Oxford University Press, 1998.
35. R. M. Neal. Annealed importance sampling. Statistics and Computing, 11:125-139, 2001. (Pubitemid 33638185)
36. H. Nickisch and C. E. Rasmussen. Approximations for binary Gaussian process classification. Journal of Machine Learning Research, 9:2035-2078, 2008.
37. J. E. Potter. Matrix quadratic solutions. SIAM Journal on Applied Mathematics, 14(3):496-501, May 1966.
38. S. Puntanen and G. P. H. Styan. Historical introduction: Issai Schur and the early development of the Schur complement. In F. Zhang, editor, The Schur Complement and Its Applications, Numerical Methods and Algorithms, pages 1-16. Springer, 2005.
39. J. Quĩnonero-Candela, C. E. Rasmussen, and C. K. I. Williams. Approximation methods for Gaussian process regression. In L. Bottou, O. Chapelle, D. DeCoste, and J. Weston, editors, Large Scale Kernel Machines, pages 203-223. MIT Press, 2007.
40. C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, Cambridge, Massachusetts, 2006.
41. R. Rifkin and A. Klautau. In defense of one-vs-all classification. Journal of Machine Learning Research, 5:101-141, 2004.
42. M. Seeger. PAC-Bayesian generalisation error bounds for Gaussian process classification. Journal of Machine Learning Research, 3:233-269, 2002.
43. M. Seeger and M. I. Jordan. Sparse Gaussian process classification with multiple classes. Technical report, University of California at Berkeley, Department of Statistics, 2004.
44. E. Snelson and Z. Ghahramani. Sparse Gaussian processes using pseudo-inputs. In Y. Weiss, B. Scḧolkopf, and J. Platt, editors, Advances in Neural Information Processing Systems, volume 18, pages 1257-1264, Cambridge, MA, 2006. MIT Press.
45. M. Titsias. Variational learning of inducing variables in sparse Gaussian processes. In D. van Dyk and M. Welling, editors, Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, volume 5 of JMLR: Workshop and Conference Proceedings Series, pages 567-574, 2009a.
46. M. Titsias. Variational model selection for sparse Gaussian process regression. Technical report, University of Manchester, School of Computer Science, 2009b. URL http://www.cs.man.ac. uk/mtitsias/papers/sparseGPv2.pdf.
47. V. N. Vapnik. Statistical Learning Theory. Wiley-Interscience, New York, 1998.
48. J. M. Ver Hoef and R. P. Barry. Constructing and fitting models for cokriging and multivariable spatial prediction. Journal of Statistical Planning and Inference, 69(2):275-294, 1998. (Pubitemid 128182034)
49. G. S. Watson. Spectral decomposition of the covariance matrix of a multinomial. Journal of the Royal Statistical Society, Series B (Methodological), 58(1):289-291, 1996.
50. J.Weston and C.Watkins. Support vector machines for multi-class pattern recognition. In Proceedings of the 7th European Symposium on Artificial Neural Network, 1999.
51. C. K. I. Williams and M. Seeger. Using the Nystr ̈om method to speed up kernel machines. In T. K. Leen, T. G. Diettrich, and V. Tresp, editors, Advances in Neural Information Processing Systems, volume 13, pages 682-688, Cambridge, MA, 2001. MIT Press.
52. C. K.Williams and D. Barber. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1342-1351, 1998. (Pubitemid 128741378)