Classification with incomplete data using dirichlet process priors

Journal of Machine Learning Research

Volumn 11, Issue , 2010, Pages 3269-3311

  Wang, Chunping ^a   Liao, Xuejun ^a   Carin, Lawrence ^a   Dunson, David B ^b  

^a Duke University   (United States)

^b Duke University   (United States)

Author keywords

Classification; Dirichlet process; Expert; Incomplete data; Multitask learning; Variational Bayesian

Indexed keywords

CLASSIFICATION; DIRICHLET PROCESS; EXPERT; INCOMPLETE DATA; MULTITASK LEARNING; VARIATIONAL BAYESIAN;

BAYESIAN NETWORKS; CLASSIFIERS;

DATA HANDLING;

EID: 79551481555     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (57)

1. J. H. Albert and S. Chib. Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88:669-679, 1993.
2. R. K. Ando and T. Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817-1853, 2005.
3. H. Attias. A variational Bayesian framework for graphical models. In Advances in Neural Information Processing Systems (NIPS), 2000.
4. M. J. Beal. Variational Algorithms for Approximate Bayesian Inference. PhD dissertation, University College London, Gatsby Computational Neuroscience Unit, 2003.
5. D.M. Blei andM. I. Jordan. Variational inference for Dirichlet process mixtures. Bayesian Analysis, 1(1):121-144, 2006.
6. R. Caruana. Multitask learning. Machine Learning, 28:41-75, 1997.
7. G. Chechik, G. Heitz, G. Elidan, P. Abbeel, and D. Koller. Max-margin classification of data with absent features. Journal of Machine Learning Research, 9:1-21, 2008.
8. L. M. Collins, J. L. Schafer, and C. M. Kam. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4):330-351, 2001.
9. A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society B, 39:1-38, 1977.
10. U. Dick, P. Haider, and T. Scheffer. Learning from incomplete data with infinite imputations. In International Conference on Machine Learning (ICML), 2008.
11. D. B. Dunson, N. Pillai, and J.-H. Park. Bayesian density regression. Journal of the Royal Statistical Society: Series B, 69, 2007.
12. M. D. Escobar and M. West. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90:577-588, 1995.
13. T. Ferguson. A Bayesian analysis of some nonparametric problems. The Annals of Statistics, 1: 209-230, 1973.
14. A. E. Gelfand, S. E. Hills, A. Racine-Poon, and A. F. M. Smith. Illustration of Bayesian inference in normal data models using Gibbs sampling. Journal of American Statistical Association, 85: 972-985, 1990.
15. Z. Ghahramani and M. J. Beal. Variational inference for Bayesian mixtures of factor analysers. In Advances in Neural Information Processing Systems (NIPS) 12, pages 449-455. MIT Press, 2000.
16. Z. Ghahramani and G. E. Hinton. The EM algorithm for mixtures of factor analyzers. Technical Report CRG-TR-96-1, Department of Computer Science, University of Toronto, 1996.
17. Z. Ghahramani and M. I. Jordan. Learning from incomplete data. Technical report, Massachusetts Institute of Technology, 1994.
18. T. Graepel. Kernel matrix completion by semidefinite programming. In Proceedings of the International Conference on Artificial Neural Networks, pages 694-699, 2002.
19. J. Hanley and B. McNeil. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143:29-36, 1982.
20. L. Hannah, D. Blei, and W. Powell. Dirichlet process mixtures of generalized linear models. In Artificial Intelligence and Statistics (AISTATS), pages 313-320, 2010.
21. J. Ibrahim. Incomplete data in generalized linear models. Journal of the American Statistical Association, 85:765-769, 1990.
22. H. Ishwaran and L. F. James. Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 96:161-173, 2001.
23. R. A. Jacobs, M. I. Jordon, S. J. Nowlan, and G. E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3:79-87, 1991.
24. M. I. Jordan and R. A. Jacobs. Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6:181-214, 1994.
25. K. Kurihara, M. Welling, and Y. W. Teh. Collapsed variational Dirichlet process mixture models. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pages 2796-2801, 2007.
26. Percy Liang and Michael I. Jordan. An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators. In Proceedings of the International Conference on Machine Learning (ICML), pages 584-591, 2008.
27. X. Liao, H. Li, and L. Carin. Quadratically gated mixture of experts for incomplete data classification. In Proceedings of the International Conference on Machine Learning (ICML), pages 553-560, 2007.
28. Q. Liu, X. Liao, and L. Carin. Semi-supervised multitask learning. In Neural Information Processing Systems, 2007.
29. S. N. MacEachern and P. Müller. Estimating mixture of Dirichlet process models. Journal of Computational and Graphical Statistics, 7, 1998.
30. E. Meeds and S. Osindero. An alternative infinite mixture of Gaussian process experts. In NIPS 18, pages 883-890. MIT Press, 2006.
31. P. Müller, A. Erkanli, and M. West. Bayesian curve fitting using multivariate normal mixtures. Biometrika, 83:67-79, 1996.
32. R. M. Neal. Probabilistic inference using Markov chain Monte Carlo methods. Technical report, Department of Computer Science, University of Toronto, 1993.
33. D. J. Newman, S. Hettich, C. L. Blake, and C. J. Merz. UCI repository of machine learning databases. http://www.ics.uci.edu/-mlearn/MLRepository.html, 1998.
34. A. Y. Ng and M. I. Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes. In Advances in Neural Information Processing Systems (NIPS), 2002.
35. C. E. Rasmussen and Z. Ghahramani. Infinite mixtures of Gaussian process experts. In NIPS 14. MIT Press, 2002.
36. A. Rodríguez, D. B. Dunson, and A. E. Gelfang. Bayesian nonparametric functional data analysis through density estimation. Biometrika, 96, 2009.
37. D. B. Rubin. Inference and missing data. Biometrika, 63:581-592, 1976.
38. D. B. Rubin. Multiple Imputation for Nonresponse in Surveys. John Wiley &Sons, Inc., 1987.
39. J. L. Schafer and J. W. Graham. Missing data: Our view of the state of the art. Psychological Methods, 7:147-177, 2002.
40. J. Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, 1:639-650, 1994.
41. B. Shahbaba and R. Neal. Nonlinear models using Dirichlet process mixtures. Journal of Machine Learning Research, 10:1829-1850, 2009.
42. P. K. Shivaswamy, C. Bhattacharyya, and A. J. Smola. Second order cone programming approaches for handling missing and uncertain data. Journal of Machine Learning Research, 7:1283-1314, 2006.
43. A. Smola, S. Vishwanathan, and T. Hofmann. Kernel methods for missing variables. In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, 2005.
44. Y. W. Teh, M. J. Beal M. I. Jordan, and D. M. Blei. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101:1566-1581, 2006.
45. M. E. Tipping. The relevance vector machine. In T. K. Leen S. A. Solla and K. R. Müller, editors, Advances in Neural Information Processing Systems (NIPS), volume 12, pages 652-658. MIT Press, 2000.
46. V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 1995.
47. X. Wang, A. Li, Z. Jiang, and H. Feng. Missing value estimation for DNA microarray gene expression data by support vector regression imputation and orthogonal coding scheme. BMC Bioinformatics, 7:32, 2006.
48. S. R. Waterhouse and A. J. Robinson. Classification using hierarchical mixtures of experts. In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing IV, pages 177- 186, 1994.
49. M. West, P. Müller, and M. D. Escobar. Hierarchical priors and mixture models, with application in regression and density estimation. In P. R. Freeman and A. F. Smith, editors, Aspects of Uncertainty, pages 363-386. John Wiley, 1994.
50. D.Williams and L. Carin. Analytical kernel matrix completion with incomplete multi-view data. In Proceedings of the International Conference on Machine Learning (ICML) Workshop on Learning with Multiple Views, pages 80-86, 2005.
51. D. Williams, X. Liao, Y. Xue, L. Carin, and B. Krishnapuram. On classification with incomplete data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3):427-436, 2007.
52. L. Xu, M. I. Jordan, and G. E. Hinton. An alternative model for mixtures of experts. In Advances in Neural Information Processing Systems (NIPS) 7, pages 633-640, 1995.
53. Y. Xue, X. Liao, L. Carin, and B. Krishnapuram. Multi-task learning for classification with Dirichlet process priors. Journal of Machine Learning Research, 8:35-63, 2007. (Pubitemid 46155123)
54. K. Yu, A. Schwaighofer, V. Tresp, W.-Y. Ma, and H. Zhang. Collaborative ensemble learning: Combining collaborative and content-based information filtering via hierarchical Bayes. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, pages 616-623, 2003.
55. J. Zhang, Z. Ghahramani, and Y. Yang. Learning multiple related tasks using latent independent component analysis. In Advances in Neural Information Processing Systems, 2006.
56. Y. Zhang, L. M. Collins, H. Yu, C. Baum, and L. Carin. Sensing of unexploded ordnance with magnetometer and induction data: theory and signal processing. IEEE Transactions on Geoscience and Remote Sensing, 41(5):1005-1015, 2003.
57. X. Zhu. Semi-supervised learning literature survey. Technical Report 1530, Computer Sciences, University of Wisconsin-Madison, 2005.