General latent feature models for heterogeneous datasets

Journal of Machine Learning Research

Volumn 21, Issue , 2020, Pages

  Valera, Isabel ^a,b   Pradier, Melanie F ^c   Lomeli, Maria ^d   Ghahramani, Zoubin ^d,e  

^a SAARLAND UNIVERSITY   (Germany)

^b MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

^c HARVARD UNIVERSITY   (United States)

^d UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^e UBER TECHNOLOGIES INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA HANDLING; INFERENCE ENGINES; INFORMATION ANALYSIS; ITERATIVE METHODS;

EXPLORATORY DATA ANALYSIS; HETEROGENEOUS DATASETS; INFERENCE ALGORITHM; LATENT VARIABLE MODELS; LINEAR COMBINATIONS; MISSING INFORMATION; PRIOR DISTRIBUTION; REAL-WORLD DATASETS;

DISTRIBUTED DATABASE SYSTEMS;

EID: 85087333635     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (78)

1. O. Aguilar and M. West. Bayesian dynamic factor models and portfolio allocation. Journal of Business & Economic Statistics, 18(3):338–357, 2000.
2. R. M. Alvarez and J. Nagler. Economics, issues and the perot candidacy: voter choice in the 1992 presidential election. American Journal of Political Science, pages 714–744, 1995.
3. T. W. Anderson and Herman Rubin. Statistical inference in factor analysis. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume 5: Contributions to Econometrics, Industrial Research, and Psychometry, pages 111–150, Berkeley, Calif., 1956. University of California Press. URL https://projecteuclid.org/euclid.bsmsp/1200511860.
4. C. Blanco, R. F. Krueger, D. S. Hasin, S. M. Liu, S. Wang, B. T. Kerridge, T. Saha, and M. Olfson. Mapping common psychiatric disorders: Structure and predictive validity in the National Epidemiologic Survey on Alcohol and Related Conditions. Journal of the American Medical Association Psychiatry, 70(2):199–208, 2013.
5. T. Broderick, J. Pitman, and M. I. Jordan. Feature allocations, probability functions, and paintboxes. Bayesian Analysis, 2013.
6. W. Buntine and A. Jakulin. Discrete Component Analysis. arXiv:math/0604410, 3940: 1–33, 2006.
7. D. P. Byar and S. B. Green. The choice of treatment for cancer patients based on covariate information: application to prostate cancer. Bulletin du Cancer, 67:477–490, 1980.
8. Pew Research Centre. 25th anniversary of the web. Available on: http://www.pewinternet.org/datasets/january-2014-25th-anniversary-of-the-webomnibus/, 2014.
9. W. Chu and Z. Ghahramani. Gaussian processes for ordinal regression. J. Mach. Learn. Res., 6:1019–1041, December 2005. ISSN 1532-4435.
10. M. Collins, S. Dasgupta, and R. E. Schapire. A generalization of principal components analysis to the exponential family. In Advances in neural information processing systems, pages 617–624, 2002.
11. G. Conti, S. Frühwirth-Schnatter, J.J. Heckman, and R. Piatek. Bayesian exploratory factor analysis. Journal of Econometrics, 2014.
12. P. Cortez, A. Cerdeira, F. Almeida, T. Matos, and J. Reis. Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems. Dataset available on: http://archive.ics.uci.edu/ml/datasets.html, 47(4):547–553, 2009.
13. B. P. Dohrenwend. Sociocultural and social-psychological factors in the genesis of mental disorders. Journal of Health and Social Behavior, 16(4):365–392, 1975.
14. F. Doshi-Velez and Z. Ghahramani. Accelerated sampling for the indian buffet process. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 273–280, 2009.
15. F. Doshi-Velez, K. T. Miller, J. Van Gael, and Y. W. Teh. Variational inference for the indian buffet process. In AISTATS, 2009.
16. J. Eggermont, J. N. Kok, and W. A Kosters. Genetic programming for data classification: Partitioning the search space. In Proceedings of the 2004 ACM symposium on Applied computing, pages 1001–1005, 2004.
17. S. Frühwirth-Schnatter and H. F. Lopes. Parsimonious bayesian factor analysis when the number of factors is unknown. Unpublished Working Paper, Booth Business, 2010.
18. J. Geweke and G. Zhou. Measuring the Pricing Error of the Arbitrage Pricing Theory. CEMA Working Papers 276, China Economics and Management Academy, Central University of Finance and Economics, 1996.
19. Z. Ghahramani and G. Hinton. The EM algorithm for mixture of factor analysers. Technical report, University of Toronto, 1996.
20. M. Girolami and S. Rogers. Variational Bayesian multinomial probit regression with Gaussian process priors. Neural Computation, 18:2006, 2005.
21. P. Gopalan, F. J. R. Ruiz, R. Ranganath, and D. M. Blei. Bayesian Nonparametric Poisson Factorization for Recommendation Systems. International Conference on Artificial Intelligence and Statistics (AISTATS), 2014.
22. D. Görür and C. E. Rasmussen. Nonparametric mixtures of factor analyzers. In Signal Processing and Communications Applications Conference, 2009. SIU 2009. IEEE 17th, pages 708–711. IEEE, 2009.
23. D. Greene, G. Cagney, N. Krogan, and P. Cunningham. Ensemble non-negative matrix factorization methods for clustering proteinâĂŞprotein interactions. Bioinformatics, 24 (15):1722–1728, August 2008. ISSN 1367-4803. doi: 10.1093/bioinformatics/btn286. URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3493126/.
24. T. L. Griffiths and Z. Ghahramani. The Indian buffet process: an introduction and review. Journal of Machine Learning Research, 12:1185–1224, 2011.
25. S. Gunasekar, P. Ravikumar, and J. Ghosh. Exponential family matrix completion under structural constraints. In Proceedings of the 31st International Conference on Machine Learning, pages 1917–1925, Bejing, China, 22–24 Jun 2014. PMLR.
26. L. A. Hannah, D. M. Blei, and W. B. Powell. Dirichlet process mixtures of generalized linear models. Journal of Machine Learning Research, 12(Jun):1923–1953, 2011.
27. M. D. Hoffman, D. M. Blei, and P. R. Cook. Bayesian Nonparametric Matrix Factorization for Recorded Music. In ICML, pages 439–446, 2010.
28. A. B. Hollingshead and F. C. Redlich. Social stratification and psychiatric disorders. American Sociological Review, 18(2):163–169, 1953.
29. M. C Hughes, G. Hope, L. Weiner, T. H McCoy, R. H Perlis, E. B Sudderth, and F. Doshi-Velez. Prediction-constrained topic models for antidepressant recommendation. arXiv preprint arXiv:1712.00499, 2017.
30. A. Hyvärinen. Independent componen analysis by minimisation of mutual information. Technical report, University of technology, laboratory of computer and information science., 1997.
31. R. Kay. Treatment effects in competing-risks analysis of prostate cancer data. Biometrics, 42(1):203–211, 1986.
32. R. C. Kessler, K. A. McGonagle, M. Swartz, D. G. Blazer, and C. B. Nelson. Sex and depression in the national comorbidity survey i: Lifetime prevalence, chronicity and recurrence. Journal of Affective Disorders, 29(2):85–96, 1993.
33. M. E. Khan, A. Aravkin, M. Friedlander, and M. Seeger. Fast dual variational inference for non-conjugate latent gaussian models. In International Conference on Machine Learning, pages 951–959, 2013.
34. Mohammad E Khan, Guillaume Bouchard, Kevin P Murphy, and Benjamin M Marlin. Variational bounds for mixed-data factor analysis. In Advances in Neural Information Processing Systems, pages 1108–1116, 2010.
35. D. P Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
36. A. Klami, S. Virtanen, and S. Kaski. Bayesian exponential family projections for coupled data sources. arXiv preprint arXiv:1203.3489, 2012.
37. D. Lacy and B. C Burden. The vote-stealing and turnout effects of ross perot in the 1992 us presidential election. American Journal of Political Science, pages 233–255, 1999.
38. J. Lee, P. M´’uller, K. Gulukota, and Ji Y. A Bayesian feature allocation model for tumor heterogeneity. Annals of Applied Statistics, 2015.
39. P. Lewis, C. McCracken, and R. Hunt. Politics: Who cares. American Demographics, pages 16–23, 1994.
40. X.-B. Li. A Bayesian approach for estimating and replacing missing categorical data. J. Data and Information Quality, 1(1):3:1–3:11, June 2009. ISSN 1936-1955.
41. H. Lopes and M. West. Bayesian model assessment in factor analysis. Statistica Sinica, 14 (1):41–67, 2004.
42. M. Lunn and D. McNeil. Applying cox regression to competing risks. Biometrics, 51(2): 524–532, 1995.
43. K. Mansouri, T. Ringsted, D. Ballabio, R. Todeschini, and V. Consonni. Quantitative structure activity relationship models for ready biodegradability of chemicals. Journal of Chemical Information and Modeling. Dataset available on: http://archive.ics.uci.edu/ml/datasets.html, 2013.
44. M. Arjumand Masood and Finale Doshi-Velez. A particle-based variational approach to Bayesian Non-negative Matrix Factorization. Journal of Machine Learning Research, 20 (90):1–56, 2019.
45. R. Mazumder, T. Hastie, and R. Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. Journal of machine learning research, 11(Aug):2287–2322, 2010.
46. K. Miller, M. I. Jordan, and T. L. Griffiths. Nonparametric Latent Feature Models for Link Prediction. In Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1276–1284. Curran Associates, Inc., 2009. URL http://papers.nips.cc/paper/3846-nonparametric-latent-feature-models-for-link-prediction.pdf.
47. S. Mohamed, Z. Ghahramani, and K. A. Heller. Bayesian exponential family PCA. In Advances in neural information processing systems, pages 1089–1096, 2009.
48. J. Paisley and L. Carin. Nonparametric Factor Analysis with Beta Process Priors. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML’09, pages 777–784, New York, NY, USA, 2009. ACM.
49. K. Pearson. On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 1901.
50. M. F Pradier, V. Stojkoski, Z. Utkovski, L. Kocarev, and F. Perez-Cruz. Sparse three-parameter restricted indian buffet process for understanding international trade. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2476–2480. IEEE, 2018.
51. M. F. Pradier, B. Reis, Lori Jukofsky, F. Milletti, T. Ohtomo, F. Perez-Cruz, and O. Puig. Case-control indian buffet process identifies biomarkers of response to codrituzumab. BMC cancer, 19(1):278, 2019.
52. C. Reed and Z. Ghahramani. Scaling the indian buffet process via submodular maximization. In International Conference on Machine Learning, pages 1013–1021, 2013.
53. C. P. Robert. Simulation of truncated normal variables. Statistics and computing, 5(2): 121–125, 1995.
54. M. E. Roberts, B. M. Stewart, and D. Tingley. Navigating the local modes of big data. Computational Social Science, 51, 2016.
55. A. S. Ross, M. C. Hughes, and F. Doshi-Velez. Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations. arXiv preprint arXiv:1703.03717, 2017.
56. S. Roweis. EM algorithms for PCA and SPCA. In Neural Information Processing Systems, 1997.
57. F. J. R. Ruiz, I. Valera, C. Blanco, and F. Perez-Cruz. Bayesian nonparametric modeling of suicide attempts. Advances in Neural Information Processing Systems, 25:1862–1870, 2012.
58. F. J. R. Ruiz, I. Valera, C. Blanco, and F. Perez-Cruz. Bayesian nonparametric comorbidity analysis of psychiatric disorders. Journal of Machine Learning Research., 2013.
59. F. J. R. Ruiz, I. Valera, C. Blanco, and F. Perez-Cruz. Bayesian Nonparametric Comorbidity Analysis of Psychiatric Disorders. Journal of Machine Learning Research, 15(1):1215–1247, January 2014.
60. R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems, 2007.
61. R. Salakhutdinov and A. Mnih. Bayesian probabilistic matrix factorization using Markov Chain Monte Carlo. Proceedings of the 25th International Conference on Machine Learning (ICML’08), pages 880–887, 2008.
62. J. L. Schafer and J. W Graham. Missing data: our view of the state of the art. Psychological methods, 7(2):147, 2002.
63. S. Seaman, J. Galati, D. Jackson, and J. Carlin. What is meant by”Missing at Random? Statistical Science, 2013.
64. A. P. Singh and G. J. Gordon. A Unified View of Matrix Factorization Models. In Walter Daelemans, Bart Goethals, and Katharina Morik, editors, Machine Learning and Knowledge Discovery in Databases, volume 5212, pages 358–373. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008.
65. M. Tanner and W. Wong. The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82:528–550, 1987.
66. Y. W. Teh and D. Görür. Indian buffet processes with power-law behavior. In Advances in Neural Information Processing Systems, volume 22, pages 1838–1846, 2009.
67. L. L. Thurstone. Multiple factor analysis. The University of Chicago Press, 1931.
68. M. Tipping and C. Bishop. Probabilistic principal component analysis. Technical report NCRG/97/010, Asto University, Department of computer science and applied mathematics, 1997.
69. M. Titsias. The infinite gamma-Poisson feature model. Advances in Neural Information Processing Systems, 19, 2007.
70. A. Todeschini, F. Caron, and M. Chavent. Probabilistic low-rank matrix completion with adaptive spectral regularization algorithms. In C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 845–853. Curran Associates, Inc., Dec. 2013.
71. Z. Utkovski, M. F. Pradier, V. Stojkoski, F. Perez-Cruz, and L. Kocarev. Economic complexity unfolded: Interpretable model for the productive structure of economies. PloS one, 13(8), 2018.
72. I. Valera and Z. Ghahramani. General table completion using a bayesian nonparametric model. Advances in Neural Information Processing Systems 27 (NIPS 2014), 27:981–989, 2014.
73. I. Valera and Z. Ghahramani. Automatic discovery of the statistical types of variables in a dataset. In International Conference on Machine Learning, pages 3521–3529, 2017.
74. I. Valera, F. J. R. Ruiz, P. M. Olmos, C. Blanco, and F. Perez-Cruz. Infinite continuous feature model for psychiatric comorbidity analysis. Neural computation, 2016.
75. C. Wang and D. M Blei. Variational inference in nonconjugate models. Journal of Machine Learning Research, 14(Apr):1005–1031, 2013.
76. S. Weich and G. Lewis. Poverty, unemployment, and common mental disorders: population based cohort study. BMJ, 317(7151):115–119, 1998.
77. M. M. Weissman, Bland R., P. R. Joyce, S. Newman, J.E. Wells, and H.-U. Wittchen. Sex differences in rates of depression: cross-national perspectives. Journal of Affective Disorders, 29:77 – 84, 1993. Special Issue Toward a New Psychobiology of Depression in Women.
78. S. Williamson, C. Wang, K. Heller, and D. Blei. The IBP compound Dirichlet process and its application to focused topic modeling. Proceedings of the 27th Annual International Conference on Machine Learning, 2010.