A correlated worker model for grouped, imbalanced and multitask data

32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016

Volumn , Issue , 2016, Pages 537-546

  Nguyen, An T ^a   Wallace, Byron C ^b   Lease, Matthew ^b  

^a University of Texas at Austin   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; IMAGE RESOLUTION; MATRIX ALGEBRA;

ASYMMETRIC COSTS; BINARY CLASSIFICATION; BIOMEDICAL CITATIONS; CLASSIFICATION TASKS; CONFUSION MATRICES; IMBALANCED CLASSIFICATION; MORPHOLOGICAL CLASSIFICATIONS; VARIATIONAL INFERENCE;

CLASSIFICATION (OF INFORMATION);

EID: 85002216153     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Bi, W., Wang, L., Kwok, J. T., and Tu, Z. (2014). Learning to predict from crowdsourced data. In Uncertainty in Artificial Intelligence.
2. Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M. A., Guo, J., Li, P., and Riddell, A. (2015). Stan: a probabilistic programming language. Journal of Statistical Software.
3. Dahabreh, I. J., Trikalinos, T. A., Lau, J., and Schmid, C. (2012). An Empirical Assessment of Bivariate Methods for Meta-Analysis of Test Accuracy. Agency for Healthcare Research and Quality (US).
4. Dawid, A. P. and Skene, A. M. (1979). Maximum likelihood estimation of observer error-rates using the em algorithm. Applied statistics, pages 20-28.
5. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society. Series B (methodological), pages 1-38.
6. Felt, P., Ringger, E., Seppi, K., Black, K., and Haertel, R. (2015). Early gains matter: A case for preferring generative over discriminative crowdsourcing models. In Conference of the North American Chapter of the Association for Computational Linguistics.
7. Hedges, L. V. (1994). Fixed effects models. The handbook of research synthesis, pages 285-299.
8. Kajino, H., Tsuboi, Y., and Kashima, H. (2012). A convex formulation for learning from crowds. In Twenty-Sixth AAAI Conference on Artificial Intelligence.
9. Kim, H.-C. and Ghahramani, Z. (2012). Bayesian classifier combination. In International conference on artificial intelligence and statistics, pages 619-627.
10. Koller, D. and Friedman, N. (2009). Probabilistic graphical models: principles and techniques. MIT press.
11. Lakkaraju, H., Leskovec, J., Kleinberg, J., and Mullainathan, S. (2015). A Bayesian framework for modeling human evaluations. In Proceedings of the 2015 SIAM International Conference on Data Mining, pages 181-189.
12. Liu, C. and Wang, Y.-m. (2012). Truelabel+ confusions: A spectrum of probabilistic models in analyzing multiple ratings. In Proceedings of the 29th International Conference on Machine Learning (ICML-12), pages 225-232.
13. Liu, Q., Peng, J., and Ihler, A. T. (2012). Variational inference for crowdsourcing. In Advances in Neural Information Processing Systems, pages 692-700.
14. Nguyen, A. T., Wallace, B. C., and Lease, M. (2015). Combining crowd and expert labels using decision theoretic active learning. In Proceedings of the 3rd AAAI Conference on Human Computation (HCOMP).
15. Patil, A., Huard, D., and Fonnesbeck, C. J. (2010). Pymc: Bayesian stochastic modelling in python. Journal of statistical software, 35(4):1.
16. Raykar, V. C., Yu, S., Zhao, L. H., Valadez, G. H., Florin, C., Bogoni, L., and Moy, L. (2010). Learning from crowds. Journal of Machine Learning Research, 11:1297-1322.
17. Reitsma, J. B., Glas, A. S., Rutjes, A. W., Scholten, R. J., Bossuyt, P. M., and Zwinderman, A. H. (2005). Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. Journal of clinical epidemiology, 58(10):982-990.
18. Sheshadri, A. and Lease, M. (2013). Square: A benchmark for research on computing crowd consensus. In First AAAI Conference on Human Computation and Crowdsourcing.
19. Simpson, E., Roberts, S., Psorakis, I., and Smith, A. (2013). Dynamic Bayesian combination of multiple imperfect classifiers. In Decision Making and Imperfection, pages 1-35. Springer.
20. Spiegelhalter, D. J., Thomas, A., Best, N. G., and Gilks, W. R. (1995). Bugs: Bayesian inference using gibbs sampling, version 0.50. MRC Biostatistics Unit, Cambridge.
21. Venanzi, M., Guiver, J., Kazai, G., Kohli, P., and Shokouhi, M. (2014). Community-based Bayesian aggregation models for crowdsourcing. In Proceedings of the 23rd international conference on World wide web, pages 155-164. ACM.
22. Wainwright, M. J. and Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and TrendsR in Machine Learning, 1(1-2):1-305.
23. Wang, C. and Blei, D. M. (2013). Variational inference in nonconjugate models. The Journal of Machine Learning Research, 14(1):1005-1031.
24. Werling, K., Chaganty, A. T., Liang, P. S., and Manning, C. D. (2015). On-the-job learning with Bayesian decision theory. In Advances in Neural Information Processing Systems, pages 3447-3455.
25. Willett, K. W., Lintott, C. J., Bamford, S. P., Masters, K. L., Simmons, B. D., Casteels, K. R., Edmondson, E. M., Fortson, L. F., Kaviraj, S., Keel, W. C., et al. (2013). Galaxy zoo 2: detailed morphological classifications for 304 122 galaxies from the sloan digital sky survey. Monthly Notices of the Royal Astronomical Society, page stt1458.