Learning Supervised Topic Models for Classification and Regression from Crowds

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 39, Issue 12, 2017, Pages 2409-2422

  Rodrigues, Filipe ^a   Lourenco, Mariana ^b   Ribeiro, Bernardete ^b   Pereira, Francisco C ^a,c  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

^b UNIVERSITY OF COIMBRA   (Portugal)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

crowdsoucing; multiple annotators; supervised learning; Topic models

Indexed keywords

INFERENCE ENGINES; STOCHASTIC SYSTEMS; SUPERVISED LEARNING;

CROWDSOUCING; LARGE DATASETS; MULTIPLE ANNOTATORS; REGRESSION PROBLEM; RELATED VARIABLES; STATE-OF-THE-ART APPROACH; TOPIC MODEL; VARIATIONAL INFERENCE;

STOCHASTIC MODELS;

EID: 85030708259     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2017.2648786     Document Type: Article

References (36)

1. D. Blei, A. Ng, and M. I. Jordan, "Latent Dirichlet allocation, " J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
2. L. Fei-Fei and P. Perona, "A Bayesian hierarchical model for learning natural scene categories, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2005, vol. 2, pp. 524-531.
3. C. Wang, D. Blei, and L. Fei-Fei, "Simultaneous image classification and annotation, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1903-1910.
4. J. Niebles, H. Wang, and L. Fei-Fei, "Unsupervised learning of human action categories using spatial-temporal words, " Int. J. Comput. Vis., vol. 79, no. 3, pp. 299-318, 2008.
5. E. Erosheva, S. Fienberg, and C. Joutard, "Describing disability through individual-level mixture models for multivariate binary data, " Ann. Appl. Statist., vol. 1, no. 2, 2007, Art. no. 346.
6. E. Airoldi, D. Blei, S. Fienberg, and E. Xing, "Combining stochastic block models and mixed membership for statistical network analysis, " in Statistical Network Analysis. Berlin, Germany: Springer, 2007, pp. 57-74.
7. J. Mcauliffe and D. Blei, "Supervised topic models, " in Proc. Advances Neural Inf. Process. Syst., 2008, pp. 121-128.
8. J. Zhu, A. Ahmed, and E. Xing, "MedLDA: Maximum margin supervised topic models, " J. Mach. Learn. Res., vol. 13, no. 1, pp. 2237-2278, 2012.
9. R. Snow, B. O'Connor, D. Jurafsky, and A. Ng, "Cheap and fastbut is it good?: Evaluating non-expert annotations for natural language tasks, " in Proc. Conf. Empirical Methods Natural Language Process., 2008, pp. 254-263.
10. F. Rodrigues, F. Pereira, and B. Ribeiro, "Learning from multiple annotators: distinguishing good from random labelers, " Pattern Recognit. Lett., vol. 34, pp. 1428-1436, 2013.
11. P. Welinder, S. Branson, P. Perona, and S. Belongie, "The multidimensional wisdom of crowds, " in Proc. Advances Neural Inf. Process. Syst., 2010, pp. 2424-2432.
12. Y. Yan, R. Rosales, G. Fung, R. Subramanian, and J. Dy, "Learning from multiple annotators with varying expertise, " Mach. Learn., vol. 95, no. 3, pp. 291-327, 2014.
13. F. Rodrigues, M. Lourenço, F. Pereira, and B. Ribeiro, "Learning supervised topic models from crowds, " in Proc. 3rd AAAI Conf. Human Comput. Crowdsourcing, 2015.
14. M. Hoffman, D. Blei, C. Wang, and J. Paisley, "Stochastic variational inference, " J. Mach. Learn. Res., vol. 14, pp. 1303-1347, 2013.
15. S. Lacoste-Julien, F. Sha, and M. Jordan, "DiscLDA: Discriminative learning for dimensionality reduction and classification, " in Proc. Advances Neural Inf. Process. Syst., 2009, pp. 897-904.
16. D. Ramage, D. Hall, R. Nallapati, and C. Manning, "Labeled LDA: A supervised topic model for credit attribution in multi-labeled corpora, " in Proc. Conf. Empirical Methods Natural Language Process., 2009, pp. 248-256.
17. D. Mimno and A. McCallum, "Topic models conditioned on arbitrary features with Dirichlet-multinomial regression, " in Proc. 24th Conf. Uncertainty Artif. Intell., 2008, pp. 411-418.
18. M. Rabinovich and D. Blei, "The inverse regression topic model, " in Proc. 31st Int. Conf. Mach. Learn., 2014, pp. 199-207.
19. M. Taddy, "Multinomial inverse regression for text analysis, " J. Amer. Statist. Assoc., vol. 108, no. 503, pp. 755-770, 2013.
20. A. P. Dawid and A. M. Skene, "Maximum likelihood estimation of observer error-rates using the EM algorithm, " J. Roy. Statist. Soc. Series C, vol. 28, no. 1, pp. 20-28, 1979.
21. P. Smyth, U. Fayyad, M. Burl, P. Perona, and P. Baldi, "Inferring ground truth from subjective labelling of venus images, " in Proc. Advances Neural Inf. Process. Syst., 1995, pp. 1085-1092.
22. V. Raykar, et al., "Learning from crowds, " J. Mach. Learn. Res., vol. 11, pp. 1297-1322, 2010.
23. F. Rodrigues, F. Pereira, and B. Ribeiro, "Gaussian process classification and active learning with multiple annotators, " in Proc. 31st Int. Conf. Mach. Learn., 2014, pp. 433-441.
24. P. Groot, A. Birlutiu, and T. Heskes, "Learning from multiple annotators with Gaussian processes, " in Proc. Int. Conf. Artif. Neural Netw., 2011, vol. 6792, pp. 159-164.
25. Y. Yan, et al., "Modeling annotator expertise: Learning when everybody knows a bit of something, " J. Mach. Learn. Res., vol. 9, pp. 932-939, 2010.
26. M. I. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul, "An introduction to variational methods for graphical models, " Mach. Learn., vol. 37, no. 2, pp. 183-233, 1999.
27. J. Chuang, S. Gupta, C. Manning, and J. Heer, "Topic model diagnostics: Assessing domain relevance via topical alignment, " in Proc. 30th Int. Conf. Mach. Learn., 2013, pp. 612-620.
28. J. Nocedal and S. Wright, Numerical Optimization. Singapore: World Scientific, 2006.
29. H. Robbins and S. Monro, "A stochastic approximation method, " Ann. Math. Statist., vol. 22, no. 3, pp. 400-407, Sep. 1951.
30. K. Lang, "NewsWeeder: Learning to filter netnews, " in Proc. Int. Conf. Mach. Learn., 1995, pp. 331-339.
31. D. Lewis, "Reuters-21578 text categorization test collection, " 1997.
32. B. Russell, A. Torralba, K. Murphy, and W. Freeman, "LabelMe: A database and web-based tool for image annotation, " Int. J. Comput. Vis., vol. 77, no. 1-3, pp. 157-173, 2008.
33. D. Lowe, "Object recognition from local scale-invariant features, " in Proc. Int. Conf. Comput. Vis., 1999, vol. 2, pp. 1150-1157.
34. A. Bosch, A. Zisserman, and X. Munoz, "Scene classification via pLSA, " in Proc. Eur. Conf. Comput. Vis., 2006, pp. 517-530.
35. D. Maua and F. Cozman, "Representing and classifying user reviews, " 2009.
36. B. Pang and L. Lee, "Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales, " in Proc. 43rd Annu. Meeting Assoc. Comput. Linguistics, 2005, pp. 115-124.