FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn 2015-August, Issue , 2015, Pages 745-754

  Ma, Fenglong ^a   Li, Yaliang ^a   Li, Qi ^a   Qiu, Minghui ^b   Gao, Jing ^a   Zhi, Shi ^c   Su, Lu ^a   Zhao, Bo ^d   Ji, Heng ^e   Han, Jiawei ^c  

^a UNIVERSITY AT BUFFALO   (United States)

^b SINGAPORE MANAGEMENT UNIVERSITY   (Singapore)

^c UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^d LINKEDIN CORPORATION   (United States)

^e RENSSELAER POLYTECHNIC INSTITUTE   (United States)

Author keywords

Crowdsourcing; Source reliability; Truth discovery

Indexed keywords

RELIABILITY;

CROWDSOURCING; DATA AGGREGATION; HIGH QUALITY SOURCE; PROBABILISTIC MODELING; REAL-WORLD DATASETS; RELIABILITY DEGREE; SET OF QUESTIONS; TRUTH DISCOVERY;

DATA MINING;

EID: 84954098959     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2783258.2783314     Document Type: Conference Paper

References (31)

1. B. I. Aydin, Y. S. Yilmaz, Y. Li, Q. Li, J. Gao, and M. Demirbas. Crowdsourcing for multiple-choice question answering. In Twenty-Sixth IAAI Conference, pages 2946-2953, 2014.
2. A. P. Dawid and A. M. Skene. Maximum likelihood estimation of observer error-rates using the EM algorithm. In Applied Statistics, pages 20-28, 1979.
3. G. Demartini, D. E. Difallah, and P. Cudré-Mauroux. Zencrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In Proceedings of the 21st International Conference on World Wide Web, pages 469-478, 2012.
4. X. L. Dong, L. Berti-Equille, and D. Srivastava. Integrating conflicting data: the role of source dependence. In Proceedings of the VLDB Endowment, 2(1):550-561, 2009.
5. X. L. Dong, B. Saha, and D. Srivastava. Less is more: Selecting sources wisely for integration. In Proceedings of the VLDB Endowment, 6(2):37-48, 2012.
6. A. Galland, S. Abiteboul, A. Marian, and P. Senellart. Corroborating information from disagreeing views. In Proceedings of the Third ACM International Conference on Web Search and Data Mining, pages 131-140, 2010.
7. T. L. Griffiths and M. Steyvers. Finding scientific topics. In Proceedings of the National Academy of Sciences of the United States of America, 101(Suppl 1):5228-5235, 2004.
8. H. Ji, R. Grishman, and H. T. Dang. Overview of the TAC 2011 knowledge base population track. In Third Text Analysis Conference, 2011.
9. J. Guo, S. Xu, S. Bao, and Y. Yu. Tapping on the potential of Q&A community by recommending answer providers. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, pages 921-930, 2008.
10. Q. Li, Y. Li, J. Gao, L. Su, B. Zhao, M. Demirbas, W. Fan, and J. Han. A confidence-aware approach for truth discovery on long-tail data. In Proceedings of the VLDB Endowment, 8(4):425-436, 2014.
11. Q. Li, Y. Li, J. Gao, B. Zhao, W. Fan, and J. Han. Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pages 1187-1198, 2014.
12. Y. Li, J. Gao, C. Meng, Q. Li, L. Su, B. Zhao, W. Fan, and J. Han. A Survey on Truth Discovery. In ArXiv Preprint ArXiv:1505.02463, 2015.
13. S. Mukherjee, G. Weikum, and C. Danescu-Niculescu-Mizil. People on drugs: Credibility of user statements in health communities. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 65-74, 2014.
14. J. Pasternack and D. Roth. Knowing what to believe (when you already know something). In Proceedings of the 23rd International Conference on Computational Linguistics, pages 877-885, 2010.
15. J. Pasternack and D. Roth. Latent credibility analysis. In Proceedings of the 22nd International Conference on World Wide Web, pages 1009-1020, 2013.
16. V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from crowds. In Journal of Machine Learning Research, 11:1297-1322, 2010.
17. R. Snow, B. O'Connor, D. Jurafsky, and A. Y. Ng. Cheap and fast-but is it good? Evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 254-263, 2008.
18. L. Su, Q. Li, S. Hu, S. Wang, J. Gao, H. Liu, T. F. Abdelzaher, J. Han, X. Liu, Y. Gao, and L. Kaplan. Generalized decision aggregation in distributed sensing systems. In 2014 IEEE Real-Time Systems Symposium (RTSS), pages 1-10, 2014.
19. M. Venanzi, J. Guiver, G. Kazai, P. Kohli, and M. Shokouhi. Community-based bayesian aggregation models for crowdsourcing. In Proceedings of the 23rd International Conference on World Wide Web, pages 155-164, 2014.
20. V. Vydiswaran, C. Zhai, and D. Roth. Content-driven trust propagation framework. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 974-982, 2011.
21. H. M. Wallach. Topic modeling: Beyond bag-of-words. In Proceedings of the 23rd International Conference on Machine Learning, pages 977-984, 2006.
22. D. Wang, L. Kaplan, H. Le, and T. Abdelzaher. On truth discovery in social sensing: A maximum likelihood estimation approach. In Proceedings of the 11th International Conference on Information Processing in Sensor Networks, pages 233-244, 2012.
23. P. Welinder, S. Branson, P. Perona, and S. J. Belongie. The multidimensional wisdom of crowds. In Advances in Neural Information Processing Systems, pages 2424-2432, 2010.
24. J. Whitehill, T.-f. Wu, J. Bergsma, J. R. Movellan, and P. L. Ruvolo. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In Advances in Neural Information Processing Systems, pages 2035-2043, 2009.
25. L. Yang, M. Qiu, S. Gottipati, F. Zhu, J. Jiang, H. Sun, and Z. Chen. CQARank: jointly model topics and expertise in community question answering. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, pages 99-108, 2013.
26. X. Yin, J. Han, and P. S. Yu. Truth discovery with multiple conflicting information providers on the web. In IEEE Transactions on Knowledge and Data Engineering, 20(6):796-808, 2008.
27. B. Zhao and J. Han. A probabilistic model for estimating real-valued truth from conflicting sources. In Proceedings of the 10th International Workshop on Quality in Databases, 2012.
28. B. Zhao, B. I. Rubinstein, J. Gemmell, and J. Han. A bayesian approach to discovering truth from conflicting sources for data integration. In Proceedings of the VLDB Endowment, 5(6):550-561, 2012.
29. T. Zhao, N. Bian, C. Li, and M. Li. Topic-level expert modeling in community question answering. In SIAM International Conference on Data Mining, pages 776-784, 2013.
30. W. X. Zhao, J. Jiang, J. Weng, J. He, E.-P. Lim, H. Yan, and X. Li. Comparing twitter and traditional media using topic models. In Proceedings of the 33rd European Conference on Advances in Information Retrieval, pages 338-349, 2011.
31. D. Zhou, S. Basu, Y. Mao, and J. C. Platt. Learning from the wisdom of crowds by minimax entropy. In Advances in Neural Information Processing Systems, pages 2195-2203, 2012.