CEKA: A tool for mining the wisdom of crowds

Journal of Machine Learning Research

Volumn 16, Issue , 2015, Pages 2853-2858

  Zhang, Jing ^a,b   Sheng, Victor S ^c   Nicholson, Bryce A ^c   Wu, Xindong ^a,d  

^a HEFEI UNIVERSITY OF TECHNOLOGY   (China)

^b NANJING UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

^c UNIVERSITY OF CENTRAL ARKANSAS   (United States)

^d UNIVERSITY OF VERMONT   (United States)

Author keywords

Crowdsourcing; Inference; Learning from crowds; Multiple noisy labeling; Noise handling; Repeated labeling simulation

Indexed keywords

ARTIFICIAL INTELLIGENCE; CROWDSOURCING; FUNCTION EVALUATION; LEARNING SYSTEMS;

INFERENCE; INFERENCE ALGORITHM; LEARNING FROM CROWDS; LEARNING MODELS; NOISE HANDLING; REPEATED LABELING SIMULATION; STATE OF THE ART; WISDOM OF CROWDS;

INFERENCE ENGINES;

EID: 84961574739     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (15)

1. Carla E Brodley and Mark A Friedl. Identifying mislabeled training data. Journal of Artificial Intelligence Research, 11:131-161, 1999.
2. Alexander Philip Dawid and Allan M Skene. Maximum likelihood estimation of observer error-rates using the em algorithm. Applied Statistics, pages 20-28, 1979.
3. Gianluca Demartini, Djellel Eddine Difallah, and Philippe Cudré-Mauroux. Zencrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In World Wide Web, pages 469-478. ACM, 2012.
4. Dragan Gamberger, Nada Lavrac, and Ciril Groselj. Experiments with noise filtering in a medical domain. In ICML, pages 143-151, 1999.
5. Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. The weka data mining software: an update. ACM SIGKDD Explorations Newsletter, 11(1):10-18, 2009.
6. Jeff Howe. The rise of crowdsourcing. Wired Magazine, 14(6):1-4, 2006.
7. David R Karger, Sewoong Oh, and Devavrat Shah. Iterative learning for reliable crowdsourcing systems. In NIPS, pages 1953-1961, 2011.
8. Taghi M Khoshgoftaar and Pierre Rebours. Improving software quality prediction by noise filtering techniques. Journal of Computer Science and Technology, 22(3):387-396, 2007.
9. Quoc Viet Hung Nguyen, Thanh Tam Nguyen, Ngoc Tran Lam, and Karl Aberer. Batc: a benchmark for aggregation techniques in crowdsourcing. In ACM SIGIR, pages 1079-1080. ACM, 2013.
10. Vikas C Raykar, Shipeng Yu, Linda H Zhao, Gerardo Hermosillo Valadez, Charles Florin, Luca Bogoni, and Linda Moy. Learning from crowds. Journal of Machine Learning Research, 11:1297-1322, 2010.
11. Aashish Sheshadri and Matthew Lease. Square:. In The First AAAI Conference on Human Computation and Crowdsourcing, pages 156-164, 2013.
12. Choh-Man Teng. Correcting noisy data. In ICML, pages 239-248, 1999.
13. Isaac Triguero, José A Sáez, Julián Luengo, Salvador García, and Francisco Herrera. On the characterization of noise filters for self-training semi-supervised in nearest neighbor classification. Neurocomputing, 132:30-41, 2014.
14. Jacob Whitehill, Ting-fan Wu, Jacob Bergsma, Javier R Movellan, and Paul L Ruvolo. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In NIPS, pages 2035-2043, 2009.
15. Jing Zhang, Xindong Wu, and Victor S Sheng. Imbalanced multiple noisy labeling. IEEE Transaction on Kownledge and Data Engineering, 27(2):489-503, 2015.