An evaluation of aggregation techniques in crowdsourcing

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 8181 LNCS, Issue PART 2, 2013, Pages 1-15

  Quoc Viet Hung, Nguyen ^a   Tam, Nguyen Thanh ^a   Tran, Lam Ngoc ^a   Aberer, Karl ^a  

^a EPFL   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

AGGREGATION TECHNIQUES; COMPUTATION TIME; EMPIRICAL STUDIES; IN-DEPTH ANALYSIS; PERFORMANCE COMPARISON; PERFORMANCE METRICS; PRACTICAL GUIDELINES; STATE-OF-THE-ART METHODS;

SYSTEMS ENGINEERING; WORLD WIDE WEB;

BENCHMARKING;

EID: 84887447348     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-41154-0_1     Document Type: Conference Paper

References (26)

1. von Ahn, L., et al.: Labeling images with a computer game. In: CHI (2004)
2. von Ahn, L., et al.: recaptcha: Human-based character recognition via web security measures. Science (2008)
3. Hung, N.Q.V., Tam, N.T., Miklós, Z., Aberer, K.: On leveraging crowdsourcing techniques for schema matching networks. In: Meng, W., Feng, L., Bressan, S., Winiwarter, W., Song, W. (eds.) DASFAA 2013, Part II. LNCS, vol. DASFAA, pp. 139-154. Springer, Heidelberg (2013)
4. Difallah, D.E., et al.: Mechanical cheat: Spamming schemes and adversarial techniques on crowdsourcing platforms. In: CrowdSearch (2012)
5. Doan, A., et al.: Crowdsourcing systems on the world-wide web. CACM (2011)
6. Hosmer, D.W., et al.: Applied logistic regression. Wiley-Interscience Publication (2000)
7. Ipeirotis, P.G., et al.: Quality management on amazon mechanical turk. In: HCOMP (2010)
8. Kamar, E., et al.: Combining human and machine intelligence in large-scale crowdsourcing. In: AAMAS (2012)
9. Kamar, E., et al.: Incentives for truthful reporting in crowdsourcing. In: AAMAS (2012)
10. Karger, D., et al.: Iterative learning for reliable crowdsourcing systems. In: NIPS (2011)
11. Nguyen, Q.V.H., et al.: Batc - a benchmark for aggregation techniques in crowdsourcing. In: SIGIR (2013)
12. Kazai, G., et al.: Worker types and personality traits in crowdsourcing relevance labels. In: CIKM (2011)
13. Khattak, F., et al.: Quality Control of Crowd Labeling through Expert Evaluation. In: NIPS (2011)
14. Nguyen, Q.V.H., et al.: Collaborative schema matching reconciliation. In: CoopIS (2013)
15. Kuncheva, L., et al.: Limits on the majority vote accuracy in classifier fusion. Pattern Anal. Appl. (2003)
16. Law, E., et al.: Human Computation. Morgan & Claypool Publishers (2011)
17. Lee, K., et al.: The social honeypot project: protecting online communities from spammers. In: WWW (2010)
18. Mason, W., et al.: Conducting behavioral research on amazon mechanical turk. BRM (2012)
19. Quinn, A.J., et al.: Human computation: a survey and taxonomy of a growing field. In: CHI (2011)
20. Nguyen, Q.V.H., et al.: Minimizing Human Effort in Reconciling Match Networks. In: ER (2013)
21. Raykar, V., et al.: Supervised learning from multiple experts: Whom to trust when everyone lies a bit. In: ICML (2009)
22. Raykar, V.C., et al.: Learning from crowds. Mach. Learn. Res. (2010)
23. Ross, J., et al.: Who are the crowdworkers?: shifting demographics in mechanical turk. In: CHI (2010)
24. Vuurens, J., et al.: How much spam can you take? an analysis of crowdsourcing results to increase accuracy. In: CIR (2011)
25. Whitehill, J., et al.:Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In: NIPS (2009)
26. Yan, T., et al.: CrowdSearch: exploiting crowds for accurate real-time image search on mobile phones. In: MobiSys (2010)