Effective result inference for context-sensitive tasks in crowdsourcing

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 9642, Issue , 2016, Pages 33-48

  Fang, Yili ^a   Sun, Hailong ^a   Li, Guoliang ^b   Zhang, Richong ^a   Huai, Jinpeng ^a  

^a BEIHANG UNIVERSITY   (China)

^b TSINGHUA UNIVERSITY   (China)

Author keywords

Context sensitive tasks; Crowdsourcing; Result inference; Smart assembly model

Indexed keywords

CHARACTER RECOGNITION; CROWDSOURCING; DATABASE SYSTEMS; MARKOV PROCESSES; TURPENTINE;

ASSEMBLY MODEL; CONTEXT INFORMATION; CONTEXT SENSITIVE TASKS; DECISION MODELING; HANDWRITING RECOGNITION; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; RESULT INFERENCE; STATE-OF-THE-ART APPROACH;

ITERATIVE METHODS;

EID: 84962339162     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-319-32025-0_3     Document Type: Conference Paper

References (20)

1. Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K.: Soylent: a word processor with a crowd inside. In: UIST, pp. 313-322. ACM (2010)
2. Boim, R., Greenshpan, O., Milo, T., Novgorodov, S., Polyzotis, N., Tan, W.C.: Asking the right questions in crowd data sourcing. In: ICDE (2012)
3. Dai, P., Lin, C.H., Weld, D.S., et al.: Pomdp-based control of workflows for crowdsourcing. Artif. Intell. 202, 52-85 (2013)
4. Dredze, M., Talukdar, P.P., Crammer, K.: Sequence learning from data with multiple labels. In: Workshop Co-Chairs, p. 39 (2009)
5. Feng, J., Li, G., Wang, H., Feng, J.: Incremental quality inference in crowdsourcing. In: Bhowmick, S.S., Dyreson, C.E., Jensen, C.S., Lee, M.L., Muliantara, A., Thalheim, B. (eds.) DASFAA 2014, Part II. LNCS, vol. 8422, pp. 453-467. Springer, Heidelberg (2014)
6. Ipeirotis, P.G., Gabrilovich, E.: Quizz: targeted crowdsourcing with a billion (potential) users. In: WWW, pp. 143-154 (2014)
7. Law, E., Ahn, L.V.: Human computation. Synth. Lect. Artif. Intell. Mach. Learn. 5(3), 21-22 (2011)
8. Law, E., Zhang, H.: Towards large-scale collaborative planning: answering highlevel search queries using human computation. In: AAAI (2011)
9. Lee, K., Tamilarasan, P., Caverlee, J.: Crowdturfers, campaigns, and social media: tracking and revealing crowdsourced manipulation of social media. In: ICWSM (2013)
10. Liu, Q., Peng, J., Ihler, A.T.: Variational inference for crowdsourcing. In: NIPS, pp. 692-700. Curran Associates, Inc. (2012)
11. Liu, X., Lu, M., Ooi, B.C., Shen, Y., Wu, S., Zhang, M.: Cdas: a crowdsourcing data analytics system. PVLDB 5(10), 1040-1051 (2012)
12. Rodrigues, F., Pereira, F., Ribeiro, B.: Sequence labeling with multiple annotators. Mach. Learn. 95(2), 165-181 (2014)
13. Salek, M., Bachrach, Y., Key, P.: Hotspotting-a probabilistic graphical model for image object localization through crowdsourcing. In: AAAI (2013)
14. Sheng, V.S., Provost, F., Ipeirotis, P.G.: Get another label? improving data quality and data mining using multiple, noisy labelers. In: SIGKDD. ACM (2008)
15. Sheshadri, A., Lease, M.: Square: a benchmark for research on computing crowd consensus. In: HCOMP (2013)
16. Snow, R., O’Connor, B., Jurafsky, D., Ng, A.Y.: Cheap and fast-but is it good?: evaluating non-expert annotations for natural language tasks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (2008)
17. Tran-Thanh, L., Huynh, T.D., Rosenfeld, A., Ramchurn, S.D., Jennings, N.R.: Crowdsourcing complex workflows under budget constraints. In: AAAI (2015)
18. Von Ahn, L., Maurer, B., McMillen, C., Abraham, D., Blum, M.: reCAPTCHA: human-based character recognition via web security measures. Science 321(5895), 1465-1468 (2008)
19. Young, H.P.: Condorcet’s theory of voting. Mathèmatiques et Sci. Humaines 111, 45-59 (1990)
20. Zheng, Y., Wang, J., Li, G., Cheng, R., Feng, J.: Qasca: a quality-aware task assignment system for crowdsourcing applications. In: SIGMOD (2015)