Top-k and clustering with noisy comparisons

ACM Transactions on Database Systems

Volumn 39, Issue 4, 2014, Pages

  Davidson, Susan ^a   Khanna, Sanjeev ^a   Milo, Tova ^b   Roy, Sudeepa ^c  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b TEL AVIV UNIVERSITY   (Israel)

^c University of Washington   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CROWDSOURCING;

CLUSTERING QUERIES; CONCAVE FUNCTION; DATA ELEMENTS; ERROR MODEL; FIXED COST; HIGH PROBABILITY; SIMPLIFYING ASSUMPTIONS;

COST FUNCTIONS;

EID: 84920879928     PISSN: 03625915     EISSN: 15574644     Source Type: Journal    
DOI: 10.1145/2684066     Document Type: Conference Paper

References (44)

1. th International Colloquium on Automata, Languages, and Programming (ICALP'09). 37-48.
2. Paul André, Michael Bernstein, and Kurt Luther. 2012. Who gives a tweet? Evaluating microblog content value. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'12). ACM Press, New York, 471-474.
3. th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW'14). 989-998.
4. Paul Andre, Haoqi Zhang, Juho Kim, Lydia B. Chilton, Steven P. Dow, and Robert C. Miller. 2013. Community clustering: Leveraging an academic crowd to form coherent conference sessions. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP'13).
5. Eyal Baharad, Jacob Goldberger, Moshe Koppel, and Shmuel Nitzan. 2011. Distilling the wisdom of crowds: Weighted aggregation of decisions on multiple issues. Auton. Agents Multi-Agent Syst. 22, 1, 31-42.
6. th Annual ACM Symposium on Theory of Computing (STOC'83). 80-86.
7. Manuel Blum, Robert W. Floyd, Vaughan Pratt, Ronald L. Rivest, and Robert E. Tarjan. 1973. Time bounds for selection. J. Comput. Syst. Sci. 7, 4, 448-461.
8. th IEEE International Conference on Data Engineering (ICDE'12). 1261-1264.
9. nd International Conference on Machine Learning (ICML'05). 89-96.
10. rd Ed. The MIT Press.
11. th International Conference on Database Theory (ICDT'13). 225-236.
12. Alexander Philip Dawid and Allan M. Skene. 1979. Maximum likelihood estimation of observer error-rates using the em algorithm. J. Royal Statist. Soc. Appl. Statist. 28, 1, 20-28.
13. Uriel Feige, Prabhakar Raghavan, David Peleg, and Eli Upfal. 1994. Computing with noisy information. SIAM J. Comput. 23, 5, 1001-1018.
14. Alberto Fernández and Sergio Gómez. 2008. Solving non-uniqueness inagglomerative hierarchical clustering using multidendrograms. J. Classificat. 25, 1, 43-65.
15. th ACM Conference on Electronic Commerce (EC'13). 435-452.
16. Michael J. Franklin, Donald Kossmann, Tim Kraska, Sukriti Ramesh, and Reynold Xin. 2011. CrowdDB: Answering queries with crowdsourcing. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'11). 61-72.
17. Ryan Gomes, Peter Welinder, Andreas Krause, and Pietro Perona. 2011. Crowdclustering. In Proceedings of the Neural Information Processing Systems Conference (NIPS'11). 558-566.
18. Geoffrey M. Guisewite and Panagote M. Pardalos. 1991. Algorithms for the single-source uncapacitated minimum concave-cost network flow problem. J. Global Optim. 1, 3, 245-265.
19. Stephen Guo, Aditya Parameswaran, and Hector Garcia-Molina. 2012. So who won? Dynamic max discovery with the crowd. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'12). 385-396.
20. Hannes Heikinheimo and Antti Ukkonen. 2013. The crowd-median algorithm. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP'13).
21. th International Workshop on Information Integration on the Web in Conjunction with the Conference on World Wide Web (IIWeb/WWW'11).
22. David R. Karger, Sewoong Oh, and Devavrat Shah. 2011. Iterative learning for reliable crowdsourcing systems. In Proceedings of the Neural Information Processing Systems Conference (NIPS'11).1953-1961.
23. rd Human Computation Workshop (HCOMP'11). AAAI, 97-102.
24. Fei-Fei Li and Pietro Perona. 2005. A bayesian hierarchical model for learning natural scene categories. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). Vol. 2. 524-531.
25. Hongwei Li, Bin Yu, and Dengyong Zhou. 2013. Error rate bounds in crowdsourcing models. http://arxiv.org/pdf/1307.2674.pdf.
26. Tie-Yan Liu. 2009. Learning to rank for information retrieval. Found. Trends Inf. Retr. 3, 3, 225-331.
27. Xuan Liu, Meiyu Lu, Beng Chin Ooi, Yanyan Shen, Sai Wu, and Meihui Zhang. 2012. CDAS: A crowdsourcing data analytics system. Proc. VLDB Endow. 5, 10, 1040-1051.
28. Adam Marcus, Michael S. Bernstein, Osama Badar, David R. Karger, Samuel Madden, and Robert C. Miller. 2011a. Twitinfo: Aggregating and visualizing microblogs for event exploration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'11). 227-236.
29. Adam Marcus, Eugene Wu, David R. Karger, Samuel Madden, and Robert C. Miller. 2011b. Human-powered sorts and joins. Proc. VLDB Endow. 5, 1, 13-24.
30. Rajeev Motwani and Prabhakar Raghavan. 1995. Randomized Algorithms. Cambridge University Press, New York.
31. Aditya Parameswaran, Hyunjung Park, Hector Garcia-Molina, Neoklis Polyzotis, and Jennifer Widom. 2011. Deco: Declarative crowdsourcing. http://ilpubs.stanford.edu:8090/1015/.
32. Aditya G. Parameswaran, Hector Garcia-Molina, Hyunjung Park, Neoklis Polyzotis, Aditya Ramesh, and Jennifer Widom. 2012. CrowdScreen: Algorithms for filtering data with humans. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'12). 361-372.
33. Nicholas Pippenger. 1987. Sorting and selecting in rounds. SIAM J. Comput. 16, 6, 1032-1038.
34. Vassilis Polychronopoulos, Luca De Alfaro, James Davis, Hector Garcia-Molina, and Neoklis Polyzotis. 2013. Human-powered top-k lists. In Proceedings of the International Workshop on the Web and Databases (WebDB'13). 25-30.
35. th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'07). ACM Press, New York, 570-579.
36. Cyrus Rashtchian, Peter Young, Micah Hodosh, and Julia Hockenmaier. 2010. Collecting image annotations using amazon's mechanical turk. In Proceedings of the NAACL HLT Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk (CSLDAMT'10). Association for Computational Linguistics, 139-147.
37. Mohammad Rastegari, Chen Fang, and Lorenzo Torresani. 2011. Scalable object-class retrieval with approximate and top-k ranking. In Proceedings of the International Conference on Computer Vision (ICCV'11). 2659-2666.
38. Vikas C. Raykar, Shipeng Yu, Linda H. Zhao, Gerardo Hermosillo Valadez, Charles Florin, Luca Bogoni, and Linda Moy. 2010. Learning from crowds. J. Mach. Learn. Res. 11, 1297-1322.
39. Joachim Selke, Christoph Lofi, and Wolf-Tilo Balke. 2012. Pushing the boundaries of crowd-enabled databases with query-driven schema expansion. Proc. VLDB Endow. 5, 6, 538-549.
40. Leslie G. Valiant. 1975. Parallelism in comparison problems. SIAM J. Comput. 4, 3, 348-355.
41. st International Conference on World Wide Web (WWW'12). 989-998.
42. Jiannan Wang, Tim Kraska, Michael J. Franklin, and Jianhua Feng. 2012. CrowdER: Crowdsourcing entity resolution. Proc. VLDB Endow. 5, 11, 1483-1494.
43. Steven Euijong Whang, Peter Lofgren, and Hector Garcia-Molina. 2013. Question selection for crowd entity resolution. Proc. VLDB Endow. 6, 6, 349-360.
44. Jinfeng Yi, Rong Jin, Anil K. Jain, Shaili Jain, and Tianbao Yang. 2012. Semi-crowdsourced clustering: Generalizing crowd labeling by robust distance metric learning. In Proceedings of the Neural Information Processing Systems Conference (NIPS'12). 1781-1789.