Crowdsourced clustering: Querying edges vs triangles

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 1324-1332

  Vinayak, Ramya Korlakai ^a   Hassibi, Babak ^a  

^a California Institute of Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BUDGET CONTROL; CONVEX OPTIMIZATION;

ADJACENCY MATRICES; AMAZON MECHANICAL TURKS; BUDGET CONSTRAINT; EDGE DENSITIES; EXACT RECOVERIES; GENERATIVE MODEL; PARTIAL OBSERVATION; REAL DATA SETS;

CLUSTERING ALGORITHMS;

EID: 85019171270     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Vikas C. Raykar, Shipeng Yu, Linda H. Zhao, Gerardo Hermosillo Valadez, Charles Florin, Luca Bogoni, and Linda Moy. Learning from crowds. J. Mach. Learn. Res., 11:1297-1322, August 2010.
2. Rion Snow, Brendan O'Connor, Daniel Jurafsky, and Andrew Y. Ng. Cheap and fast-but is it good?: Evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '08, pages 254-263, 2008.
3. Luis Von Ahn, Benjamin Maurer, Colin McMillen, David Abraham, and Manuel Blum. reCAPTCHA: Human-based character recognition via web security measures. Science, 321(5895):1465-1468, 2008.
4. A. Sorokin and D. Forsyth. Utility data annotation with Amazon Mechanical Turk. In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW '08. IEEE Computer Society Conference on, pages 1-8. IEEE, June 2008.
5. Peter Welinder, Steve Branson, Serge Belongie, and Pietro Perona. The multidimensional wisdom of crowds. In Neural Information Processing Systems Conference (NIPS), 2010.
6. Jinfeng Yi, Rong Jin, Anil K. Jain, Shaili Jain, and Tianbao Yang. Semi-crowdsourced clustering: Generalizing crowd labeling by robust distance metric learning. In Neural Information Processing Systems Conference (NIPS), 2012.
7. Robert Simpson, Kevin R. Page, and David De Roure. Zooniverse: Observing the world's largest citizen science platform. In Proceedings of the 23rd International Conference on World Wide Web, WWW '14 Companion, 2014.
8. Chris Lintott, Megan E. Schwamb, Charlie Sharzer, Debra A. Fischer, Thomas Barclay, Michael Parrish, Natalie Batalha, Steve Bryson, Jon Jenkins, Darin Ragozzine, Jason F. Rowe, Kevin Schawinski, Rovert Gagliano, Joe Gilardi, Kian J. Jek, Jari-Pekka Pääkkönen, and Tjapko Smits. Planet hunters: New kepler planet candidates from analysis of quarter 2, 2012. cite arxiv:1202.6007Comment: Submitted to AJ.
9. David R. Karger, Sewoong Oh, and Devavrat Shah. Iterative learning for reliable crowdsourcing systems. In Neural Information Processing Systems Conference (NIPS), 2011.
10. David R. Karger, Sewoong Oh, and Devavrat Shah. Budget-optimal task allocation for reliable crowdsourcing systems. Operations Research, 62(1):1-24, 2014.
11. Aditya Vempaty, Lav R. Varshney, and Pramod K. Varshney. Reliable crowdsourcing for multi-class labeling using coding theory. CoRR, abs/1309.3330, 2013.
12. Denny Zhou, Sumit Basu, Yi Mao, and John C. Platt. Learning from the wisdom of crowds by minimax entropy. In Advances in Neural Information Processing Systems 25, pages 2195-2203. 2012.
13. Qiang Liu, Jian Peng, and Alexander T Ihler. Variational inference for crowdsourcing. In Neural Information Processing Systems Conference (NIPS). 2012.
14. Yuchen Zhang, Xi Chen, Dengyong Zhou, and Michael I. Jordan. Spectral methods meet EM: A provably optimal algorithm for crowdsourcing. In Neural Information Processing Systems Conference (NIPS), 2014.
15. Ryan G. Gomes, Peter Welinder, Andreas Krause, and Pietro Perona. Crowdclustering. In Advances in Neural Information Processing Systems 24, pages 558-566. 2011.
16. Ramya Korlakai Vinayak, Samet Oymak, and Babak Hassibi. Graph clustering with missing data: Convex algorithms and analysis. In Neural Information Processing Systems Conference (NIPS), 2014.
17. Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, and Adam Tauman Kalai. Adaptively learning the crowd kernel. CoRR, abs/1105.1033, 2011.
18. Michael Wilber, Sam Kwak, and Serge Belongie. Cost-effective hits for relative similarity comparisons. In Human Computation and Crowdsourcing (HCOMP), Pittsburgh, November 2014.
19. Eric Heim, Hamed Valizadegan, and Milos Hauskrecht. Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2014, chapter Relative Comparison Kernel Learning with Auxiliary Kernels, pages 563-578. Springer Berlin Heidelberg.
20. L. van der Maaten and K. Weinberger. Stochastic triplet embedding. In Machine Learning for Signal Processing (MLSP), 2012 IEEE International Workshop on, pages 1-6, Sept 2012.
21. Catherine Wah, Grant Van Horn, Steve Branson, Subhransu Maji, Pietro Perona, and Serge Belongie. Similarity comparisons for interactive fine-grained categorization. In CVPR, pages 859-866. IEEE, 2014.
22. Hannes Heikinheimo and Antti Ukkonen. The crowd-median algorithm. In HCOMP. AAAI, 2013.
23. A. P. Dawid and A. M. Skene. Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1):20-28, 1979.
24. Paul W. Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social Networks, 5(2):109-137, 1983.
25. Anne Condon and Richard M. Karp. Algorithms for graph partitioning on the planted partition model. Random Struct. Algorithms, 18(2):116-140, 2001.
26. Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. In ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS, pages 849-856. MIT Press, 2001.
27. Marina Meila. Comparing clusterings-an information based distance. J. Multivar. Anal., 98(5):873-895, May 2007.
28. Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, June 2011.
29. C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011.