Active visual recognition with expertise estimation in crowdsourcing

Proceedings of the IEEE International Conference on Computer Vision

Volumn , Issue , 2013, Pages 3000-3007

  Long, Chengjiang ^a   Hua, Gang ^a   Kapoor, Ashish ^b  

^a STEVENS INSTITUTE OF TECHNOLOGY   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Active Learning; Crowdsourcing; Expectation Propagation; Gaussian Processes; Visual Recognition

Indexed keywords

BAYESIAN NETWORKS; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); INFERENCE ENGINES;

ACTIVE LEARNING; CROWDSOURCING; EXPECTATION PROPAGATION; GAUSSIAN PROCESSES; VISUAL RECOGNITION;

OBJECT RECOGNITION;

EID: 84898792886     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2013.373     Document Type: Conference Paper

References (30)

1. V. Ambati, S. Vogel, and J. Carbonell. Active learning and crowdsourcing for machine translation. In LREC, 2010.
2. R. U. Ashish Kapoor, Kristen Grauman and T. Darrell. Active learning with gaussian processes for object categorization. ICCV, 2007.
3. S. Chen, J. Zhang, G. Chen, and C. Zhang. What if the irresponsible teachers are dominating? a method of training on samples and clustering on teachers. In AAAI, 2010.
4. O. Dekel and O. Shamir. Good learners for evil teachers. In ICML, 2009.
5. O. Dekel and O. Shamir. Vox populi: Collecting high-quality labels from a crowd. In COLT, 2009.
6. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, June 2009.
7. P. Donmez, J. Carbonell, and J. Schneider. A probabilistic framework to learn from multiple annotators with time-varying accuracy. In SDM, 2010.
8. P. Donmez, J. G. Carbonell, and J. Schneider. Efficiently learning the accuracy of labeling sources for selective sampling. In SIGKDD, 2009.
9. S. Ebert, M. Fritz, and B. Schiele. Ralf: A reinforced active learning formulation for object class recognition. In CVPR, 2012.
10. M. Gibbs and D. Mackay. Variational gaussian process classifiers. T-NN, 2000.
11. A. Kapoor, G. Hua, A. Akbarzadeh, and S. Baker. Which faces to tag: Adding prior constraints into active learning. ICCV, 2009.
12. H.-C. Kim and Z. Ghahramani. Outlier robust gaussian process classification. In SSPR/SPR, pages 896-905, 2008.
13. A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
14. Y. Lin, F. Lv, S. Zhu, M. Yang, T. Cour, K. Yu, L. Cao, and T. Huang. Large-scale image classification: Fast feature extraction and svm training. In CVPR, 2011.
15. C. Loy, T. Hospedales, T. Xiang, and S. Gong. Stream-based joint exploration-exploitation active learning. In CVPR, 2012.
16. T. Minka. A family of algorithms for approximate Bayesian inference. Ph.d. thesis, MIT, 2001.
17. R. M. Neal. Monte carlo implementation of gaussian process models for bayesian regression and classification. Technical Report CRGTR972, University of Toronto, 1997.
18. M. Opper and O. Winther. Gaussian processes for classification: Mean-field algorithms. NC, 2000.
19. V. C. Raykar and S. Yu. Eliminating spammers and ranking annotators for crowdsourced labeling tasks. JMLR, 2012.
20. V. C. Raykar, S. Yu, L. H. Zhao, A. Jerebko, C. Florin, G. H. Valadez, L. Bogoni, and L. Moy. Supervised learning from multiple experts: whom to trust when everyone lies a bit. In ICML, 2009.
21. J. Sanchez and F. Perronnin. High-dimensional signature compression for large-scale image classification. In CVPR, 2011.
22. E. H. Spriggs, F. D. L. Torre, and M. Hebert. Temporal segmentation and activity classification from first-person sensing. In CVPRW, 2009.
23. S. Vijayanarasimhan and K. Grauman. Large-scale live active learning: Training object detectors with crawled data and crowds. In CVPR, 2011.
24. P. Welinder, S. Branson, S. Belongie, and P. Perona. The multidimensional wisdom of crowds. In NIPS, 2010.
25. P. Welinder and P. Perona. Online crowdsourcing: Rating annotators and obtaining cost-effective labels. In CVPRW, 2010.
26. C. Williams and D. Barber. Bayesian classification with gaussian processes. T-PAMI, 20(12):1342-1351, dec 1998.
27. Y. Yan, R. Rosales, G. Fung, and J. Dy. Active learning from multiple knowledge sources. In AISTATS, 2012.
28. Y. Yan, R. Rosales, G. Fung, and J. G. Dy. Active learning from crowds. ICML, pages 1161-1168, 2011.
29. L. Zhao, G. Sukthankar, and R. Sukthankar. Incremental relabeling for active learning with noisy crowdsourced annotations. In Social-Com, 2011.
30. C. Zhu, R. H. Byrd, P. Lu, and J. Nocedal. Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization. ACM Trans. Math. Softw., 1997.