A probabilistic framework to learn from multiple annotators with time-varying accuracy

Proceedings of the 10th SIAM International Conference on Data Mining, SDM 2010

Volumn , Issue , 2010, Pages 826-837

  Donmez, Pinar ^a   Carbonell, Jaime ^a   Schneider, Jeff ^b  

^a Carnegie Mellon University ^*  (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN NETWORKS;

EMPIRICAL ANALYSIS; INCREMENTAL LEARNING; LABELING ACCURACIES; PARTICLE FILTERING METHODS; PROBABILISTIC FRAMEWORK; SEQUENTIAL BAYESIAN ESTIMATION; TIME VARYING RELIABILITY; WEIGHTED SAMPLES;

DATA MINING;

EID: 80055061019     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1137/1.9781611972801.72     Document Type: Conference Paper

References (12)

1. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for on-line non-linear/non-gaussian bayesian tracking. IEEE Transactions on Signal Processing, 50:174-188, 2001.
2. Blake and C. J. Merz. UCI repository of machine learning databases, 1998.
3. J. Carpenter, P. Clifford, and P. Fearnhead. Improved particle filter for non-linear problems. In IEEE Proceedings on Radar and Sonar Navigation, 1999.
4. O. Dekel and O. Shamir. Good learners for evil teachers. In Proceedings of the 26th International Conference on Machine Learning, June 2009.
5. P. Donmez, J. G. Carbonell, and J. Schneider. Efficiently learning the accuracy of labeling sources for selective sampling. In Proceedings of the 15th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '09), pages 259-268, 2009.
6. P. Donmez, G. Lebanon, and K. Balasubramanian. Unsupervised estimation of classification and regression error rates. Technical Report CMU-LTI-09-15, Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA.
7. A. Doucet and D. Crisan. A survey of convergence results on particle filtering for practitioners. IEEE Transactions on Signal Processing, 2002.
8. T. Pham, M. Worring, and A. Smeulders. Face detection by aggregated bayesian network classifiers. Pattern Recognition Letters, 23.
9. V. C. Raykar, S. Yu, L. Zhao, A. Jerebko, C. Florin, G. Valadez, L. Bogoni, and L. Moy. Supervised learning from multiple experts: Whom to trust when everyone lies a bit. In Proceedings of the 26th International Conference on Machine Learning, pages 889-896, June 2009.
10. B. Ripley. Stochastic Simulation. Wiley, New York, 1987.
11. V. Sheng, F. Provost, and P. G. Ipeirotis. Get another label? improving data quality and data mining using multiple, noisy labelers. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '08), pages 614-622, 2008.
12. R. Snow, O'Connor, D. Jurafsky, and A. Ng. Cheap and fast-but is it good? evaluating non-expert annotations for natural language tasks. In Proc. of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2008.