Towards anytime active learning: Interrupting experts to reduce annotation costs

Proceedings of the ACM SIGKDD 2013 Workshop on Interactive Data Exploration and Analytics, IDEA 2013

Volumn , Issue , 2013, Pages 87-94

  Ramirez Loaiza, Maria E ^a   Culotta, Aron ^a   Bilgic, Mustafa ^a  

^a ILLINOIS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Active learning; Anytime algorithms; Empirical evaluation; Value of information

Indexed keywords

ARTIFICIAL INTELLIGENCE; BUDGET CONTROL; CLASSIFICATION (OF INFORMATION); DATA MINING; INFORMATION RETRIEVAL SYSTEMS; LEARNING SYSTEMS; QUALITY CONTROL;

ACTIVE LEARNING; ACTIVE LEARNING METHODS; ANYTIME ALGORITHM; DECISION-THEORETIC; EMPIRICAL EVALUATIONS; IMPROVE PERFORMANCE; METHOD MODEL; PERFORMANCE EFFICIENCY; TIME BUDGET; VALUE OF INFORMATION;

COSTS;

EID: 84887449698     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2501511.2501524     Document Type: Conference Paper

References (15)

1. M. Bilgic and L. Getoor. Value of information lattice: Exploiting probabilistic independence for effective feature subset acquisition. Journal of Artificial Intelligence Research (JAIR), 41:69-95, 2011.
2. D. Cohn, L. Atlas, and R. Ladner. Improving generalization with active learning. Machine Learning, 15(2):201-221, 1994.
3. P. Donmez and J. G. Carbonell. Proactive learning: : Costsensitive active learning with multiple imperfect oracles. In Proceeding of the 17th ACM conference on Information and knowledge mining CIKM '08, page 619, oct 2008.
4. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9:1871-1874, 2008.
5. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The WEKA data mining software: An update. SIGKDD Explorations, 11, 2009.
6. R. A. Howard. Information value theory. IEEE Transactions on Systems Science and Cybernetics, 2(1):22-26, 1966.
7. A. Kapoor, E. Horvitz, and S. Basu. Selective supervision: Guiding supervised learning with decision-theoretic active learning. International Joint Conference on Artificial Intelligence (IJCAI), 2007.
8. D. D. Lewis, Y. Yang, T. G. Rose, and F. Li. Rcv1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361-397, Dec. 2004.
9. A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.
10. P. Melville, F. Provost, M. Saar-Tsechansky, and R. Mooney. Economical active feature-value acquisition through expected utility estimation. In Proc. of the KDD Workshop on Utilitybased Data Mining, 2005.
11. K. Nigam, A. McCallum, S. Thrun, and T. Mitchell. Learning to classify text from labeled and unlabeled documents. In Proceedings of the National Conference on Artificial Intelligence, pages 792-799, 1998.
12. N. Roy and A. McCallum. Toward optimal active learning through sampling estimation of error reduction. In International Conference on Machine Learning, pages 441-448, 2001.
13. B. Settles, M. Craven, and S. Ray. Multiple-instance active learning. In Neural Information Processing Systems, pages 1289-1296, 2008.
14. B. C. Wallace, K. Small, C. E. Brodley, and T. A. Trikalinos. Who should label what? instance allocation in multiple expert active learning. In Proc. of the SIAM International Conference on Data Mining (SDM), 2011.
15. Y. Zheng, S. Scott, and K. Deng. Active Learning from Multiple Noisy Labelers with Varied Costs. In IEEE 10th International Conference on Data Mining (ICDM), pages 639-648, 2010.