Modeling annotation time to reduce workload in comparative effectiveness reviews

IHI'10 - Proceedings of the 1st ACM International Health Informatics Symposium

Volumn , Issue , 2010, Pages 28-35

  Wallace, Byron C ^a   Small, Kevin ^a   Brodley, Carla E ^a   Lau, Joseph ^a   Trikalinos, Thomas A ^a  

^a TUFTS UNIVERSITY   (United States)

Author keywords

active learning; applications; medical; text classification

Indexed keywords

ACTIVE LEARNING; ANNOTATION TOOL; CLASSIFICATION MODELS; COMPARATIVE EFFECTIVENESS; HEALTH-CARE DECISIONS; MACHINE LEARNING SYSTEMS; MACHINE-LEARNING; MEDICAL; OPEN-SOURCE; SCIENTIFIC LITERATURE; SCREENING SYSTEM; SEMI-AUTOMATED; SYSTEMATIC REVIEW; TEXT CLASSIFICATION;

HARMONIC ANALYSIS; HEALTH CARE; INFORMATION SCIENCE; LEARNING SYSTEMS; TEXT PROCESSING;

TIME SERIES ANALYSIS;

EID: 78650925531     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1882992.1882999     Document Type: Conference Paper

References (16)

1. P. Wheeler, E. Balk, K. Bresnahan, B. Shephard, J. Lau, D. DeVine, M. Chung, and K. Miller. Criteria for determining disability in infants and children: short stature. Evidence Report/Technology Assessment No. 73. Prepared by New England Medical Center Evidence-based Practice Center under Contract No. 290-97-001, Mar 2003.
2. C. Cole, G. Binney, P. Casey, J. Fiascone, J. Hagadorn, C. Kim, C. Wang, D. Devine, K. Miller, and J. Lau. Criteria for determining disability in infants and children: Low birth weight. Evidence Report/Technology Assessment No. 70. Prepared by New England Medical Center Evidence-based Practice Center under Contract No. 290-97-0019, 2002.
3. E. Perrin, C. Cole, D. Frank, S. Glicken, N. Guerina, K. Petit, R. Sege, M. Volpe, P. Chew, C. MeFadden, D. Devine, K. Miller, and J. Lau. Criteria for determining disability in infants and children: failure to thrive. Evidence Report/Technology Assessment No. 72. Prepared by New England Medical Center Evidence-based Practice Center under Contract No. 290-97-0019, Mar 2003.
4. S. Arora, E. Nyberg, and C. Rosé. Estimating annotation cost for active learning in a multi-annotator environment. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL) Workshop on Active Learning for Natural Language Processing, pages 18-26. Association for Computational Linguistics, 2009.
5. J. Baldridge and A. Palmer. How well does active learning actually work?: Time-based evaluation of cost-reduction strategies for language documentation. In Empirical Methods on Natural Language Processing (EMNLP), pages 296-305. Association for Computational Linguistics, 2009.
6. P. Donmez and J. G. Carbonell. Proactive learning: cost-sensitive active learning with multiple imperfect oracles. In Conference on Information and Knowledge Management (CIKM), pages 619-628, 2008.
7. R. Haertel, K. Seppi, E. Ringger, and J. Carroll. Return on investment for active learning. In NIPS Workshop on Cost Sensitive Learning, 2009.
8. N. Japkowicz. Learning from imbalanced data sets: A comparison of various strategies. AAAI Workshop on Learning from Imbalanced Data Sets, 2000.
9. A. Mccallum and K. Nigam. Employing EM and pool-based active learning for text classification. In International Conference on Machine Learning (ICML), pages 350-358, San Francisco, CA, USA, 1998.
10. B. Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison, 2009.
11. B. Settles, M. Craven, and L. Friedland. Active learning with real annotation costs. In Proceedings of the Neural Information Processing Systems (NIPS) Workshop on Cost-Sensitive Learning, pages 1069-1078. Citeseer, 2008.
12. K. Tomanek and F. Olsson. A web survey on the use of active learning to support annotation of text data. In NAACL Workshop on AL for NLP, pages 45-48, June 2009.
13. S. Tong and D. Koller. Support vector machine active learning with applications to text classification. In Journal of Machine Learning Research, pages 999-1006, 2000.
14. A. J. Vickers and E. B. Elkin. Decision curve analysis: A novel method for evaluating prediction models. Medical Decision Making, 26:565-574, 2006.
15. B. C. Wallace, K. Small, C. E. Brodley, and T. A. Trikalinos. Active learning for biomedical citation screening. In Knowledge Discovery and Data mining (KDD), 2010.
16. B. C. Wallace, T. A. Trikalinos, J. Lau, C. E. Brodley, and C. H. Schmid. Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics, 11, 2010.