Semi-supervised sequence labeling with self-learned features

Proceedings - IEEE International Conference on Data Mining, ICDM

Volumn , Issue , 2009, Pages 428-437

  Qi, Yanjun ^a   Kuksa, Pavel ^b   Collobert, Ronan ^a   Sadamasa, Kunihiko ^a   Kavukcuoglu, Koray ^c   Weston, Jason ^d  

^a NEC LABORATORIES AMERICA   (United States)

^b RUTGERS UNIVERSITY   (United States)

^c POLYTECHNIC UNIVERSITY   (United States)

^d GOOGLE INC   (United States)

Author keywords

Information extraction; Self learned features; Semi supervised feature learning; Semi supervised learning; Sequence labeling; Structural output learning

Indexed keywords

CLASS BOUNDARY; CLASS LABELS; CURRENT MODELS; FEATURE LEARNING; INFORMATION EXTRACTION; INFORMATION EXTRACTION SYSTEMS; NAME RECOGNITION; NAMED ENTITY RECOGNITION; NATURAL LANGUAGES; PART OF SPEECH TAGGING; SELF-TRAINING; SEMI-SUPERVISED; SEMI-SUPERVISED LEARNING; SEQUENCE LABELING; SUPERVISED CLASSIFIERS; TARGET CLASS;

INFORMATION ANALYSIS; LABELS; LOCOMOTIVES; RAILROAD CARS; SPEECH RECOGNITION; SUPERVISED LEARNING;

FEATURE EXTRACTION;

EID: 77951193230     PISSN: 15504786     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDM.2009.40     Document Type: Conference Paper

References (37)

1. F. Jiao, S. Wang, C.-H. Lee, R. Greiner, and D. Schuurmans, "Semi-supervised conditional random fields for improved sequence segmentation and labeling," in Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics., 2006, pp. 209-216.
2. H. Scudder, "Probability of error of some adaptive patternrecognition machines," IEEE Transactions on Information Theory, vol. 11, no. 3, pp. 363-371, 1965.
3. R. J. Kate and R. J. Mooney, "Semi-supervised learning for semantic parsing using support vector machines," in Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Short Papers (NAACL/HLT-2007), 2007, pp. 81-84.
4. Z. Kozareva, B. Bonev, and A. Montoyo, "Self-training and co-training applied to spanish named entity recognition," in MICAI 2005: Advances in Artificial Intelligence, 2005.
5. A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in COLT' 98:. New York, NY, USA: ACM, 1998, pp. 92-100.
6. M. Collins and Y. Singer, "Unsupervised models for named entity classification," in Proceedings of the Joint SIGDAT Conference on EMNLP, 1999, pp. 100-110.
7. H. Shan and D. Gildea, "Self-training and co-training for semantic role labeling: Primary report," University of Rochester, Comp. Sci. Dept., Tech. Rep. TR891, 2006.
8. T. Zhang, F. Damerau, and D. Johnson, "Text chunking using regularized winnow," in ACL '01: Proceedings of the 39th Annual Meeting on Association for Computational Linguistics, Morristown, NJ, USA, 2001, pp. 539-546.
9. H. Daumé III, "Cross-task knowledge-constrained self training," in Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, Honolulu, Hawaii, October 2008, pp. 680-688.
10. Y. Qi, R. Collobert, P. Kuksa, K. Kavukcuoglu, and J. Weston, "Combining labeled and unlabeled data for word-class distribution learning," in CIKM'09: Proceedings of the eighth ACM Conference on Information and Knowledge Management. ACM, 2009.
11. Erik and F. De Meulder, "Introduction to the conll-2003 shared task: language-independent named entity recognition," in Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003, 2003, pp. 142-147.
12. K. Toutanova, D. Klein, C. D. Manning, and Y. Singer, "Feature-rich part-of-speech tagging with a cyclic dependency network," in NAACL '03: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, Morristown, NJ, USA, 2003, pp. 173-180.
13. L. Smith, L. Tanabe, R. Ando, et al., and J. W. Wilbur, "Overview of biocreative ii gene mention recognition," Genome Biology, vol. 9, no. Suppl 2, 2008.
14. R. Collobert and J. Weston, "A unified architecture for nlp: deep neural networks with multitask learning," in ICML '08: Proceedings of the 25th international conference on Machine learning, 2008, pp. 160-167.
15. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in ICML'01:, 2001.
16. C. D. Manning and H. Schtze, Foundations of Statistical Natural Language Processing. MIT press, 1999.
17. A. Gersho and R. M. Gray, Vector quantization and signal compression. Norwell, MA, USA: Kluwer Academic Publishers, 1991.
18. "Crf++: Yet another crf toolkit." [Online]. Available: http://crfpp.sourceforge.net/.
19. D. Nadeau and S. Sekine, "A survey of named entity recognition and classfication," Linguisticae Investigationes, vol. 30, no. 1, pp. 3-26, 2007.
20. T. Joachims, "Transductive inference for text classification using support vector machines," in ICML '99: Proceedings of the Sixteenth International Conference on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1999, pp. 200-209.
21. R. Collobert, F. Sinz, J. Weston, and L. Bottou, "Large scale transductive svms," vol. 7. Cambridge, MA, USA: MIT Press, 2006, pp. 1687-1712.
22. X. Zhu, Z. Ghahramani, and J. Lafferty, "Semi-supervised learning using gaussian fields and harmonic functions," in ICML'03: Proceedings of the 20th International Conference on Machine Learning, 2003, pp. 912-919.
23. Y. Grandvalet and Y. Bengio, "Semi-supervised learning by entropy minimization," in Advances in Neural Information Processing Systems 17, L. K. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, pp. 529-536.
24. K. Nigam, A. K. McCallum, S. Thrun, and T. Mitchell, "Text classification from labeled and unlabeled documents using em," vol. 39, no. 2-3. Hingham, MA, USA: Kluwer Academic Publishers, 2000, pp. 103-134.
25. O. Chapelle, B. Schölkopf, and A. Zien, Eds., Semi- Supervised Learning (Adaptive Computation and Machine Learning). MIT Press, 2006.
26. H. Daumée III, "Semi-supervised or semi-unsupervised?" in Proceedings of the NAACL HLT Workshop on Semisupervised Learning for Natural Language Processing, vol. Invited Position Paper,, no. 84-85, Boulder, Colorado, USA, June 2009, pp. 19-27.
27. S. Dasgupta and V. Ng, "Discriminative models for semisupervised natural language learning", invited position paper," in Proceedings of the NAACL HLT Workshop on Semisupervised Learning for Natural Language Processing, Boulder, Colorado, USA, June 2009, pp. 84-85.
28. D. Pierce and C. Cardie, "Limitations of co-training for natural language learning from large datasets," in In Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing, 2001, pp. 1-9.
29. A. Blum, "Semi-supervised learning using randomized mincuts," in ICML '04: Proceedings of the twenty-first international conference on Machine learning, 2004.
30. G. Mann and A. McCallum, "Generalized expectation criteria for semi-supervised learning of conditional random fields," in ACL-08: Association for Computational Linguistics., 2008, pp. 870-878.
31. A. Goldberg and X. Zhu, "Keepin' it real: Semi-supervised learning with realistic tuning," in Proceedings of the NAACL HLT Workshop on Semi-supervised Learning for Natural Language Processing, Boulder, Colorado, USA, June 2009, pp. 19-27.
32. R. K. Ando and T. Zhang, "A framework for learning predictive structures from multiple tasks and unlabeled data," Journal of Machine Learning Research, vol. 6, pp. 1817- 1853, 2005.
33. R. E. Schapire, M. Rochery, M. G. Rahim, and N. Gupta, "Incorporating prior knowledge into boosting," in ICML '02: Proceedings of the Nineteenth International Conference on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2002, pp. 538-545.
34. X. Wu and R. Srihari, "Incorporating prior knowledge with weighted margin support vector machines," in KDD '04: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. New York, NY, USA: ACM, 2004, pp. 326-333.
35. G. Druck, G. Mann, and A. McCallum, "Learning from labeled features using generalized expectation criteria," in SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, 2008, pp. 595-602.
36. A. Haghighi and D. Klein, "Prototype-driven learning for sequence models," in Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, Morristown, NJ, USA, 2006, pp. 320-327.
37. R. Florian, A. Ittycheriah, H. Jing, and T. Zhang, "Named entity recognition through classifier combination," in Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003, 2003, pp. 168-171.