A comparison and improvement of online learning algorithms for sequence labeling

24th International Conference on Computational Linguistics - Proceedings of COLING 2012: Technical Papers

Volumn , Issue , 2012, Pages 1147-1162

  He, Zhengyan ^a   Wang, Houfeng ^a  

^a Peking University   (China)

Author keywords

Named entity recognition; Online learning; Sequence labeling; Stochastic gradient descent

Indexed keywords

CONDITIONAL RANDOM FIELD; NAMED ENTITY RECOGNITION; ON-LINE LEARNING METHODS; ONLINE LEARNING; ONLINE LEARNING ALGORITHMS; SEQUENCE LABELING; STOCHASTIC GRADIENT DESCENT; TUNING OF PARAMETERS;

COMPUTATIONAL LINGUISTICS; LEARNING ALGORITHMS; TIME VARYING NETWORKS;

E-LEARNING;

EID: 84876782574     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Bordes, A., Bottou, L., and Gallinari, P. (2009). Sgd-qn: Careful quasi-newton stochastic gradient descent. The Journal of Machine Learning Research, 10:1737-1754.
2. Bottou, L. and Bousquet, O. (2008). The tradeoffs of large scale learning. Advances in neural information processing systems, 20:161-168.
3. Collins, M. (2002). Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, pages 1-8. Association for Computational Linguistics.
4. Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., and Singer, Y. (2006). Online passiveaggressive algorithms. The Journal of Machine Learning Research, 7:551-585.
5. Hsu, C., Huang, H., Chang, Y., and Lee, Y. (2009). Periodic step-size adaptation in second-order gradient descent for single-pass on-line structured learning. Machine learning, 77(2):195-224.
6. Lafferty, J., McCallum, A., and Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data.
7. Lavergne, T., Cappé, O., and Yvon, F. (2010). Practical very large scale crfs. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 504-513, Uppsala, Sweden. Association for Computational Linguistics.
8. LeCun, Y., Bottou, L., Orr, G., and Müller, K. (1998). Efficient backprop. Neural networks: Tricks of the trade, pages 546-546.
9. Malouf, R. et al. (2002). A comparison of algorithms for maximum entropy parameter estimation. In Proceedings of the sixth conference on natural language learning (CoNLL-2002), pages 49-55.
10. Martins, A., Gimpel, K., Smith, N., Xing, E., Figueiredo, M., and Aguiar, P. (2010). Learning structured classifiers with dual coordinate ascent. Technical report, DTIC Document.
11. Martins, A., Smith, N., Figueiredo, M., and Aguiar, P. (2011). Structured sparsity in structured prediction. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1500-1511, Edinburgh, Scotland, UK. Association for Computational Linguistics.
12. Mohit, B., Schneider, N., Bhowmick, R., Oflazer, K., and Smith, N. A. (2012). Recall-oriented learning of named entities in arabic wikipedia. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 162-173, Avignon, France. Association for Computational Linguistics.
13. Murata, N. (1998). A statistical study of on-line learning. Online Learning and Neural Networks. Cambridge University Press, Cambridge, UK.
14. Nadeau, D. and Sekine, S. (2007). A survey of named entity recognition and classification. Lingvisticae Investigationes, 30(1):3-26.
15. Poon, H. and Vanderwende, L. (2010). Joint inference for knowledge extraction from biomedical literature. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 813-821, Los Angeles, California. Association for Computational Linguistics.
16. Rabiner, L. (1989). A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286.
17. Ratinov, L. and Roth, D. (2009). Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pages 147-155, Boulder, Colorado. Association for Computational Linguistics.
18. Sha, F. and Pereira, F. (2003). Shallow parsing with conditional random fields. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 134-141.
19. Shalev-Shwartz, S., Singer, Y., and Srebro, N. (2007). Pegasos: Primal estimated sub-gradient solver for svm. In Proceedings of the 24th international conference on Machine learning, pages 807-814. ACM.
20. Smith, N. (2011). Linguistic structure prediction. Synthesis Lectures on Human Language Technologies, 4(2):1-274.
21. Song, H.-J., Son, J.-W., Noh, T.-G., Park, S.-B., and Lee, S.-J. (2012). A cost sensitive part-ofspeech tagging: Differentiating serious errors from minor errors. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1025-1034, Jeju Island, Korea. Association for Computational Linguistics.
22. Sun, W. and Uszkoreit, H. (2012). Capturing paradigmatic and syntagmatic lexical relations: Towards accurate chinese part-of-speech tagging. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jeju Island, Korea.
23. Sun, W. and Xu, J. (2011). Enhancing chinese word segmentation using unlabeled data. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 970-979, Edinburgh, Scotland, UK. Association for Computational Linguistics.
24. Sun, X., Wang, H., and Li, W. (2012). Fast online training with frequency-adaptive learning rates for chinese word segmentation and new word detection. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 253-262, Jeju Island, Korea. Association for Computational Linguistics.
25. Tsochantaridis, I., Joachims, T., Hofmann, T., and Altun, Y. (2006). Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research, 6(2):1453.
26. Vishwanathan, S., Schraudolph, N., Schmidt, M., and Murphy, K. (2006). Accelerated training of conditional random fields with stochastic gradient methods. In Proceedings of the 23rd international conference on Machine learning, pages 969-976. ACM.
27. Xu, W. (2011). Towards optimal one pass large scale learning with averaged stochastic gradient descent. Arxiv preprint arXiv:1107.2490.