Practical very large scale CRFs

ACL 2010 - 48th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Volumn , Issue , 2010, Pages 504-513

  Lavergne, Thomas ^a   Cappé, Olivier ^b   Yvon, François ^a  

^a UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

^b INSTITUT TELECOM   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CARDINALITIES; COMPACT PARAMETERS; COMPUTATIONAL EFFORT; CONDITIONAL RANDOM FIELDS (CRFS); PENALTY TERM; REGULARIZATION STRATEGIES;

COMPUTATIONAL LINGUISTICS;

EID: 84859972823     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (36)

1. Galen Andrew and Jianfeng Gao. 2007. Scalable training of l1-regularized log-linear models. In Proceedings of the International Conference on Machine Learning, pages 33-40, Corvalis, Oregon.
2. Léon Bottou. 2004. Stochastic learning. In Olivier Bousquet and Ulrike von Luxburg, editors, Advanced Lectures on Machine Learning, Lecture Notes in Artificial Intelligence, LNAI 3176, pages 146-168. Springer Verlag, Berlin.
3. Léon Bottou. 2007. Stochastic gradient descent (sgd) implementation. http://leon.bottou.org/projects/sgd.
4. Stanley Chen. 2009. Performance prediction for exponential language models. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 450-458, Boulder, Colorado, June.
5. Trevor Cohn. 2006. Efficient inference in large conditional random fields. In Proceedings of the 17th European Conference on Machine Learning, pages 606-613, Berlin, September.
6. Thomas G. Dietterich, Adam Ashenfelter, and Yaroslav Bulatov. 2004. Training conditional random fields via gradient tree boosting. In Proceedings of the International Conference on Machine Learning, Banff, Canada.
7. Miroslav Dudík, Steven J. Phillips, and Robert E. Schapire. 2004. Performance guarantees for regularized maximum entropy density estimation. In John Shawe-Taylor and Yoram Singer, editors, Proceedings of the 17th annual Conference on Learning Theory, volume 3120 of Lecture Notes in Computer Science, pages 472-486. Springer.
8. Jenny Rose Finkel, Alex Kleeman, and Christopher D. Manning. 2008. Efficient, feature-based, conditional random field parsing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 959-967, Columbus, Ohio.
9. Jerome Friedman, Trevor Hastie, and Rob Tibshirani. 2008. Regularization paths for generalized linear models via coordinate descent. Technical report, Department of Statistics, Stanford University.
10. Jianfeng Gao, Galen Andrew, Mark Johnson, and Kristina Toutanova. 2007. A comparative study of parameter estimation methods for statistical natural language processing. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 824-831, Prague, Czech republic.
11. Minwoo Jeong, Chin-Yew Lin, and Gary Geunbae Lee. 2009. Efficient inference of crfs for large-scale natural language data. In Proceedings of the Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, pages 281-284, Suntec, Singapore.
12. Jun'ichi Kazama and Jun'ichi Tsujii. 2003. Evaluation and extension of maximum entropy models with inequality constraints. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 137-144.
13. Taku Kudo. 2005. CRF++: Yet another CRF toolkit. http://crfpp. sourceforge.net/.
14. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In Proceedings of the International Conference on Machine Learning, pages 282-289. Morgan Kaufmann, San Francisco, CA.
15. Percy Liang, Hal Daumé, III, and Dan Klein. 2008. Structure compilation: trading structure for features. In Proceedings of the 25th international conference on Machine learning, pages 592-599.
16. Dong C. Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45:503-528.
17. Gideon Mann, Ryan McDonald, Mehryar Mohri, Nathan Silberman, and Dan Walker. 2009. Efficient large-scale distributed training of conditional maximum entropy models. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I.Williams, and A.Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1231-1239.
18. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn treebank. Computational Linguistics, 19(2):313-330.
19. Jorge Nocedal and Stephen Wright. 2006. Numerical Optimization. Springer.
20. Naoaki Okazaki. 2007. CRFsuite: A fast implementation of conditional random fields (CRFs).http://www.chokkan.org/software/crfsuite/.
21. Chris Pal, Charles Sutton, and Andrew McCallum. 2006. Sparse forward-backward using minimum divergence beams for fast training of conditional random fields. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, Toulouse, France.
22. Simon Perkins, Kevin Lacker, and James Theiler. 2003. Grafting: Fast, incremental feature selection by gradient descent in function space. Journal of Machine Learning Research, 3:1333-1356.
23. Vasin Punyakanok, Dan Roth, Wen tau Yih, and Dav Zimak. 2005. Learning and inference over constrained output. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 1124-1129.
24. Xian Qian, Xiaoqian Jiang, Qi Zhang, Xuanjing Huang, and Lide Wu. 2009. Sparse higher order conditional random fields for improved sequence labeling. In Proceedings of the Annual International Conference on Machine Learning, pages 849-856.
25. Stefan Riezler and Alexander Vasserman. 2004. Incremental feature selection and l1 regularization for relaxed maximum-entropy modeling. In Dekang Lin and Dekai Wu, editors, Proceedings of the conference on Empirical Methods in Natural Language Processing, pages 174-181, Barcelona, Spain, July.
26. Antoine Rozenknop. 2002. Modèles syntaxiques probabilistes non-génératifs. Ph.D. thesis, Dpt. d'informatique, École Polytechnique Fédérale de Lausanne.
27. Terrence J. Sejnowski and Charles R. Rosenberg. 1987. Parallel networks that learn to pronounce english text. Complex Systems, 1.
28. Libin Shen, Giorgio Satta, and Aravind Joshi. 2007. Guided learning for bidirectional sequence classification. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 760-767, Prague, Czech Republic.
29. Nataliya Sokolovska, Thomas Lavergne, Olivier Cappé, and François Yvon. 2010. Efficient learning of sparse conditional random fields for supervised sequence labelling. IEEE Selected Topics in Signal Processing.
30. Charles Sutton and Andrew McCallum. 2006. An introduction to conditional random fields for relational learning. In Lise Getoor and Ben Taskar, editors, Introduction to Statistical Relational Learning, Cambridge, MA. The MIT Press.
31. Jun Suzuki and Hideki Isozaki. 2008. Semi-supervised sequential labeling and segmentation using gigaword scale unlabeled data. In Proceedings of the Conference of the Association for Computational Linguistics on Human Language Technology, pages 665-673, Columbus, Ohio.
32. Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. J.R.Statist.Soc.B, 58(1):267-288.
33. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, pages 173-180.
34. Yoshimasa Tsuruoka, Jun'ichi Tsujii, and Sophia Ananiadou. 2009. Stochastic gradient descent training for l1-regularized log-linear models with cumulative penalty. In Proceedings of the Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, pages 477-485, Suntec, Singapore.
35. S. V. N. Vishwanathan, Nicol N. Schraudolph, Mark Schmidt, and Kevin Murphy. 2006. Accelerated training of conditional random fields with stochastic gradient methods. In Proceedings of the 23th International Conference on Machine Learning, pages 969-976. ACM Press, New York, NY, USA.
36. Hui Zhou and Trevor Hastie. 2005. Regularization and variable selection via the elastic net. J. Royal. Stat. Soc. B., 67(2):301-320.