Conditional random field with high-order dependencies for sequence labeling and segmentation

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 981-1009

  Cuong, Nguyen Viet ^a   Ye, Nan ^a   Lee, Wee Sun ^a   Chieu, Hai Leong ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b DSO NATIONAL LABORATORIES   (Singapore)

Author keywords

Conditional random field; High order feature; Label sparsity; Segmentation; Semi Markov conditional random field; Sequence labeling

Indexed keywords

ALGORITHMS; IMAGE SEGMENTATION; INFERENCE ENGINES;

CONDITIONAL RANDOM FIELD; FIRST-ORDER DEPENDENCY; HIGH-ORDER; INFERENCE ALGORITHM; SEQUENCE LABELING; SOURCES OF INFORMATIONS;

RANDOM PROCESSES;

EID: 84899873818     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (35)

1. Aron Culotta, David Kulp, and Andrew McCallum. Gene prediction with conditional random fields. Technical Report UM-CS-2005-028, University of Massachusetts, Amherst, 2005.
2. Thomas G. Dietterich, Adam Ashenfelter, and Yaroslav Bulatov. Training conditional random fields via gradient tree boosting. In Proceedings of the 21st International Conference on Machine Learning, 2004.
3. Richard Durbin. Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press, 1998.
4. Shai Fine, Yoram Singer, and Naftali Tishby. The hierarchical hidden Markov model: analysis and applications. Machine Learning, 32(1):41-62, 1998.
5. Zhou GuoDong, Su Jian, Zhang Jie, and Zhang Min. Exploring various knowledge in relation extraction. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 427-434, 2005.
6. Josée S. Heemskerk. A probabilistic context-free grammar for disambiguation in morphological parsing. In Proceedings of the 6th Conference of the European Chapter of the Association for Computational Linguistics, pages 183-192, 1993.
7. Cecil Huang and Adnan Darwiche. Inference in belief networks: a procedural guide. International Journal of Approximate Reasoning, 15(3):225-263, 1996.
8. Sorin Istrail. Statistical mechanics, three-dimensionality and NP-completeness: I. Universality of intractability for the partition function of the Ising model across non-planar lattices. In Proceedings of the 32nd Annual ACM Symposium on Theory of Computing, pages 87-96, 2000.
9. Frederick Jelinek, John D. Laérty, and Robert L. Mercer. Basic methods of probabilistic context free grammars. In Speech Recognition and Understanding. Recent Advances, Trends, and Applications. Springer, 1992.
10. Robert H. Kassel. A Comparison of Approaches to On-line Handwritten Character Recog- nition. PhD thesis, Massachusetts Institute of Technology, 1995.
11. Pushmeet Kohli, M. Pawan Kumar, and Philip H. S. Torr. P3 & beyond: solving energies with higher order cliques. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8, 2007.
12. Nikos Komodakis and Nikos Paragios. Beyond pairwise energies: eficient optimization for higher-order MRFs. In IEEE Conference on Computer Vision and Pattern Recognition, pages 2985-2992, 2009.
13. John Laérty, Andrew McCallum, and Fernando C.N. Pereira. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning, 2001.
14. Dong C. Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(3):503-528, 1989. (Pubitemid 20660315)
15. Yang Liu, Andreas Stolcke, Elizabeth Shriberg, and Mary Harper. Using conditional random fields for sentence boundary detection in speech. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 451-458, 2005.
16. Wei Lu and Hwee Tou Ng. Better punctuation prediction with dynamic conditional random fields. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 177-186, 2010.
17. Andrew McCallum and Wei Li. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In Proceedings of the 7th Conference on Computational Natural Language Learning, pages 188-191, 2003.
18. Kevin P. Murphy and Mark A. Paskin. Linear-time inference in hierarchical HMMs. In Advances in Neural Information Processing Systems, pages 833-840, 2002.
19. Viet Cuong Nguyen, Nan Ye, Wee Sun Lee, and Hai Leong Chieu. Semi-Markov conditional random field with high-order features. In ICML Workshop on Structured Sparsity: Learning and Inference, 2011.
20. Eric W. Noreen. Computer Intensive Methods for Testing Hypotheses: An Introduction. Wiley, 1989.
21. Michael Paul. Overview of the IWSLT 2009 evaluation campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 3-27, 2009.
22. Xian Qian and Yang Liu. Sequence labeling with non-negative weighted higher order features. In Proceedings of the 26th Conference on Artificial Intelligence, 2012.
23. Xian Qian, Xiaoqian Jiang, Qi Zhang, Xuanjing Huang, and Lide Wu. Sparse higher order conditional random fields for improved sequence labeling. In Proceedings of the 26th International Conference on Machine Learning, pages 849-856, 2009.
24. Lawrence R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286, 1989.
25. Carsten Rother, Pushmeet Kohli, Wei Feng, and Jiaya Jia. Minimizing sparse higher order energy functions of discrete variables. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1382-1389, 2009.
26. Sunita Sarawagi and William W. Cohen. Semi-Markov conditional random fields for information extraction. In Advances in Neural Information Processing Systems, pages 1185- 1192, 2004.
27. Fei Sha and Fernando Pereira. Shallow parsing with conditional random fields. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 134-141, 2003.
28. Daniel Tarlow, Inmar E. Givoni, and Richard S. Zemel. HOP-MAP: eficient message passing with high order potentials. In International Conference on Artificial Intelligence and Statistics, pages 812-819, 2010.
29. Ben Taskar, Carlos Guestrin, and Daphne Koller. Max-margin Markov networks. In Advances in Neural Information Processing Systems, 2004.
30. Tran T. Truyen, Dinh Q. Phung, Hung H. Bui, and Svetha Venkatesh. Hierarchical semi- Markov conditional random fields for recursive sequential data. In Advances in Neural Information Processing Systems, pages 1657-1664, 2008.
31. Ioannis Tsochantaridis, Thomas Hofmann, Thorsten Joachims, and Yasemin Altun. Support vector machine learning for interdependent and structured output spaces. In Proceedings of the 21st International Conference on Machine Learning, 2004.
32. Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. ACE 2005 multilingual training corpus. Linguistic Data Consortium, Philadelphia, 2006.
33. Nan Ye, Wee Sun Lee, Hai Leong Chieu, and Dan Wu. Conditional random fields with high-order features for sequence labeling. In Advances in Neural Information Processing Systems, pages 2196-2204, 2009.
34. Alexander Yeh. More accurate tests for the statistical significance of result differences. In Proceedings of the 18th International Conference on Computational Linguistics, pages 947-953, 2000.
35. Min Zhang, Jie Zhang, and Jian Su. Exploring syntactic features for relation extraction using a convolution tree kernel. In Proceedings of the Human Language Technology Con- ference of the North American Chapter of the Association for Computational Linguistics, pages 288-295, 2006.