Conditional random fields in speech, audio, and language processing

Proceedings of the IEEE

Volumn 101, Issue 5, 2013, Pages 1054-1075

  Fosler Lussier, Eric ^a   He, Yanzhang ^a   Jyothi, Preethi ^a   Prabhavalkar, Rohit ^a  

^a The Ohio State University   (United States)

Author keywords

Automatic speech recognition (ASR); natural language processing (NLP); random fields; statistical learning

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS; SPEECH RECOGNITION;

AUTOMATIC SPEECH RECOGNITION; CONDITIONAL RANDOM FIELD; CONDITIONAL RANDOM FIELDS(CRFS); NATURAL LANGUAGE PROCESSING; PROBABILISTIC MODELS; PROBABILISTIC SEQUENCES; RANDOM FIELDS; STATISTICAL LEARNING;

RANDOM PROCESSES;

EID: 84876691724     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2013.2248112     Document Type: Review

References (93)

1. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in Proc. Int. Conf. Mach. Learn., Williamstown, MA, USA, Jun. 2001, pp. 282-289.
2. D. Yu and L. Deng, "Deep-structured hidden conditional random fields for phonetic recognition," in Proc. Annu. Conf. Int. Speech Commun. Assoc., Makuhari, Chiba, Japan, Sep. 2010, pp. 2986-2989.
3. C. Sutton and A. McCallum, "An introduction to conditional random fields for relational learning," in Introduction to Statistical Relational Learning, L. Getoor and B. Taskar, Eds. Cambridge, MA, USA: MIT Press, 2007, pp. 93-128.
4. A. Gunawardana, M. Mahajan, A. Acero, and J. C. Platt, "Hidden conditional random fields for phone classification," in Proc. Eur. Conf. Speech Commun. Technol., Lisbon, Portugal, Sep. 2005, pp. 1117-1120.
5. G. Heigold, H. Ney, P. Lehnen, T. Gass, and R. Schlüter, "Equivalence of generative and log-linear models," IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 5, pp. 1138-1148, Jul. 2011.
6. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
7. A. McCallum, D. Freitag, and F. Pereira, "Maximum entropy Markov models for information extraction and segmentation," in Proc. Int. Conf. Mach. Learn., Stanford, CA, USA, Jun. 2000, pp. 591-598.
8. D. Klein, "Introduction to classification: Likelihoods, margins, features, and kernels," in Proc. Human Lang. Technol. Conf. NAACLVTut., Rochester, NY, USA, Apr. 2007.
9. L. Bottou, "Une approche th́eorique de l'apprentissage connexionniste: Applications ̀a la reconnaissance de la parole," Ph.D. dissertation, Dept. Electr. Eng. Comput. Sci., Universit́e de Paris XI, Paris, France, 1991.
10. E. McDermott, "Discriminative training for speech recognition," Ph.D. dissertation, Dept. Comput. Sci. Eng., Waseda Univ., Tokyo, Japan, 1997.
11. C. Sutton, A. McCallum, and K. Rohanimanesh, "Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data," J. Mach. Learn. Res., vol. 8, pp. 693-723, Mar. 2007.
12. H. Wallach, "Efficient training of conditional random fields," M.S. thesis, Schl. Cogn. Sci., Univ. Edinburgh, Edinburgh, 2002.
13. F. Sha and F. Pereira, "Shallow parsing with conditional random fields," in Proc. Human Lang. Technol. Conf. North Amer. Chapter Assoc. Comput. Linguist., Edmonton, ON, Canada, May 2003, vol. 1, pp. 134-141.
14. J. Hammersley and P. Clifford, "Markov fields on finite graphs and lattices," 1971. [Online]. Available: http://www.statslab.cam.ac.uk/~grg/ books/hammfest/hamm-cliff.pdf.
15. Y. Hifny and S. Renals, "Speech recognition using augmented conditional random fields," IEEE Trans. Audio Speech Lang. Process., vol. 17, no. 2, pp. 354-365, Feb. 2009.
16. A. L. Berger, S. D. Della Pietra, and V. J. D. Della Pietra, "A maximum entropy approach to natural language processing," Comput. Linguist., vol. 22, no. 1, pp. 39-71, Mar. 1996.
17. T. Minka, "Algorithms for maxiumum-likelihood logistic regression," Stat. Dept., Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. 758, 2001.
18. R. H. Byrd, P. Lu, J. Nocedal, and C. Y. Zhu, "A limited memory algorithm for bound constrained optimization," SIAM J. Sci. Comput., vol. 16, no. 6, pp. 1190-1208, Sep. 1995.
19. M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm," in Proc. IEEE Int. Conf. Neural Netw., San Francisco, CA, USA, Mar. 1993, vol. 1, pp. 586-591.
20. G. Zweig and P. Nguyen, "A segmental CRF approach to large vocabulary continuous speech recognition," in Proc. IEEE Workshop Autom. Speech Recognit. Understand., Merano, Italy, Dec. 2009, pp. 152-157.
21. X. He, L. Deng, and W. Chou, "Discriminative learning in sequential pattern recognition," IEEE Signal Process. Mag., vol. 25, no. 5, pp. 14-36, Sep. 2008.
22. C.-H. Lee, "Discriminative trainingVFundamentals and applications," presented at the Interspeech Tut. Session, Makuhari, Chiba, Japan, Sep. 2010.
23. M. Collins, "Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms," in Proc. Conf. Empirical Methods Natural Lang. Process., Prague, Czech Republic, Jul. 2002, pp. 1-8.
24. J. Suzuki, E. McDermott, and H. Isozaki, "Training conditional random fields with multivariate evaluation measures," in Proc. Annu. Meeting Assoc. Comput. Linguist., Sydney, Australia, Jul. 2006, pp. 217-224.
25. D. Povey and P. Woodland, "Minimum phone error and I-smoothing for improved discriminative training," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., May 2002, vol. 1, pp. 105-108.
26. Y. Xiong, J. Zhu, H. Huang, and H. Xu, "Minimum tag error for discriminative training of conditional random fields," Inf. Sci., vol. 179, no. 1-2, pp. 169-179, Jan. 2009.
27. M. Kim. (2010). Large margin cost-sensitive learning of conditional random fields. Pattern Recognit. [Online]. 43(10), pp. 3683-3692. Available: http://www.sciencedirect.com/science/article/pii/S0031320310002141
28. A. Ratnaparkhi, "A maximum entropy model for part-of-speech tagging," in Proc. Conf. Empirical Methods Natural Lang. Process., E. Brill and K. Church, Eds., Somerset, NJ, USA, May 1996, pp. 133-142.
29. R. Rosenfeld, "Two decades of statistical language modeling: Where do we go from here?" Proc. IEEE, vol. 88, no. 8, pp. 1270-1278, Aug. 2000.
30. J. Chien and C. Chueh, "Joint acoustic and language modeling for speech recognition," Speech Commun., vol. 52, no. 3, pp. 223-235, Mar. 2010.
31. M. I. Layton and M. J. F. Gales, "Augmented statistical models for speech recognition," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Toulouse, France, May 2006, vol. 1, pp. 129-132.
32. A. Ragni and M. J. F. Gales, "Derivative kernels for noise robust ASR," in Proc. IEEE Workshop Autom. Speech Recognit. Understand., Dec. 2011, pp. 119-124.
33. J. Morris and E. Fosler-Lussier, "Conditional random fields for integrating local discriminative classifiers," IEEE Trans. Audio Speech Lang. Process., vol. 16, no. 3, pp. 617-628, Mar. 2008.
34. M. Lehr and I. Shafran, "Learning a discriminative weighted finite-state transducer for speech recognition," IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 5, pp. 1360-1367, Jul. 2011.
35. B. Roark, M. Saraclar, M. Collins, and M. Johnson, "Discriminative language modeling with conditional random fields and the perceptron algorithm," in Proc. Annu. Meeting Assoc. Comput. Linguist., Barcelona, Spain, Jul. 2004, pp. 47-54.
36. Y. Kubo, S. Watanabe, and A. Nakamura, "Decoding network optimization using minimum transition error training," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Kyoto, Japan, Mar. 2012, pp. 4197-4200.
37. Y. H. Abdel-Haleem, "Conditional random fields for continuous speech recognition," Ph.D. dissertation, Dept. Comput. Sci., Univ. Sheffield, Sheffield, U.K., 2006.
38. G. Zweig, "Classification and recognition with direct segment models," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Kyoto, Japan, Mar. 2012, pp. 4161-4164.
39. V. N. Vapnik, The Nature of Statistical Learning Theory. New York, NY, USA: Springer-Verlag, 1995.
40. J. Lafferty, X. Zhu, and Y. Liu, "Kernel conditional random fields: Representation and clique selection," in Proc. Int. Conf. Mach. Learn., Banff, AB, Canada, Jul. 2004, pp. 64-71.
41. J. Peng, L. Bo, and J. Xu, "Conditional neural fields," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2009, pp. 1419-1427.
42. R. Prabhavalkar and E. Fosler-Lussier, "Backpropagation training for multilayer conditional random field based phone recognition," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Dallas, TX, USA, Mar. 2010, pp. 5534-5537.
43. Y. Fujii, K. Yamamoto, and S. Nakagawa, "Automatic speech recognition using hidden conditional neural fields," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Prague, Czech Republic, May 2011, pp. 5036-5039.
44. B. Kingsbury, "Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Taipei, Taiwan, Apr. 2009, pp. 3761-3764.
45. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., vol. 18, no. 7, pp. 1527-1554, May 2006.
46. D. Yu, S. Wang, and L. Deng, "Sequential labeling using deep-structured conditional random fields," IEEE J. Sel. Top. Signal Process., vol. 4, no. 6, pp. 965-973, Dec. 2010.
47. S. Wiesler and H. Ney, "A convergence analysis of log-linear training," in Proc. Adv. Neural Inf. Process. Syst., J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Weinberger, Eds., Granada, Spain, Mar. 2011, pp. 657-665.
48. T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning. New York, NY, USA: Springer-Verlag, 2001.
49. D. Johnson, D. Ellis, C. Oei, C. Wooters, P. Faerber, N. Morgan, and K. Asanovic, "ICSI Quicknet software package," 2004. [Online]. Available: http://www.icsi.berkeley. edu/Speech/qn.html.
50. I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun, "Large margin methods for structured and interdependent output variables," J. Mach. Learn. Res., vol. 6, pp. 1453-1484, Sep. 2005.
51. B. Taskar, C. Guestrin, and D. Koller, "Max-margin Markov networks," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2003, pp. 25-32.
52. M. Gales, S. Watanabe, and E. Fosler-Lussier, "Structured discriminative models for speech recognition," Signal Process. Mag., vol. 29, no. 6, pp. 70-81, Nov. 2012.
53. D. Koller and N. Friedman, Probabilistic Graphical Models: Principles and Techniques, 1st ed. Cambridge, MA, USA: MIT Press, 2009.
54. K. P. Murphy, "Dynamic Bayesian networks: Representation, inference and learning," Ph.D. dissertation, Comput. Sci. Div., Dept. Electr. Eng. Comput. Sci., Univ. California, Berkeley, CA, USA, 2002.
55. R. Prabhavalkar, Z. Jin, and E. Fosler-Lussier, "Monaural segregation of voiced speech using discriminative random fields," in Proc. Annu. Conf. Int. Speech Commun. Assoc., Brighton, U.K., Sep. 2009, pp. 856-859.
56. F. R. Kschischang, B. J. Frey, and H. A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
57. J. Pearl, "Fusion, propagation, and structuring in belief networks," Artif. Intell., vol. 29, no. 3, pp. 241-288, Sep. 1986.
58. K. P. Murphy, Y. Weiss, and M. I. Jordan, "Loopy belief propagation for approximate inference: An empirical study," in Proc. Int. Conf. Uncertainty Artif. Intell., Stockholm, Sweden, Jul. 1999, pp. 467-475.
59. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul, "An introduction to variational methods for graphical models," Mach. Learn., vol. 37, no. 2, pp. 183-233, Nov. 1999.
60. M. J. Wainwright and M. I. Jordan, "Graphical models, exponential families, and variational inference," Found. Trends Mach. Learn., vol. 1, no. 1-2, pp. 1-305, Jan. 2008.
61. R. Prabhavalkar, E. Fosler-Lussier, and K. Livescu, "A factored conditional random field model for articulatory feature forced transcription," in Proc. IEEE Workshop Autom. Speech Recognit. Understand., Dec. 2011, pp. 77-82.
62. Y.-H. Sung and D. Jurafsky, "Hidden conditional random fields for phone recognition," in Proc. IEEE Workshop Autom. Speech Recognit. Understand., Merano, Italy, Dec. 2009, pp. 107-112.
63. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Stat. Soc., vol. 39, no. 1, pp. 1-38, 1977.
64. S. Sarawagi and W. W. Cohen, "Semi-Markov conditional random fields for information extraction," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2004, pp. 1185-1192.
65. Y. He and E. Fosler-Lussier, "Efficient segmental conditional random fields for phone recognition," in Proc. Annu. Conf. Int. Speech Commun. Assoc., Portland, OR, USA, Sep. 2012, pp. 1898-1901.
66. P. Liang, "Semi-supervised learning for natural language," M.S. thesis, Dept. Electr. Eng. Comput. Sci., Massachusetts Inst. Technol. (MIT), Cambridge, MA, USA, 2005.
67. G. Andrew, "A hybrid Markov/semi-Markov conditional random field for sequence segmentation," in Proc. Conf. Empirical Methods Natural Lang. Process., Sydney, Australia, Jul. 2006, pp. 465-472.
68. G. Zweig, P. Nguyen, D. Van Compernolle, K. Demuynck, L. Atlas, P. Clark, G. Sell, M. Wang, F. Sha, H. Hermansky, D. Karakos, A. Jansen, S. Thomas, G. S. V. S. Sivaram, S. Bowman, and J. Kao, "Speech recognition with segmental conditional random fields: A summary of the JHU CLSP 2010 summer workshop," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Prague, Czech Republic, May 2011, pp. 5044-5047.
69. M. Layton, "Augmented statistical models for classifying sequence data," Ph.D. dissertation, Dept. Eng., Cambridge Univ., Cambridge, U.K., 2006.
70. B. H. Juang and S. Katagiri, "Discriminative learning for minimum error classification," IEEE Trans. Signal Process., vol. 40, no. 12, pp. 3043-3054, Dec. 1992.
71. X. Huang, A. Acero, and H. W. Hon, Spoken Language Processing: A Guide to Theory, Algorithm and System Development. Englewood Cliffs, NJ, USA: Prentice-Hall, 2001.
72. G. Heigold, P. Lehnen, R. Schlüter, and H. Ney, "On the equivalence of Gaussian and log-linear HMMs," in Proc. Annu. Conf. Int. Speech Commun. Assoc., Brisbane, Australia, Sep. 2008, pp. 273-276.
73. H. Bourlard and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach. Norwell, MA, USA: Kluwer, 1993.
74. H. Hermansky, D. P. W. Ellis, and S. Sharma, "Tandem connectionist feature extraction for conventional HMM systems," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Jun. 2000, vol. 3, pp. 1635-1638.
75. J. S. Bridle, "Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition," in Neurocomputing: Algorithms, Architectures and Applications, Fogelman-Soulie and Herault, Eds. New York, NY, USA: Springer-Verlag, 1990, pp. 227-236.
76. Y. Konig, H. Bourlard, and N. Morgan, "REMAP: Recursive estimation and maximization of a posteriori probabilitiesVApplication to transition-based connectionist speech recognition," in Proc. Adv. Neural Inf. Process. Syst., Denver, CO, USA, Nov. 1995, pp. 388-394.
77. Y. Shi and M. Wang, "A dual-layer CRFs based joint decoding method for cascaded segmentation and labeling tasks," in Proc. Int. Joint Conf. Artif. Intell., Hyderabad, India, Jan. 2007, pp. 1707-1712.
78. N. A. Smith, D. A. Smith, and R. W. Tromble, "Context-based morphological disambiguation with random fields," in Proc. Human Lang. Technol. Conf./Conf. Empirical Methods Natural Lang. Process., Vancouver, BC, Canada, Oct. 2005, pp. 475-482.
79. T. Kudo, K. Yamamoto, and Y. Matsumoto, "Applying conditional random fields to Japanese morphological analysis," in Proc. Conf. Empirical Methods Natural Lang. Process., Barcelona, Catalunya, Spain, Jul. 2004, pp. 230-237.
80. A. McCallum and W. Li, "Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons," in Proc. Conf. Natural Lang. Learn., Edmonton, AB, Canada, May 2003, pp. 188-191.
81. J. Finkel, T. Grenager, and C. Manning, "Incorporating non-local information into information extraction systems by Gibbs sampling," in Proc. Annu. Meeting Assoc. Comput. Linguist., Ann Arbor, MI, USA, Jun. 2005, pp. 363-370.
82. J. Finkel, A. Kleeman, and C. Manning, "Efficient, feature-based, conditional random field parsing," in Proc. Annu. Meeting Assoc. Comput. Linguist., Human Lang. Technol., Columbus, OH, USA, Jun. 2008, pp. 959-967.
83. J. Finkel and C. Manning, "Nested named entity recognition," in Proc. Conf. Empirical Methods Natural Lang. Process., Suntec, Singapore, Aug. 2009, pp. 141-150.
84. J. Finkel and C. Manning, "Joint parsing and named entity recognition," in Proc. Human Lang. Technol. Conf. North Amer. Chapter Assoc. Comput. Linguist., Boulder, CO, USA, May 2009, pp. 326-334.
85. F. Peng and A. McCallum, "Accurate information extraction from research papers using conditional random fields," in Proc. Human Lang. Technol. Conf. North Amer. Chapter Assoc. Comput. Linguist., Boston, MA, USA, May 2004, pp. 329-336.
86. C. Sutton and A. McCallum, "Collective segmentation and labeling of distant entities in information extraction," Univ. Massachusetts, Amherst, MA, USA, Tech. Rep., 2004.
87. J. Morris and E. Fosler-Lussier, "Crandem: Conditional random fields for word recognition," in Proc. Annu. Conf. Int. Speech Commun. Assoc., Brighton, U.K., Sep. 2009, pp. 3063-3066.
88. J. J. Morris, "A study on the use of conditional random fields for automatic speech recognition," Ph.D. dissertation, Dept. Comput. Sci. Eng., The Ohio State Univ., Columbus, OH, USA, 2010.
89. D. Wang and S. King, "Letter-to-sound pronunciation prediction using conditional random fields," IEEE Signal Process. Lett., vol. 18, no. 2, pp. 122-125, Feb. 2011.
90. S. Hahn, M. Dinarelli, C. Raymond, F. Lefevre, P. Lehnen, R. de Mori, A. Moschitti, H. Ney, and G. Riccardi, "Comparing stochastic approaches to spoken language understanding in multiple languages," IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 6, pp. 1569-1583, Aug. 2011.
91. D. Wang, "On ideal binary mask as the computational goal of auditory scene analysis," in Speech Separation by Humans and Machines, P. Divenyi, Ed. New York, NY, USA: Springer-Verlag, 2005, pp. 181-197.
92. C. Joder, S. Essid, and G. Richard, "A conditional random field framework for robust and scalable audio-to-score matching," IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 8, pp. 2385-2397, Nov. 2011.
93. E. Benetos and S. Dixon, "Joint multi-pitch detection using harmonic envelope estimation for polyphonic music transcription," IEEE J. Sel. Top. Signal Process., vol. 5, no. 6, pp. 1111-1123, Oct. 2011.