Equivalence of Generative and Log-Linear Models

IEEE Transactions on Audio, Speech and Language Processing

Volumn 19, Issue 5, 2011, Pages 1138-1148

  Heigold, Georg ^a   Ney, Hermann ^a   Lehnen, Patrick ^a   Gass, Tobias ^a   Schlüter, Ralf ^a  

^a RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

Conditional random field (CRF); Gaussian mixture model (GMM); hidden Markov model (HMM); log linear model

Indexed keywords

EID: 85008035419     PISSN: 15587916     EISSN: 15587924     Source Type: Journal    
DOI: 10.1109/TASL.2010.2082532     Document Type: Article

References (28)

1. J. Anderson, “Logistic discrimination,” in Handbook of Statistics 2, P. Krishnaiah and L. Kanal, Eds. Amsterdam, The Netherlands: North-Holland, 1982, pp. 169–191.
2. A. Ng and M. Jordan, “On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes,” in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2002.
3. L. Saul and D. Lee, “Multiplicative updates for classification by mixture models,” in Advances in Neural Information Processing Systems (NIPS), T. G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2002.
4. J. Lafferty, A. McCallum, and F. Pereira, “Conditional random fields: Probabilistic models for segmenting and labeling sequence data,” in Int. Conf Machine Learning (ICML), San Francisco, CA, Jun.–Jul. 2001.
5. C. Sutton and A. McCallum, “An introduction to conditional random fields for relational learning,” in Introduction to Statistical Relational Learning, L. Getoor and B. Taskar, Eds. Cambridge, MA: MIT Press, 2007.
6. A. Gunawardana, M. Mahajan, A. Acero, and J. Platt, “Hidden conditional random fields for phone classification,” in Proc. Interspeech, Lisbon, Portugal, Sep. 2005.
7. Y. Hifny and S. Renals, “Speech recognition using augmented conditional random fields,” IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 2, pp. 354–365, Feb. 2009.
8. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York: Wiley, 2001.
9. G. Heigold, R. Schlüter, and H. Ney, “On the equivalence of Gaussian HMM and Gaussian HMM-like hidden conditional random fields,” in Proc. Interspeech, Antwerp, Belgium, Aug. 2007.
10. G. Heigold, P. Lehnen, R. Schlüter, and H. Ney, “On the equivalence of Gaussian and log-linear HMMs,” in Proc. Interspeech, Brisbane, Australia, Sep. 2008.
11. G. Heigold, S. Wiesler, M. Nußbaum, P. Lehnen, R. Schlüter, and H. Ney, “Discriminative HMMs, log-linear models, and CRFs: What is the difference?,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Dallas, TX, Mar. 2010, pp. 5546–5549.
12. F. Sha and L. Saul, “Comparison of large margin training to other discriminative methods for phonetic recognition by hidden Markov models,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Honolulu, HI, Apr. 2007, pp. 313–316.
13. M. Mohri, “Weighted automata algorithms,” in Handbook of Weighted Automata, M. Droste, W. Kuich, and H. Vogler, Eds. New York: Springer, 2009.
14. E. Jaynes, Probability Theory: The Logic of Science. Cambridge, MA: MIT Press, 2003.
15. S. Della Pietra, V. Della Pietra, and J. Lafferty “Inducing features of random fields,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 4, pp. 380–393, Apr. 1997.
16. S. Chen and R. Rosenfeld, A Gaussian Prior for Smoothing Maximum Entropy Models Comput. Sci. Dept., Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMUCS-99-108, 1999.
17. G. Heigold, D. Rybach, R. Schlüter, and H. Ney, “Investigations on convex optimization using log-linear HMMs for digit string recognition,” in Proc. Interspeech, Brighton, U.K., Sep. 2009.
18. F. Sha and L. Saul, “Large margin hidden Markov models for automatic speech recognition,” in Advances in Neural Information Processing Systems 19 (NIPS). Cambridge, MA: MIT Press, 2007.
19. D. Keysers, T. Deselaers, C. Gollan, and H. Ney “Deformation models for image recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 8, pp. 1422–1435, Aug. 2007.
20. A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Statist. Soc., vol. 39, no. B, 1977.
21. M. Riedmiller and H. Braun, “A direct adaptive method for faster backpropagation learning: The Rprop algorithm,” in Proc. IEEE Int. Conf. Neural Netw. (ICNN), San Francisco, CA, 1993, pp. 586–591.
22. L. Devillers, H. Maynard, and S. Rosset et al., “The french media/evalda project: The evaluation of the understanding capability of spoken language dialog systems,” in Proc. Int. Conf. Lang. Resources Eval. (LREC), Lisbon, Portugal, May 2004.
23. S. Hahn, P. Lehnen, C. Raymond, and H. Ney, “A comparison of various methods for concept tagging for spoken language understanding,” in Proc. Int. Conf. Lang. Resources Eval. (LREC), Marrakech, Morocco, May 2008.
24. C. Rao and M. Rao, Matrix Algebra and Its Applications to Statistics and Econometrics. Singapaore: Word Scientific, 1998.
25. J. Renze and E. Weisstein, “Intermediate value theorem,” in Math-World—A Wolfram Web Resource [Online]. Available: http://math-world.wolfram.com/IntermediateValueTheorem.html 2010
26. T. Eisele, R. Haeb-Umbach, and D. Langmann, “A comparative study of linear feature transformation techniques for automatic speech recognition,” in Prof. Int. Conf. Spoken Lang. Process. (ICSLP), Philadelphia, PA, Oct. 1996.
27. J. Loof, C. Gollan, S. Hahn, G. Heigold, B. Hoffmeister, C. Plahl, D. Rybach, R. Schlüter, and H. Ney, “The RWTH 2007 TC-star evaluation system for european english and Spanish,” in Prof. Interspeech, Antwerp, Belgium, Aug. 2007.
28. B. Hoffmeister, C. Plahl, P. Fritz, G. Heigold, J. Loof, R. Schlüter, and H. Ney, “Development of the 2007 RWTH Mandarin LVCSR system,” in Prof. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Kyoto, Japan, Dec. 2007.