Comparative analysis of Viterbi Training and Maximum Likelihood estimation for HMMs

Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011

Volumn , Issue , 2011, Pages

  Allahverdyan, Armen ^a,c   Galstyan, Aram ^b  

^a YEREVAN PHYSICS INSTITUTE   (Armenia)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^c Laboratoire de Statistique et Processus   (France)

Author keywords

[No Author keywords available]

Indexed keywords

HIDDEN MARKOV MODELS; MAXIMUM LIKELIHOOD ESTIMATION; TRELLIS CODES;

COMPARATIVE ANALYZES; HIDDEN-MARKOV MODELS; MA XIMUM LIKELIHOODS; MAXIMUM LIKELIHOOD ESTIMATOR; MAXIMUM-LIKELIHOOD; MAXIMUM-LIKELIHOOD APPROACH; MAXIMUM-LIKELIHOOD ESTIMATION; OBSERVED DATA; PARAMETERS ESTIMATION; VITERBI;

ASYMPTOTIC ANALYSIS;

EID: 85162377681     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. A. E. Allahverdyan, Entropy of Hidden Markov Processes via Cycle Expansion, J. Stat. Phys. 133, 535 (2008).
2. A.E. Allahverdyan and A. Galstyan, On Maximum a Posteriori Estimation of Hidden Markov Processes, Proc. of UAI, (2009).
3. R. Artuso. E. Aurell and P. Cvitanovic, Recycling of strange sets, Nonlinearity 3, 325 (1990).
4. P. Baldi and S. Brunak, Bioinformatics, MIT Press, Cambridge, USA (2001).
5. L. E. Baum and T. Petrie, Statistical inference for probabilistic functions of finite state Markov chains, Ann. Math. Stat. 37, 1554 (1966).
6. J.M. Benedi, J.A. Sanchez, Estimation of stochastic context-free grammars and their use as language models, Comp. Speech and Lang. 19, pp. 249-274 (2005).
7. D. Blackwell and L. Koopmans, On the identifiability problem for functions of finite Markov chains, Ann. Math. Statist. 28, 1011 (1957).
8. S. B. Cohen and N. A. Smith, Viterbi training for PCFGs: Hardness results and competitiveness of uniform initialization, Procs. of ACL (2010).
9. Y. Ephraim and N. Merhav, Hidden Markov processes, IEEE Trans. Inf. Th., 48, 1518-1569, (2002).
10. L.Y. Goldsheid and G.A. Margulis, Lyapunov indices of a product of random matrices, Russ. Math. Surveys 44, 11 (1989).
11. R. N. Gutenkunst et al., Universally Sloppy Parameter Sensitivities in Systems Biology Models, PLoS Computational Biology, 3, 1871 (2007).
12. G. Han and B. Marcus, Analyticity of entropy rate of hidden Markov chains, IEEE Trans. Inf. Th., 52, 5251 (2006).
13. R. A. Horn and C. R. Johnson, Matrix Analysis (Cambridge University Press, New Jersey, USA, 1985).
14. H. Ito, S. Amari, and K. Kobayashi, Identifiability of Hidden Markov Information Sources, IEEE Trans. Inf. Th., 38, 324 (1992).
15. D. Janzing, On causally asymmetric versions of Occam's Razor and their relation to thermodynamics, arXiv:0708.3411 (2007).
16. B. H. Juang and L. R. Rabiner, The segmental k-means algorithm for estimating parameters of hidden Markov models, IEEE Trans. Acoustics, Speech, and Signal Processing, ASSP-38, no.9, pp.1639-1641, (1990).
17. B. G. Leroux, Maximum-Likelihood Estimation for Hidden Markov Models, Stochastic Processes and Their Applications, 40, 127 (1992).
18. N. Merhav and Y. Ephraim, Maximum likelihood hidden Markov modeling using a dominant sequence of states, IEEE Transactions on Signal Processing, vol.39, no.9, pp.2111-2115 (1991).
19. F. Qin, Restoration of single-channel currents using the segmental k-means method based on hidden Markov modeling, Biophys J 86, 14881501 (2004).
20. L. R. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition, Proc. IEEE, 77, 257 (1989).
21. L. J. Rodriguez and I. Torres, Comparative Study of the Baum-Welch and Viterbi Training Algorithms, Pattern Recognition and Image Analysis, Lecture Notes in Computer Science, 2652/2003, 847 (2003).
22. D. Ruelle, Statistical Mechanics, Thermodynamic Formalism, (Reading, MA: Addison-Wesley, 1978).
23. J. Sanchez, J. Benedi, F. Casacuberta, Comparison between the inside-outside algorithm and the Viterbi algorithm for stochastic context-free grammars, in Adv. in Struct. and Synt. Pattern Recognition (1996).
24. V. I. Spitkovsky, H. Alshawi, D. Jurafsky, and C. D. Manning, Viterbi Training Improves Unsupervised Dependency Parsing, in Proc. of the 14th Conference on Computational Natural Language Learning (2010).
25. A. Vaswani, A. Pauls, and D. Chiang, Efficient optimization of an MDL-inspired objective function for unsupervised part-of-speech tagging, in Proc. ACL (2010).