On the memory complexity of the forward-backward algorithm

Pattern Recognition Letters

Volumn 31, Issue 2, 2010, Pages 91-99

  Khreich, Wael ^a   Granger, Eric ^a   Miri, Ali ^b   Sabourin, Robert ^a  

^a ÉCOLE DE TECHNOLOGIE SUPÉRIEURE   (Canada)

^b UNIVERSITY OF OTTAWA   (Canada)

Author keywords

Baum Welch; Complexity analysis; Forward Filtering Backward Smoothing; Forward backward; Hidden Markov Models

Indexed keywords

BACKWARD SMOOTHING; BAUM-WELCH; COMPLEXITY ANALYSIS; COMPUTATIONAL OVERHEADS; FORWARD BACKWARD ALGORITHMS; FORWARD FILTERING BACKWARD SMOOTHING; MEMORY COMPLEXITY; TIME COMPLEXITY;

ALGORITHMS; COMPUTATIONAL COMPLEXITY; COMPUTATIONAL EFFICIENCY; FLOATING BREAKWATERS; OBJECT RECOGNITION;

HIDDEN MARKOV MODELS;

EID: 71549124351     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2009.09.023     Document Type: Article

References (35)

1. Askar M., and Derin H. A recursive algorithm for the bayes solution of the smoothing problem. IEEE Trans. Automatic Control 26 2 (1981) 558-561
2. Baum, L.E., 1972. An inequality and associated maximization technique in statistical estimation for probabilistic functions of markov processes. In: Proc. 3rd Symposium Inequalities III, Univ. Calif., Los Angeles, Calif., 1969, 1-8 (dedicated to the memory of Theodore S. Motzkin).
3. Baum L.E., Petrie G.S., and Weiss N. A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Annal. Math. Statist. 41 1 (1970) 164-171
4. Baum L.E., and Petrie T. Statistical inference for probabilistic functions of finite state markov chains. Annal. Math. Statist. 37 (1966) 1554-1563
5. Cappe, O., 2001. Ten years of HMMs. .
6. Cappe, O., Moulines, E., 2005. Recursive computation of the score and observed information matrix in hidden markov models. In: IEEE/SP 13th Workshop on Statistical Signal Processing, 2005, pp. 703-708.
7. Chang R., and Hancock J. On receiver structures for channels having memory. IEEE Trans. Inform. Theory 12 4 (1966) 463-468
8. Churbanov, A., Winters-Hilt, S., 2008. Implementing em and viterbi algorithms for hidden markov model in linear memory. BMC Bioinform. 9 (224) (it has supplement file for formulas).
9. Coppersmith D., and Winograd S. Matrix multiplication via arithmetic progressions. J. Symbolic Comput. 9 (1990) 251-280
10. Dempster A., Laird N., and Rubin D. Maximum likelihood estimation from incomplete data via the EM algorithm. J. Roy. Statist. Soc., Ser. B 39 1 (1977) 1-38
11. Devijver P. Baum's forward-backward algorithm revisited. Pattern Recognition Lett. 3 (1985) 369-373
12. Elliott R.J., Aggoun L., and Moore J.B. Hidden Markov Models: Estimation and Control (1995), Springer-Verlag
13. Ephraim Y., and Merhav N. Hidden markov processes. IEEE Trans. Inform. Theory 48 6 (2002) 1518-1569
14. Florez-Larrahondo, G., 2005. Incremental learning of discrete hidden Markov models. Ph.D. Thesis, Mississippi State University, Department of Computer Science and Engineering.
15. Grice J., Hughey R., and Speck D. Reduced space sequence alignment. CABIOS 13 1 (1997) 45-53
16. Koenig S., and Simmons R.G. Unsupervised learning of probabilistic models for robot navigation. Proc. - IEEE Internat. Conf. on Robotics Automat. 3 (1996) 2301-2308
17. Krogh A., Brown M., Mian I.S., Sjolander K., and Haussler D. Hidden Markov-models in computational biology applications to protein modeling. J. Mol. Biol. 235 5 (1994) 1501-1531
18. Lane, T.D., 2000. Machine learning techniques for the computer security domain of anomaly detection. Ph.D. thesis, Purdue University, W. Lafayette, IN.
19. LeGland, F., Mevel, L., 1997. Recursive estimation in hidden markov models. In: 36th IEEE Conf. Decision Control, San Diego, CA, December.
20. Levinson S.E., Rabiner L.R., and Sondhi M.M. An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition. Bell System Tech. J. 62 (1983) 1035-1074
21. Li, Z., Evans, R., 1992. Optimal filtering, prediction and smoothing of hidden markov models. In: Proc. of the 31st IEEE Conf. on Decision and Control, vol. 4, 1992, pp. 3299-3304.
22. Lindgren G. Markov regime models for mixed distributions and switching regressions. Scandinavian J. Statist. 5 (1978) 81-91
23. Mann, T.P., 2006. Numerically stable hidden markov Model implementation, an HMM scaling tutorial, February.
24. Meyer I.M., and Durbin R. Gene structure conservation aids similarity based gene prediction. Nucleic Acids Res. 32 2 (2004) 776-783
25. Miklos I., and Meyer I.M. A linear memory algorithm for Baum-Welch training. BMC Bioinform. 6 (2005) 231
26. Narciso, T.L., 1993. Adaptive channel/code matching, Ph.D. thesis. University of Southern California, California, United States.
27. Ott G. Compact encoding of stationary markov sources. IEEE Trans. on Inform. Theory 13 1 (1967) 82-86
28. Rabiner L. A tutorial on Hidden Markov Models and selected applications in speech recognition. Proc. IEEE 77 2 (1989) 257-286
29. Raviv J. Decision making in markov chains applied to the problem of pattern recognition. IEEE Trans. Inform. Theory 13 4 (1967) 536-551
30. Shue L., Anderson D., and Dey S. Exponential stability of filters and smoothers for hidden markov models. IEEE Trans. Signal Process. 46 8 (1998) 2180-2194
31. Sivaprakasam, S., Shanmugan, S.K., 1995. A forward-only recursion based hmm for modeling burst errors in digital channels. In: IEEE Global Telecommunications Conference, vol. 2, GLOBECOM '95, 1995, pp. 1054-1058.
32. Tarnas C., and Hughey R. Reduced space hidden markov model training. Bioinformatics 14 (1998) 401-406
33. Wang W., Guan X.-H., and Zhang X.-L. Modeling program behaviors by hidden Markov models for intrusion detection. Proc. 2004 Internat. Conf. on Machine Learning Cybernet. 5 (2004) 2830-2835
34. Warrender, C., Forrest, S., Pearlmutter, B., 1999. Detecting intrusions using system calls: alternative data models. In: Proceedings of the IEEE Computer Society Symposium on Research in Security and Privacy, Oakland, CA, USA, pp. 133-145.
35. Yeung D.-Y., and Ding Y. Host-based intrusion detection using dynamic and static behavioral models. Pattern Recognition 36 1 (2003) 229-243