Hidden Markov processes: Theory and applications to biology

Hidden Markov Processes: Theory and Applications to Biology

Volumn , Issue , 2014, Pages 1-287

  Vidyasagar, M ^a  

^a UNIVERSITY OF TEXAS AT DALLAS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BIOINFORMATICS; COMPUTATION THEORY; INFORMATION THEORY; MAXIMUM LIKELIHOOD ESTIMATION; MOLECULAR BIOLOGY; VITERBI ALGORITHM;

BAUM-WELCH ALGORITHMS; BIOLOGICAL APPLICATIONS; HIDDEN MARKOV PROCESS; HITTING PROBABILITIES; LARGE DEVIATION THEORY; MARKOV CHAIN MONTE CARLO; PERRON-FROBENIUS THEOREM; PROFILE HIDDEN MARKOV MODEL;

HIDDEN MARKOV MODELS;

EID: 84937390640     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Book

References (145)

1. N. Alon, M. Krivelevich, I. Newman, and M. Szegedy, "Regular languages are testable with a constant number of queries," Foundations of Computer Sciences, IEEE, New York, 645-655, 1999.
2. N. Alon, M. Krivelevich, I. Newman, and M. Szegedy, "Regular languages are testable with a constant number of queries," SIAM Journal on Computing, 30, 1842-1862, 2001.
3. S. F. Altschul, "The statistics of similarity sequence scores," http://www.ncbi.nlm.nih.gov/BLAST/tutorial/Altschul-1.html.
4. S. F. Altschul, W. Gish, W. Q. Miller, E. W. Myers, and D. J. Lipman, "Basic local alignment search tool," Journal of Molecular Biology, 215, 403-410, 1990.
5. S. F. Altschul, T. L. Madden, A. A. Schafier, J. Zhang, Z. Zhang, W. Q. Miller, and D. J. Lipman, "Gapped BLAST and PSI-BLAST: A new generation of protein database search programs," Nucleic Acids Research, 25, 3389-3401, 1997.
6. B. D. O. Anderson, "The realization problem for hidden Markov models," Mathematics of Control, Signals, and Systems, 12(1), 80-120, 1999.
7. B. D. O. Anderson, "From Wiener to hidden Markov models," IEEE Control Systems Magazine, 19(6), 41-51, June 1999.
8. B. D. O. Anderson, M. Deistler, L. Farina, and L. Benvenuti, "Nonnegative realization of a system with a nonnegative impulse response," IEEE Transactions on Circuits and Systems-I: Fundamental Theory and Applications, 43, 134-142, 1996.
9. L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal decoding of linear codes for minimizing symbol error rate," IEEE Transactions on Information Theory, 20(2), 284-287, March 1974.
10. L. R. Bahl and F. Jelinek, "Decoding for channels with insertions, deletions and substitutions with applications to speech recognition," IEEE Transactions on Information Theory, 21(4), 404-411, July 1975.
11. P. Baldi and S. Brunak, Bioinformatics: A Machine Learning Approach (Second Edition), MIT Press, Cambridge, MA, 2001.
12. L. E. Baum and T. Petrie, "Statistical inference for probabilistic functions of finite state Markov chains," Annals of Mathematical Statistics, 37(2), 1554-1563, 1966.
13. L. E. Baum, T. Petrie, G. Soules, and N. Weiss, "A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains," Annals of Mathematical Statistics, 41(1), 164-171, 1970.
14. R. Begleiter, R. El-Yaniv, and G. Yona, "On prediction using variable order Markov models," Journal of Artificial Intelligence Research, 22, 385-421, 2004.
15. L. Benvenuti and L. Farina, "A tutorial on the positive realization problem," IEEE Transactions on Automatic Control, 49, 651-664, 2004.
16. A. Berman and R. J. Plemmons, Nonnegative Matrices, Academic Press, New York, 1979.
17. E. Bianconi, A. Provesan, F. Facchin, et al., "An estimation of the number of cells in the human body," Annals of Human Biology, 40(6), 463-471, November-December 2013.
18. P. Billingsley, Probability and Measure, Wiley, New York, 1986.
19. P. Billingsley, Probability and Measure (Third Edition), Wiley, New York, 1995.
20. D. Blackwell and L. Koopmans, "On the identifiability problem for functions of finite Markov chains," Annals of Mathematical Statistics, 28, 1011-1015, 1957.
21. V. Blondel and V. Catarini, "Undecidable problems for probabilistic automata of fixed dimension," Theory of Computing Systems, 36, 231-245, 2003.
22. L. Breiman, "The individual ergodic theorem of information theory," Annals of Mathematical Statistics, 28, 809-811, 1957.
23. L. Breiman, Probability, SIAM Classics in Applied Mathematics, No. 7, SIAM, Philadelphia, 1992.
24. T. Brown Jr., Nucleic Acids Book, available free at http://www.atdbio.com/nucleic-acids-book.
25. C. Burge and S. Karlin, "Prediction of complete gene structure in human genomic DNA," Journal of Molecular Biology, 268, 78-94, 1997.
26. C. Burge and S. Karlin, "Finding the genes in genomic DNA," Current Opinion in Structural Biology, 8(3), 346-354, 1998.
27. J. W. Carlyle, "Identification of state-calculable functions of finite Markov chains," Annals of Mathematical Statistics, 38, 201-205, 1967.
28. J. W. Carlyle, "Stochastic finite-state system theory," Chapter 10, in System Theory, L. Zadeh and E. Polak (Eds.), McGraw-Hill, New York, 1969.
29. J.-R. Chazottes and D. Gabrielli, "Large deviations for empirical entropies of g-measures," Nonlinearity, 18(6), 2545-2563, November 2005.
30. T. M. Cover and J. A. Thomas, Elements of Information Theory (Second Edition), Wiley Interscience, New York, 2006.
31. F. Crick, What Mad Pursuit, Basic Books, New York, 1988.
32. I. Csiszár, "The method of types," IEEE Transactions on Information Theory, 4(6), 2505-2523, 1998.
33. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Academic Press, New York, 1981.
34. L. D. Davisson, G. Longo, and A. Sgarro, "The error exponent for the noiseless encoding of finite ergodic Markov sources," IEEE Transactions on Information Theory, 27(4), 431-438, July 1981.
35. A. L. Delcher, K. A. Bratke, E. C. Powers, and S. L. Salzberg, "Identifying bacterial genes and endosymbiont DNA with Glimmer," Bioinformatics, 23(6), 673-679, 2007.
36. A. L. Delcher, D. Harmon, S. Kasif, O. White, and S. L. Salzberg, "Improved microbial gene identification with GLIMMER," Nucleic Acids Research, 27(23), 4636-4641, 1999.
37. A. Dembo and S. Karlin, "Strong limit theorems of empirical functionals for large exceedances of partial sums of i.i.d. variables," Annals of Probability, 19(4), 1737-1755, 1991.
38. A. Dembo and S. Karlin, "Strong limit theorems of empirical functionals for large exceedances of partial sums of Markov variables," Annals of Probability, 19(4), 1756-1767, 1991.
39. A. Dembo, S. Karlin, and O. Zeitouni, "Critical phenomena for sequence matching with scoring," Annals of Probability, 22(4), 1993-2021, 1994.
40. A. Dembo, S. Karlin, and O. Zeitouni, "Limit distribution of maximal non-aligned two-sequence segmental score," Annals of Probability, 22(4), 2022-2039, 1994.
41. A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, Springer-Verlag, Berlin, 1998.
42. L. Devroye, L. Gyorfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition, Springer-Verlag, Berlin, 1996.
43. S. W. Dharmadhikari, "Functions of finite Markov chains," Annals of Mathematical Statistics, 34, 1022-1031, 1963.
44. S. W. Dharmadhikari, "Suficient conditions for a stationary process to be a function of a Markov chain," Annals of Mathematical Statistics, 34, 1033-1041, 1963.
45. S. W. Dharmadhikari, "A characterization of a class of functions of finite Markov chains," Annals of Mathematical Statistics, 36, 524-528, 1965.
46. S. W. Dharmadhikari, "A note on exchangeable processes with states of finite rank," Annals of Mathematical Statistics, 40(6), 2207-2208, 1969.
47. S. W. Dharmadhikari and M. G. Nadkarni, "Some regular and non-regular functions of finite Markov chains," Annals of Mathematical Statistics, 41(1), 207-213, 1970.
48. J. Dixmier, "Proof of a conjecture by Erdös and Graham concerning the problem of Frobenius," Journal of Number Theory, 34, 198-209, 1990.
49. S. R. Eddy, "Profile hidden Markov models," Bioinformatics, 14(9), 755-763, 1998.
50. Y. Ephraim and N. Merhav, "Hidden Markov processes," IEEE Transactions on Information Theory, 48(6), 1518-1569, June 2002.
51. R. V. Erickson, "Functions of Markov chains," Annals of Mathematical Statistics, 41, 843-850, 1970.
52. W. J. Ewens and G. R. Grant, Statistical Methods in Bioinformatics, Springer-Verlag, New York, 2001.
53. W. J. Ewens and G. R. Grant, Statistical Methods in Bioinformatics (Second Edition), Springer-Verlag, New York, 2006.
54. J. Feng and T. G. Kurtz, Large Deviations for Stochastic Processes, American Mathematical Society, Providence, RI, 2006.
55. M. Fliess, "Series rationelles positives et processus stochastique," Annales de l'Institut Henri Poincarfie, Section B, XI, 1-21, 1975.
56. M. Fox and H. Rubin, "Functions of processes with Markovian states," Annals of Mathematical Statistics, 39, 938-946, 1968.
57. G. Frobenius, "Über Matrizen aus positiven Elementen," S.-B. Preuss. Akad. Wiss. (Berlin), 471-476, 1908.
58. G. Frobenius, "Über Matrizen aus positiven Elementen," S.-B. Preuss. Akad. Wiss. (Berlin), 514-518, 1909.
59. G. Frobenius, "Über Matrizen aus nicht negativen Elementen," S.-B. Preuss. Akad. Wiss. (Berlin), 456-477, 1912.
60. E. J. Gilbert, "The identiábility problem for functions of Markov chains," Annals of Mathematical Statistics, 30, 688-697, 1959.
61. B. V. Gnedenko, The Theory of Probability (Fourth Edition), Chelsea, New York, 1968.
62. R. M. Gray, Probability, Random Processes, and Ergodic Properties, Springer-Verlag, New York, 1988.
63. D. Gusfield, Algorithms on Strings, Trees and Sequences, Cambridge University Press, Cambridge, UK, 1997.
64. A. Heller, "On stochastic processes derived from Markov chains," Annals of Mathematics, 36, 1286-1291, 1965.
65. W. Hoefiding, "Probability inequalities for sums of bounded random variables," Journal of the American Statistical Association, 58, 13-30, 1963.
66. J. M. van den Hof, "Realization of continuous-time positive linear systems," Systems and Control Letters, 31, 243-253, 1997.
67. J. M. van den Hof and J. H. van Schuppen, "Realization of positive linear systems using polyhedral cones," Proceedings of the 33rd IEEE Conference on Decision and Control, 3889-3893, 1994.
68. F. Den Hollander, Large Deviations, American Mathematical Society, Providence, RI, 2000.
69. International Human Genome Research Consortium, "Initial sequencing and analysis of the human genome," Nature, 409, 860-921, February 2001.
70. H. Ito, S. Amari, and K. Kobayashi, "Identiábility of hidden Markov information sources and their minimum degrees of freedom," IEEE Transactions on Information Theory, 38, 324-333, 1992.
71. F. Jelinek, Statistical Methods for Speech Recognition, MIT Press, Cambridge, MA, 1997.
72. B-H. Juang and L. R. Rabiner, "A probabilistic distance measure for hidden Markov models," AT&T Technical Journal, 64(2), 391-408, February 1985.
73. S. Kalikow, "Random Markov processes and uniform martingales," Israel Journal of Mathematics, 71(1), 33-54, 1990.
74. S. Karlin and S. F. Altschul, "Methods for assessing the statistical significance of molecular sequence features by using general scoring schemes," Proceedings of the National Academy of Sciences, 87, 2264-2268, 1990.
75. S. Karlin and S. F. Altschul, "Applications and statistics for multiple high-scoring segments in molecular sequences," Proceedings of the National Academy of Sciences, 90, 5873-5877, 1993.
76. S. Karlin and A. Dembo, "Limit distributions of maximal segmental score among Markov-dependent partial sums," Advances in Applied Probability, 24(1), 113-140, 1992.
77. S. Karlin and F. Ost, "Maximal length of common words among random letter sequences," Annals of Probability, 16, 535-563, 1988.
78. S. Karlin and H. Taylor, A First Course in Stochastic Processes, Academic Press, New York, 1975.
79. A. I. Khinchin, Mathematical Foundations of Information Theory, Dover, New York, 1957.
80. H. G. Khorana, H. Büchi, H. Ghosh, et al., "Polynucleotide synthesis and the genetic code," Cold Spring Harbor Symposia on Quantitative Biology, 31, 39-49, 1966.
81. A. N. Kolmogorov, Foundations of the Theory of Probability, Chelsea, New York, 1950.
82. T. Koski, Hidden Markov Models for Bioinformatics, Kluwer, Dordrecht, The Netherlands, 2001.
83. A. Krogh, "An introduction to hidden Markov models for biological sequences," in Computational Methods in Molecular Biology, S. Salzberg, D. Searls, and S. Kasif (Eds.), Elsevier, New York, 45-63, 1998.
84. A. Krogh, M. Brown, I. S. Mian, K. Sjölander, and D. Haussler, "Hidden Markov models in computational biology: Applications to protein modeling," Journal of Molecular Biology, 235, 1501-1531, 1994.
85. A. Krogh, I. S. Mian, and D. Haussler, "A hidden Markov model that finds genes in E. coli DNA," Nucleic Acids Research, 22(22), 4768-4778, 1994.
86. L. Kronecker, "Zur Theorie der Elimination einer Variablen aus zwei algebraischen Gleichungen," Monatsber. Königl. Preuss. Akad. Wiss. Berlin, 535-600, 1881.
87. S. Kullback and R. A. Leibler, "On information and suff-ciency," Annals of Mathematical Statistics, 22, 79-86, 1951.
88. G. Langdon, "A note on the Ziv-Lempel model for compressing individual data sequences," IEEE Transactions on Information Theory, 29, 284-287, 1983.
89. M. Lewin, "A bound for a solution of a linear Diophantine problem," Journal of the London Mathematical Society (2), 6, 61-69, 1972.
90. F. Liese and I. Vajda, "On divergences and informations in statistics and information theory," IEEE Transactions on Information Theory, 52(10), 4394-4412, October 2006.
91. R. Lohrmann, D. Söll, H. Hayatsu, E. Ohtsuka, and H. G. Khorana, "Studies on polynucleotides. LI. Syntheses of the 64 possible ribotrinucleotides derived from the four major ribomononucleotides," Journal of the American Chemical Society, 88(4), 819-829, 1966.
92. A. R. Morgan, R. D. Wells, and H. G. Khorana, "Studies on polynucleotides, lix. Further codon assignments from amino acid incorporations directed by ribopolynucleotides containing trinucleotide sequences," Proceedings of the National Academy of Sciences, 56(6) , 1899-1906, 1966.
93. A. V. Lukashin and M. Borodvsky, "GeneMark.hmm: New solutions for gene finding," Nucleic Acids Research, 26(4), 1107- 1115, 1998.
94. W. H. Majoros and S. L. Salzberg, "An empirical analysis of training protocols for probabilistic gene finders," BMC Bioinformatics, available at http://www.biomedcentral.com/1471-2105/5/206, 21 December 2004.
95. P. Massart, "The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality," Annals of Probability, 18(3), 1269-1283, 1990.
96. B. McMillan, "The basic theorems of information theory," Annals of Mathematical Statistics, 24(2), 196-219, 1953.
97. D. W. Mount, Bioinformatics: Sequence and Genome Analysis, Cold Spring Harbor Press, Cold Spring Harbor, NY, 2001.
98. S. B. Needleman and C. D. Wunsch, "A general method applicable to the search for similarities in the amino acid sequences of two proteins," Journal of Molecular Biology, 48(3), 443-453, 1970.
99. M. Nirenberg and P. Leder, "RNA codewords and protein synthesis," Science, 145(3639), 1399-1407, 25 September 1964.
100. J. R. Norris, Markov Chains, Cambridge University Press, Cambridge, UK, 1997.
101. G. Nuel, "LD-SPatt: Large deviations statistics for patterns on Markov chains," Journal of Computational Biology, 11(6), 1023-1033, 2004.
102. D. S. Ornstein and B. Weiss, "How sampling reveals a process," Annals of Probability, 18(3), 905-930, 1990.
103. O. Perron, "Zur Theorie der Matrizen," Mathematische Annalen, 64, 248-263, 1907.
104. M. Pertea, X. Lin, and S. L. Salzberg, "GeneSplicer: A new computational method for splice site detection," Nucleic Acids Research, 29(5), 1185-1190, 2001.
105. P. A. Pevzner, Computational Molecular Biology, MIT Press, Cambridge, MA, 2000.
106. G. Picci, "On the internal structure of finite-state stochastic processes," in Recent Developments in Variable Structure Systems, R. Mohler and A. Ruberti (Eds.), Lecture Notes in Economics and Mathematical Systems, No. 162, Springer-Verlag, Heidelberg, 1978.
107. D. Pollard, Convergence of Stochastic Processes, Springer-Verlag, Berlin, 1984.
108. L. W. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proceedings of the IEEE, 77(2), 257-285, February 1989.
109. Z. Rached, F. Alalaji, and L. L. Campbell, "The Kullback-Leibler divergence rate between Markov sources," IEEE Transactions on Information Theory, 50(5), 917-921, May 2004.
110. R. T. Rockafellar, Convexity, Princeton University Press, Princeton, NJ, 1970.
111. G. Rozenberg and A. Salomaa, Cornerstones in Undecidability, Prentice Hall, Englewood Clifis, NJ, 1994.
112. W. Rudin, Real and Complex Analysis, McGraw-Hill, New York, 1959.
113. S. L. Salzberg, A. L. Delcher, S. Kasif, and O. White, "Microbial gene identification using interpolated Markov models," Nucleic Acids Research, 26(2), 544-548, 1998.
114. S. L. Salzberg, M. Pertea, A. L. Delcher, M. J. Gardner, and H. Tettelin, "Interpolated Markov models for eukaryotic gene finding," Genomics, 59, 24-31, 1999.
115. A. Sayre, Rosalind Franklin and DNA, Norton, New York, 2000 (reissue of 1975 edition).
116. E. Seneta, Non-negative Matrices and Markov Chains (Second Edition), Springer-Verlag, Berlin, 1981.
117. C. E. Shannon, "A mathematical theory of communication," Bell System Technical Journal, 27, 379-423, 623-656, July, October 1948.
118. P. C. Shields, The Ergodic Theory of Discrete Sample Paths, Amer. Math. Soc., Providence, RI, 1996.
119. T. F. Smith and M. S. Waterman, "Identification of common molecular subsequences," Journal of Molecular Biology, 147, 195-197, 1981.
120. E. D. Sontag, "On certain questions of rationality and decidability," Journal of Computer and System Science, 11, 375-381, 1975.
121. D. W. Stroock, An Introduction to Markov Processes, Springer-Verlag, Berlin, 2005.
122. B. E. Suzek, M. D. Ermolaeva, M. Schreiber, and S. L. Salzberg, "A probabilistic method for identifying start codons in bacterial genomes," Bioinformatics, 17(12), 1123-1130, 2001.
123. F. Topsoe, "Bounds for entropy and divergence over a twoelement set," Journal of Inequalities in Pure and Appllied Mathematics, 2(2), 2001.
124. V. N. Vapnik, Statistical Learning Theory, John Wiley, New York, 1998.
125. J. C. Venter, M. D. Adams, E. W. Myers, et al., "The sequence of the human genome," Science, 291, 1304-1351, 16 February 2001.
126. M. Vidyasagar, A Theory of Learning and Generalization, Springer-Verlag, London, 1997.
127. M. Vidyasagar, Learning and Generalization with Applications to Neural Networks, Springer-Verlag, London, 2003.
128. M. Vidyasagar, Nonlinear Systems Analysis, SIAM Publications, Philadelphia, PA, 2003.
129. M. Vidyasagar, "Convergence of empirical means with alphamixing input sequences, and an application to PAC learning," Proceedings of the IEEE Conference on Decision and Control, Seville, Spain, 2005.
130. M. Vidyasagar, "The realization problem for hidden Markov models: The complete realization problem," Proceedings of the IEEE Conference on Decision and Control, Seville, Spain, 2005.
131. M. Vidyasagar, "The 4M (Mixed Memory Markov Model) algorithm for stochastic modelling over a finite alphabet with applications to gene-finding algorithms," Proceedings of the Symposium on the Mathematical Theory of Networks and Systems, Kyoto, Japan, 443-459, July 2006.
132. M. Vidyasagar, "A realization theory for hidden Markov models: The partial realization problem," Proceedings of the Symposium on the Mathematical Theory of Networks and Systems, Kyoto, Japan, 2145-2150, July 2006.
133. M. Vidyasagar, "The complete realization problem for hidden Markov models: A survey and some new results," Mathematics of Control, Signals and Systems, 23(1), 1-65, 2011.
134. M. Vidyasagar, S. S. Mande, Ch. V. S. K. Reddy, and V. R. Raja Rao, "The 4M (mixed memory Markov model) algorithm for finding genes in prokaryotic genomes," IEEE Transactions on Automatic Control, 53(1), 26-27, January 2008.
135. M. Vidyasagar, "An elementary derivation of the large deviation rate function for finite state Markov processes," Asian Journal of Control, 16(1), 1-19, January 2014.
136. J. D. Watson, The Double Helix: A Personal Account of the Discovery of the Structure of DNA, Touchstone, New York, 2001 (reprint of 1968 edition).
137. L. R. Welch, "Hidden Markov models and the Baum-Welch algorithm," IEEE Information Theory Society Newsletter, 53(4), 1-24, December 2003.
138. T. A. Welch, "A technique for high performance data compression," IEEE Computer, 17(6), 8-19, 1984.
139. H. Wielandt, "Unzerlegbare, nicht negativen Matrizen," Mathematische Zeitschrift, 52, 642-648, 1950.
140. M. Wilkins, The Third Man of the Double Helix, Oxford University Press, Oxford, UK, 2003.
141. L. Xie, A. Ugrinovskii, and I. R. Petersen, "Probabilistic distances between finite-state finite-alphabet hidden Markov models," IEEE Transactions on Automatic Control, 50(4), 505-511, April 2005.
142. B-J. Yoon, "Hidden Markov models and their applications in biological sequence analysis," Current Genomics, 10, 402-415, 2009.
143. M. Q. Zhang, "Computational prediction of eukaryotic proteincoding genes," Nature Reviews, 3, 698-710, September 2002.
144. J. Ziv and A. Lempel, "A universal algorithm for sequential data compression," IEEE Transactions on Information Theory, 23(3), 337-343, 1977.
145. J. Ziv and A. Lempel, "Compression of sequences via variable rate encoding," IEEE Transactions on Information Theory, 24(5), 530-536, 1978.