On prediction using variable order Markov models

Journal of Artificial Intelligence Research

Volumn 22, Issue , 2004, Pages 385-421

  Begleiter, Ron ^a   El Yaniv, Ran ^a   Yona, Golan ^b  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b Department of Computer Science and School of Operations Research and Information Engineering   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); DATA COMPRESSION; PROBLEM SOLVING; PROTEINS; TREES (MATHEMATICS);

CONTEXT TREE WEIGHTING (CTW); PREDICTION BY PARTIAL MATCH (PPM); PROBABILISTIC SUFFIX TREES (PSTS); PROTEIN CLASSIFICATION PROBLEMS;

MARKOV PROCESSES;

EID: 27344458404     PISSN: 10769757     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.1491     Document Type: Article

References (76)

1. Abe, N., & Warmuth, M. (1992). On the computational complexity of approximating probability distributions by probabilistic automata. Machine Learning, 9, 205-260.
2. Aberg, J., & Shtarkov, Y. (1997). Text compression by context tree weighting. In Proceedings Data Compression Conference (DCC), pp. 377-386. IEEE Computer Society Press.
3. Agrafiotis, D. (1997). A new method for analyzing protein sequence relationships based on sammon maps. Protein Science, 6, 287-293.
4. Arnold, R., & Bell, T. (1997). A corpus for the evaluation of lossless compression algorithms. In Designs, Codes and Cryptography, pp. 201-210.
5. Bar-Joseph, Z., El-Yaniv, R., Lischinski, D., & Werman, M. (2001). Texture mixing and texture movie synthesis using statistical learning. IEEE Transactions on Visualization and Computer Graphics, 7(2), 120-135.
6. Bejerano, G., & Yona, G. (2001). Variations on probabilistic suffix trees: Statistical modeling and the prediction of protein families.. Bioinformatics, 17(1), 23-43.
7. Bell, T., Cleary, J., & Witten, I. (1990). Text Compression. Prentice-Hall, Inc.
8. Bloom, C. (1998). Solving problems of context modeling, http://www.cbloom.com/papers/index.html.
9. Bunton, S. (1997). Semantically motivated improvements for PPM variants. The Computer Journal, 40(2/3), 76-92.
10. Burrows, M., & Wheeler, D. J. (1994). A block-sorting lossless data compression algorithm.. Tech. rep. 124, Digital Equipement Corporation.
11. CASP (2002). Protein structure prediction center, http:// predictioncenter.llnl.gov/.
12. Cheney, J. (2001). Compressing XML with multiplexed hierarchical PPM models. In Data Compression Conference, pp. 163-172.
13. Clarkson, B., & Pentland, A. (2001). Predicting daily behavior via wearable sensors. Tech. rep. Vismod TR#540, MIT.
14. Cleary, J., & Teahan, W. (1997). Unbounded length contexts for PPM. Computer Journal, 40, 67-75.
15. Cleary, J., & Witten, I. (1984). Data compression using adaptive coding and partial string matching. IEEE Transactions on Communications, COM-32(4), 396-402.
16. Cover, T., & King, R. (1978). A convergent gambling estimate of the entropy of English. IEEE Transactions on Information Theory, 24(4), 413-421.
17. Cover, T., & Thomas, J. (1991). Elements of Information Theory. John Wiley and Sons, Inc.
18. Dayhoff, M. (1976). The origin and evolution of protein superfamilies. In Federation Proceedings, pp. 2132-2138.
19. Deorowicz, S. (2003). Universal lossless data compression algorithms. Ph.D. thesis, Silesian University of Technology.
20. Ding, C., & Dubchak, I. (2001). Multi-class protein fold recognition using support vector machines and neural networks. Bioinformatics, 17, 349-358.
21. Drinic, M., & Kirovski, D. (2002). PPMexe: PPM for compressing software. In Data Compression Conference, pp. 192-201.
22. Dubnov, S., Assayag, G., Lartillot, O., & Bejerano, G. (2003). Using machine-learning methods for musical style modeling. IEEE Computer, 36(10), 73-80.
23. Feder, M., & Merhav, N. (1994). Relations between entropy and error probability. IEEE Transactions on Information Theory, 40(1), 259-266.
24. Holm, L., & Sander, C. (1994). Parser for protein folding units. Proteins, 19, 256-268.
25. Howard, P. (1993). The design and analysis of efficient lossless data compression systems. Ph.D. thesis, Brown University.
26. Jaakkola, T., Diekhans, M., & Haussler., D. (1999). Using the Fisher kernel method to detect remote protein homologies. In Proceedings of the Seventh International Conference on Intelligent Systems for Molecular Biology, pp. 149-158.
27. Jin, L., Fang, W., &: Tang, H. (2003). Prediction of protein structural classes by a new measure of information discrepancy. Computational Biology and Chemistry, 27, 373-380.
28. Kalai, A., Chen, S., Blum, A., & Rosenfeld, R. (1999). On-line algorithms for combining language models. In Proceedings of the International Conference on Accoustics, Speech, and Signal Processing.
29. Kermorvant, C., & Dupont, P. (2002). Improved smoothing for probabilistic suffix trees seen as variable order Markov chains. In European Conference on Machine Learning (ECML), pp. 185-194.
30. Krichevsky, R., & Trofimov, V. (1981). The performance of universal encoding. IEEE Transactions on Information Theory, 27, 199-207.
31. Kumarevel, T., Gromiha, M., & Ponnuswamy, M. (2000). Structural class prediction: An application of residue distribution along the sequence. Biophys Chem., 88, 81-101.
32. Langdon, G. (1983). A note on the Ziv-Lempel model for compressing individual sequences. IEEE Transactions on Information Theory, 29, 284- 287.
33. Lesk, A., & Rose, G. (1981). Folding units in globular proteins. In Proceedings of the National Academy of Sciences, No. 78, pp. 4304-4308.
34. Leslie, C., Eskin, E., Cohen, A., Weston, J., & Noble, W. (2004). Mismatch string kernels for discriminative protein classification. Bioinformatics, 20, 467-476.
35. Levitt, M., & Chothia, C. (1976). Structural patterns in globular proteins. Nature, 261, 552-558.
36. Manzini, G. (2001). An analysis of the burrows-wheeler transform. Journal of the ACM, 48(3), 407-430.
37. McAllester, D., &: Ortiz, L. (2003). Concentration inequalities for the missing mass and for histogram rule error. Journal of Machine Learning Research, 4, 895-911.
38. McCallum, A., Freitag, D., & Pereira, F. (2000). Maximum entropy Markov models for information extraction and segmentation. In Proc. 17th International Conf. on Machine Learning, pp. 591-598. Morgan Kaufmann, San Francisco, CA.
39. McGuffin, L., Bryson, K., & Jones, D. (2001). What are the baselines for protein fold recognition?. Bioinformatics, 17, 63-72.
40. Merhav, N., & Feder, M. (1995). A strong version of the redundancy-capacity theorem of universal coding. IEEE Transactions on Information Theory, 41(3), 714-722.
41. Miller, J., Goodman, R., & Smyth, P. (1993). On loss functions which minimize to conditional expected values and posterior probabilities. IEEE Transactions on Information Theory, 39(4:), 1404-1408.
42. Moffat, A. (1990). Implementing the PPM data compression scheme. IEEE Transactions on Communications, 38(11), 1917-1921.
43. Murzin, A., Brenner, S., Hubbard, T., & Chothia, C. (1995). SCOP: A structural classification of proteins database for the investigation of sequences and structures. Journal of Molecular Biology, 247, 536-540.
44. Nelson, M., & Gailly, J. (1996). The Data Compression Book (2nd ed.). MIS:Press.
45. Nevill-Manning, C., & Witten, I. (1999). Protein is incompressible. In Data Compression Conference, pp. 257-266.
46. Nisenson, M., Yariv, I., El-Yaniv, R., & Meir, R. (2003). Towards behaviometric security systems: Learning to identify a typist. In The 7th European Conference on Principles and Practice of Knowledge Discovery in Databases.
47. Orlitsky, A., Santhanam, N., & Zhang, J. (2003). Always Good Turing: Asymptotically optimal probability estimation. Science, 302(5644), 427-431.
48. Pachet, F. (2002). Playing with virtual musicians: The continuator in practice. IEEE MultiMedia, 9(3), 77-82.
49. Park, J., Karplus, K., Barrett, C., Hughey, R., Haussler, D., Hubbard, T., & Chothia, C. (1998). Sequence comparisons using multiple sequences detect three times as many remote homologues as pairwise methods. Journal of Molecular Biology, 284, 1201-1210.
50. Rabiner, L. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(3), 257-286.
51. Rabiner, L., & Juang, B. (1993). Fundamentals of Speech Recognition. Prentice-Hall, Inc.
52. Rissanen, J. (1983). A universal data compression system. IEEE Transactions on Information Theory, 29(5), 656-664.
53. Rissanen, J. (1984). Universal coding, information, prediction, and estimation. IEEE Transactions on Information Theory, 30(4), 629-636.
54. Rissanen, J., & Langdon, G. (1979). Arithmetic coding. IBM Journal of Research and Development, 23(2), 149-162.
55. Ron, D., Singer, Y., & Tishby, N. (1996). The power of amnesia: Learning probabilistic automata with variable memory length. Machine Learning, 25(2-3), 117-149.
56. Rose, G. (1979). Hierarchic organization of domains in globular proteins. Journal of Molecular Biology, 134, 447-470.
57. Sadakane, K., Okazaki, T., & Imai, H. (2000). Implementing the context tree weighting method for text compression. In Data Compression Conference, pp. 123-132.
58. Savari, S. (1997). Redundancy of the Lempel-Ziv incremental parsing rule. IEEE Transactions on Information Theory, 43, 9-21.
59. Schütze, H., & Singer, Y. (1994). Part-of-speech tagging using a variable memory Markov model. In Proceedings of the 32nd Conference on Association for Computational Linguistics, pp. 181-187. Association for Computational Linguistics.
60. Shkarin, D. (2002). PPM: One step to practicality. In Data Compression Conference, pp. 202-212.
61. Singer, Y. (1997). Adaptive mixtures of probabilistic transducers. Neural Computation, 9(8), 1711-1733.
62. Singer, Y., & Tishby, N. (1994). Dynamical encoding of cursive handwriting. Biological Cybernetics, 71(3), 227-237.
63. Sonnhammer, E., Eddy, S., & Durbin, R. (1997). Pfam: A comprehensive database of protein domain families based on seed alignments. Proteins, 28, 405-420.
64. Tatusov, R., Eugene, V., & David, J. (1997). A genomic perspective on protein families. Science, 278, 631-637.
65. Teahan, W., &. Harper, D. (2001). Using compression based language models for text categorization. In Workshop on Language Modeling and Information Retrieval, ARDA, Carnegie Mellon University, pp. 83-88.
66. Thaper, N. (2001). Using compression for source based classification of text. Master's thesis, Massachusetts Institute of Technology.
67. Tjalkens, T., Volf, P., & Willems, F. (1997). A context-tree weighting method for text generating sources. In Data Compression Conference, p. 472.
68. Tjalkens, T., & Willems, F. (1997). Implementing the context-tree weighting method: Arithmetic coding. In International Conference on Combinatorics, Information Theory and Statistics, p. 83.
69. Volf, P. (2002). Weighting Techniques in Data Compression Theory and Algorithms. Ph.D. thesis, Technische Universiteit Eindhoven.
70. Willems, F. (1998). The context-tree weighting method: Extensions. IEEE Transactions on Information Theory, 44(2), 792-798.
71. Willems, F., Shtarkov, Y., & Tjalkens, T. (1995). The context-tree weighting method: Basic properties. IEEE Transactions on Information Theory, 653-664.
72. Witten, I., & Bell, T. (1991). The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression. IEEE Transactions on Information Theory, 37(4), 1085-1094.
73. Witten, I., Bray, Z., Mahoui, M., & Teahan, B. (1999). Text mining: A new frontier for lossless compression. In Proceedings of the Conference on Data Compression, pp. 198-207. IEEE Computer Society.
74. Yona, G., & Levitt, M. (2002). Within the twilight zone: A sensitive profile-profile comparison tool based on information theory. Journal of Molecular Biology, 315, 1257-1275.
75. Yona, G., Linial, N., & Linial, M. (1999). Protomap: Automatic classification of protein sequences, a hierarchy of protein families, and local maps of the protein space. Proteins, 37, 360-378.
76. Ziv, J., & Lempel, A. (1978). Compression of individual sequences via variable-rate coding. IEEE Transactions on Information Theory, 24, 530-536.