Classification of binary vectors by stochastic complexity

Journal of Multivariate Analysis

Volumn 63, Issue 1, 1997, Pages 47-72

  Gyllenberg, Mats ^a   Koski, Timo ^b   Verlaan, Martin ^c  

^a UNIVERSITY OF TURKU   (Finland)

^b ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^c DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

AUTOCLASS; Bacterial identification; Classification maximum likelihood estimate; Information content; Maximal predictive classification; Mixture of multivariate Bernouli distributions; Principle of maximum entropy; Universal source codes

Indexed keywords

EID: 0031256803     PISSN: 0047259X     EISSN: None     Source Type: Journal    
DOI: 10.1006/jmva.1997.1687     Document Type: Article

References (54)

1. 1. Agresti, A. (1990). Categorical Data Analysis. Wiley, New York.
2. 2. Andersen, E. B. J. (1991). The Statistical Analysis of Categorical Data, 2nd ed. Springer-Verlag, Berlin.
3. 3. Ash, R. B. (1990). Information Theory. Dover, New York.
4. 4. Bayesian Learning Group, (1996). Autoclass C - General Information, http://ic-www.arc. nasa.gov./ic/projects/bayes-group/autoclass/autoclass-c-program-html.
5. 5. Boulton, D. M., and Wallace, C. S. (1969). The information content of a multistate distribution. J. Theor. Biol. 23 269-278.
6. 6. Bryant, P. (1988). On characterizing optimization based clustering criteria. J. Class. 5 81-84.
7. 7. Bryant, P., and Williamson, J. A. (1978). Asymptotic behaviour of classification maximum likelihood estimates. Biometrika 65 273-281.
8. 8. Bryant, T. N. (1993). A compilation of probabilistic bacterial identification matrices. Binary 5 207-210.
9. 9. Celeux, G., and Govaert, G. (1991). Clustering criteria for discrete data and latent class models. J. Class. 8 157-176.
10. 10. Cover, T. M., and Thomas, J. A. (1991). Elements of Information Theory. Wiley, New York.
11. 11. Cheeseman, P. (1990). On finding the most probable model. In Computational Models of Discovery and Theory Formation (J. Shrager and P. Langley, Eds.), pp. 73-96. Morgan Kaufman, San Francisco.
12. 12. Cheeseman, P., and Stutz, J. (1997). Bayesian classification (AutoClass): Theory and results. In Advances in Knowledge Discovery and Data Mining (U. M. Fayyad, G. Piatetsky-Shapiro, P. Smith, and R. Uthurusamy, Eds.). AAAI Press, Menlo Park, CA.
13. 13. Csiszár, I., and Tusnády, G. (1984). Information geometry and alternating minimization procedures. Statistics and Decisions, Suppl. 1 205-237.
14. 14. Dawid, A. P. (1984). Statistical theory. Prequential approach. J. R. Statist. Soc. A 147 278-292.
15. 15. Duda, R. O., and Hart, P. E. (1973). Pattern Classification and Scene Analysis. Wiley, New York.
16. 16. Dybowski, W., and Franklin, D. A. (1965). Conditional probability and the identification of bacteria. J. Gen. Microbiol. 39 401-405.
17. 17. Farmer III, J. J., Davis, B. R., Hickman-Brenner, F. W., McWorther, A., Huntley-Carter, G. P., Asbury, M. A., Riddle, C., Wathen-Grady, H. G., Elias, C., Fanning, G. R., Steigwalt, A. G., O'Hara, C. M., Morris, G. K., Smith, P. B., and Brenner, D. J. (1985). Biochemical identification of new species and biogroups of Enterobacteriaceae isolated from clinical specimens. J. Clin. Microbiol. 21 46-76.
18. 18. Fitingoff, B. M. (1966). Optimal coding in the case of unknown and changing message statistics. Probl. of Inf. Trans. 2 1-7.
19. 19. Fitingoff, B. M. (1967). The compression of discrete information. Probl. Inf. Trans. 3 22-29.
20. 20. Gersho, A., and Gray, R. M. (1991). Vector Quantization and Signal Compression. Kluwer, Boston.
21. 21. Gilmour, J. S. L. (1937). A taxonomic problem. Nature 134 1040-1042.
22. 22. Gilmour, J. S. L. (1961). Taxonomy. In Contemporary Botanical Thought (A. M. Macleod and L. S. Cobley, Eds.), pp. 27-45. Oliver & Boyd, Chicago.
23. 23. Gower, J. C. (1973). Classification problems. Bull. Int. Stat. Inst. 45 471-477.
24. 24. Gower, J. C. (1974). Maximal predictive classification. Biometrics 30 643-654.
25. 25. Gray, R. M., Kieffer, J. C., and Linde, Y. (1980). Locally optimal block quantizer design. Inform. Control 45 178-198.
26. 26. Gyllenberg, H. G. (1976). Development of reference systems for automatic identification of clinical isolates of bacteria. Arch. Immunol. Therap. Exp. 24 1-19.
27. 27. Gyllenberg, H. G. (1981). Continuous cumulation of identification matrices. Helsingin Yliopiston Mikrobiologian Laitoksen Julkaisuja 20 1-35.
28. 28. Gyllenberg, H. G., and Niemelä, T. K. (1975). Basic principles in computer-assisted identification of microorganisms. In New Approaches to the Identification of Microorganisms (C-G. Héden and T. Illéni, Eds.), pp. 201-223. Wiley, New York.
29. 29. Gyllenberg, H. G., Gyllenberg, M., Koski, T., Lund, T., Schindler, J., and Verlaan, M. (1997). Classification of Enterobacteriaceae by minimization of stochastic complexity. Microbiology 143 721-732.
30. 30. Gyllenberg, M., Gyllenberg, H. G., Koski, T., and Schindler, J. (1993). Nonuniqueness of numerical taxonomic structures. Binary 5 138-144.
31. 31. Gyllenberg, M., and Koski, T. (1995). A taxonomic associative memory based on neural computation. Binary 7 61-66.
32. 32. Gyllenberg, M., and Koski, T. (1996). Numerical taxonomy and the principle of maximum entropy. J. Class. 13, No. 2.
33. 33. Gyllenberg, M., Koski, T., Reilink, E., and Verlaan, M. (1994). Nonuniqueness in probabilistic identification of bacteria. J. Appl. Probab. 31 542-548.
34. 34. Hanson, R., Stutz, J., and Cheeseman, P. (1991). Bayesian Classification Theory. Technical Report FIA-90-12-7-01, NASA Ames Research Center, Artificial Intelligence Branch.
35. 35. Jeffreys, H. (1948). Theory of Probability, 2nd ed. Clarendon Press, Oxford.
36. 36. Lapage, S. P., Bascomb, S., Willcox, W. R., and Curtis, M. A. (1973). Identification of bacteria by computer: general aspects and perspectives. J. Gen. Microbiol. 77 291-315.
37. 37. Lewis, P. M. (1959). Approximating probability distributions to reduce storage requirements. Inform. Control 2 214-225.
38. 38. Mackay, D. J. C. (1991). Bayesian interpolation. Neural Comput. 4 415-447.
39. 39. Milligan, G. W., and Cooper, M. C. (1985). An examination of procedures for determining the number of clusters in a data set. Psychom. 50 159-179.
40. 40. Niemelä, T. K., and Gyllenberg, H. G. (1975). Simulation of a computer-aided-self-correcting classification method. In Biological Identification with Computers (R. J. Pankhurst, Ed.), pp. 137-151. Academic Press, London.
41. 41. Oliver, J., and Baxter, R. (1994). MML and Bayesianism. Similarities and Differences. Technical Report 206, Department of Computer Science, Monash University.
42. 42. Oliver, J., and Baxter, R. (1995). MDL and MML. Similarities and Differences. Technical Report 207, Department of Computer Science, Monash University.
43. 43. Pankhurst, R. J. (1991). Practical Taxonomic Computing. Cambridge Univ. Press, Cambridge.
44. 44. Peck, R., Fisher, L., and Van Ness, J. (1989). Approximate confidence intervals for the number of clusters. JASA 84 184-191.
45. 45. Rissanen, J. (1987). Stochastic complexity. J. R. Stat. Soc. B 49 223-239.
46. 46. Rissanen, J. (1989). Stochastic Complexity in Statistical Inquiry. World Scientific, Singapore.
47. 47. Rissanen, J. (1990). Coding and complexity. In Sequences. Combinatorics, Compression, Security and Transmission (R. M. Capocelli, Ed.), pp. 312-326. Springer-Verlag, New York.
48. 48. Schwarz, G. (1978). Estimating the dimension of a model. Ann. Stat. 6 461-464.
49. 49. Sneath, P. H. A. (1995). The history and future potential of numerical concepts in systematics: The contributions of H. G. Gyllenberg. Binary 7 32-36.
50. 50. Wallace, C. S., and Boulton, D. M. (1968). Information measure for classification. Comput. J. 11 185-194.
51. 51. Wallace, C. S., and Freeman, P. R. (1987). Estimation and inference by compact coding. J. R. Stat. Soc. B 49 240-265.
52. 52. Watanabe, S. (1960). Information theoretical aspects of inductive and deductive inference. IBM J. Res. Dev. 4 208-231.
53. 53. Willcox, W. R., Lapage, S. P., and Holmes, B. (1980). A review of numerical methods in bacterial identification. Antonie van Leeuwenhoek 46 233-299.
54. 54. Windham, M. P. (1987). Parameter modification clustering criteria. J. Class. 4 191-214.