A Theory of Program Size Formally Identical to Information Theory

Journal of the ACM (JACM)

Volumn 22, Issue 3, 1975, Pages 329-340

  Chaitin, Gregory J ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

computational complexity; entropy; information theory; instantaneous code; Kraft inequality; minimal program; probability theory; program size; random string; reeursive function theory; Turing machine

Indexed keywords

CODES, SYMBOLIC; INFORMATION THEORY;

COMPUTER METATHEORY;

EID: 0016532771     PISSN: 00045411     EISSN: 1557735X     Source Type: Journal    
DOI: 10.1145/321892.321894     Document Type: Article

References (33)

1. Solomonoff, R. J. A formal theory of inductive inference. Inform, and Contr. 7 (1964), 1-22, 224-254.
2. Kolmogorov, A. N. Three approaches to the quantitative definition of information. Problems of Inform. Transmission 1, 1 (Jan.-March 1965), 1-7.
3. Chaitin, G. J. On the length of programs for computing finite binary sequences. J. ACM 13, 4 (Oct. 1966), 547-569.
4. Chaitin, G. J. On the length of programs for computing finite binary sequences: Statistical considerations. J. ACM 16, 1 (Jan. 1969), 145-159.
5. Kolmogorov, A. N. On the logical foundations of information theory and probability theory. Problems of Inform. Transmission 5, 3 (July-Sept. 1969), 1-4.
6. Loveland, D. W. A variant of the Kolmogorov concept of complexity. Inform. and Contr. 15 (1969), 510-526.
7. Schnorr, C. P. Process complexity and effective random tests. J. Comput. and Syst. Scis. 7 (1973), 376-388.
8. Chaitin, G. J. On the difficulty of computations. IEEE Trans. IT-16 (1970), 5-9.
9. Feinstein, A. Foundations of Information Theory. McGraw-Hill, New York, 1958.
10. Fano, R. M. Transmission of Information. Wiley, New Nork, 1961.
11. Abramson, N. Information Theory and Coding. McGraw-Hill, New York, 1963.
12. Ash, R. Information Theory. Wiley-Interscience, New York, 1965.
13. Willis, D. G. Computational complexity and probability constructions. J. ACM 17, 2 (April 1970), 241-259.
14. Zvonkin, A. K., and Levin, L. A. The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms. Russ. Math. Survs. 25, 6 (Nov.-Dec. 1970), 83-124.
15. Cover, T. M. Universal gambling schemes and the complexity measures of Kolmogorov and Chaitin. Rep. No. 12, Statistics Dep., Stanford U., Stanford, Calif., 1974. Submitted to Ann. Statist.
16. Gewirtz, W. L. Investigations in the theory of descriptive complexity. Ph.D. Thesis, New York University, 1974 (to be published as a Courant Institute rep.).
17. Weiss, B. The isomorphism problem in ergodic theory. Bull. Amer. Math. Soc. 78 (1972), 668-684.
18. Rényi, A. Foundations of Probability. Holden-Day, San Francisco, 1970.
19. Fine, T. L. Theories of Probability: An Examination of Foundations. Academic Press, New York, 1973.
20. Cover, T. M. On determining the irrationality of the mean of a random variable. Ann. Statist. 1 (1973), 862-871.
21. Chaitin, G. J. Information-theoretic computational complexity. IEEE Trans. IT-20 (1974), 10-15.
22. Levin, M. Mathematical Logic for Computer Scientists. Rep. TR-131, M.I.T. Project MAC, 1974, pp. 145-147, 153.
23. Chaitin, G. J. Information-theoretic limitations of formal systems. J. ACM 21, 3 (July 1974), 403-424.
24. Minsky, M. L. Compulation: Finite and Infinite Machines. Prentice-Hall, Englewood Cliffs, N. J., 1967, pp. 54, 55, 66.
25. Minsky, M., and Papert, S. Perceptrons: An Introduction to Computational Geometry. M.I.T. Press, Cambridge, Mass., 1969, pp. 150, 153.
26. Schwartz, J. T. On Programming: An Interim Report on the SETL Project. Installment I: Generalities. Lecture Notes, Courant Institute, New York University, New York, 1973, pp. 1-20.
27. Bennett, C. H. Logical reversibility of computation. IBM J. Res. Develop. 17 (1973), 525-532.
28. Daley, R. P. The extent and density of sequences within the minimal-program complexity hierarchies. J. Comput. and Syst. Scis. (to appear).
29. Chaitin, G. J. Information-theoretic characterizations of recursive infinite strings. Submitted to Theoretical Comput. Sci.
30. Elias, P. Minimum times and memories needed to compute the values of a function. J. Comput. and Syst. Scis. (to appear).
31. Elias, P. Universal codeword sets and representations of the integers. IEEE Trans. IT (to appear).
32. Hellman, M. E. The information theoretic approach to cryptography. Center for Systems Research, Stanford U., Stanford. Calif., 1974.
33. Chaitin, G.J. Randomness and mathematical proof. Sci. Amer. 232, 5 (May 1975), in press.