Entropy, information, and computation

American Journal of Physics

Volumn 67, Issue 12, 1999, Pages 1074-1077

  Machta, J ^a  

^a UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0033269684     PISSN: 00029505     EISSN: None     Source Type: Journal    
DOI: 10.1119/1.19085     Document Type: Article

References (20)

1. L. Szilard, "On the decrease of entropy in a thermodynamic system by the intervention of intelligent beings," Z. Phys. 53, 840-856 (1929). Translation by A. Rapoport and M. Knoller, reprinted in Quantum Theory and Measurement, edited by J. A. Wheeler and W. H. Zurek (Princeton U. P., Princeton, 1983).
2. L. Szilard, "On the decrease of entropy in a thermodynamic system by the intervention of intelligent beings," Z. Phys. 53, 840-856 (1929). Translation by A. Rapoport and M. Knoller, reprinted in Quantum Theory and Measurement, edited by J. A. Wheeler and W. H. Zurek (Princeton U. P., Princeton, 1983).
3. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois Press, Urbana, 1949).
4. E. T. Jaynes, "Information theory and statistical mechanics," Phys. Rev. 106, 620-630 (1957).
5. L. Brillouin, Science and Information Theory (Academic, New York, 1956).
6. R. J. Solomonoff, "A formal theory of inductive inference, Parts I and II," Inf. Control. 7, 1-22; 7 224-254 (1964).
7. R. J. Solomonoff, "A formal theory of inductive inference, Parts I and II," Inf. Control. 7, 1-22; 7 224-254 (1964).
8. A. N. Kolmogorov, "Three approaches to the quantitative definition of information," Probl. Peredachi Inf. 1 (1), 1-7 (1965). English translation: Probl. Inf. Transm. 1, 1-7 (1965).
9. A. N. Kolmogorov, "Three approaches to the quantitative definition of information," Probl. Peredachi Inf. 1 (1), 1-7 (1965). English translation: Probl. Inf. Transm. 1, 1-7 (1965).
10. G. J. Chaitin, "On the length of programs for computing finite binary sequences," J. Assoc. Comput. Mach. 13, 547-569 (1966).
11. G. J. Chaitin, "Randomness and mathematical proof," Sci. Am. 232, No. 5, 46-52 (1975), available at http://www.cs.auckland.ac.nz/CDMTCS/ chaitin/sciamer.html.
12. C. H. Bennett, "The thermodynamics of computation - a review," Int. J. Theor. Phys. 21, 905-940 (1982).
13. W. H. Zurek, "Algorithmic randomness and physical entropy," Phys. Rev. A 40, 4731-4751 (1989).
14. W. H. Zurek, "Algorithmic information content, Church-Turing thesis, physical entropy, and Maxwell's Demon," in Complexity, Entropy and the Physics of Information, Santa Fe Institute Studies in the Sciences of Complexity, Vol. VIII, edited by W. H. Zurek (Addison-Wesley, Redwood, 1990), pp. 73-89.
15. W. T. Grandy, "Resource letter ITP-1: Information theory in physics," Am. J. Phys. 65, 466-476 (1997).
16. R. Baierlein, Atoms and Information Theory (Freeman, San Francisco, 1971).
17. A fundamental feature of algorithmic information is that there can be no algorithm that can be used to calculate the algorithmic information content of an arbitrary string of ones and zeros. This result is closely related to Gödel's incompleteness theorem; see Ref. 8.
18. A probability distribution is concisely defined if there is a short program to compute the probability of any string. Trivial probability distributions may not be concisely defined. For example, the distribution, p (s̃) = 1 for some particular string s̃ and zero otherwise, is not concisely defined if s has a high information content. The canonical ensemble of statistical mechanics is concisely defined by Eq. (5) and a procedure for calculating the energy.
19. R. Landauer, "Irreversibility and heat generation in the computing process," IBM J. Res. Dev. 5, 183-191 (1961).
20. R. Landauer, "Fundamental physical limitations of the computational process," in Computer Culture: The Scientific, Intellectual, and Social Impact of the Computer, edited by H. Pagels, Ann. (N.Y.) Acad. Sci. 426, 161-170 (1985).