Some observations on the concepts of information-theoretic entropy and randomness

Entropy

Volumn 3, Issue 1, 2001, Pages

  Smith, Jonathan D H ^a  

^a IOWA STATE UNIVERSITY   (United States)

Author keywords

Dimensional analysis; Gibbs Paradox; Information theoretic entropy; Kullback entropy; Martin Loef randomness; Second Law of Thermodynamics; Selection principle; Self delimiting algorithmic complexity; Shannon entropy; Thermodynamic entropy; Wave equation

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; ENTROPY; GIBBS FREE ENERGY; PROBABILITY; THERMODYNAMICS; WAVE EQUATIONS;

GIBBS PARADOX; INFORMATION-THEORETIC ENTROPY; KULLBACK ENTROPY; MARTIN-LOEF RANDOMNESS; SECOND LAW OF THERMODYNAMICS; SELECTION PRINCIPLE; SELF-DELIMITING ALGORITHMIC COMPLEXITY; SHANNON ENTROPY; THERMODYNAMIC ENTROPY; WAVE EQUATION;

INFORMATION THEORY;

EID: 3242784794     PISSN: 10994300     EISSN: None     Source Type: Journal    
DOI: 10.3390/e3010001     Document Type: Article

References (17)

1. Ash, R.B. (1965). Information Theory, Interscience, New York, NY.
2. Feynman, R.P., R.B. Leighton, and M. Sands (1963). The Feynman Lectures on Physics, Vol. I, Addison-Wesley, Reading, MA.
3. Garrett, A.J.M. (1991). Macroirreversibility and microreversibility reconciled, in Maximum Entropy in Action (eds. Buck, B. and V.A. Macaulay), 139, Clarendon Press, Oxford.
4. Jaynes, E.T. (1963). Information theory and statistical mechanics, in Statistical Physics, 1962 Brandeis Lectures (ed. Ford, K.W.), 181, Benjamin, New York, NY.
5. Kullback, S. (1959) Information Theory and Statistics, Wiley, New York, NY.
6. Li, M. and P. Vitanyi (1997) An Introduction to Kolmogorov Complexity and its Applications, Springer, New York, NY.
7. Martin-Löf, P. (1966). The definition of random sequences. Information and Control 9: 602.
8. O'Neill, E.L. (1963). Introduction to Statistical Optics, Addison-Wesley, Reading, MA.
9. Pauli, W. (1933). Handbuch der Physik xxiv/1: 151.
10. Prigogine, I. and C. George (1983). The Second Law as a selection principle: the microscopic theory of dissipative processes in quantum systems. Proc. Nat. Acad. Sci. 80: 4590.
11. Rumer, Yu.B. and M.Sh. Ryvkin (1977). Termodinamika, Statisticheskaya Fizika i Kinetika. English Translation (1980): Thermodynamics, Statistical Physics, and Kinetics, Mir, Moscow.
12. Rumer, Yu.B. and M.Sh. Ryvkin (1977). Termodinamika, Statisticheskaya Fizika i Kinetika. English Translation (1980): Thermodynamics, Statistical Physics, and Kinetics, Mir, Moscow.
13. Shannon, C.E. (1948). A mathematical theory of communication. Bell System Tech. J. 27: 379. Reprinted in Shannon, C.E. and W. Weaver (1949): The Mathematical Theory of Communication, University of Illinois Press, Urbana, IL.
14. Shannon, C.E. (1948). A mathematical theory of communication. Bell System Tech. J. 27: 379. Reprinted in Shannon, C.E. and W. Weaver (1949): The Mathematical Theory of Communication, University of Illinois Press, Urbana, IL.
15. Tolman, R.C. (1938). The Principles of Statistical Mechanics, Oxford University Press, Oxford.
16. Uspenskiǐ, V. A., A.L. Semenov, and A.Kh. Shen' (1990). Can an (individual) sequence of zeros and ones be random? (Russian) Uspekhi Mat. Nauk 45: 105. English Translation: Russian Mathematical Surveys 45: 121.
17. Uspenskiǐ, V. A., A.L. Semenov, and A.Kh. Shen' (1990). Can an (individual) sequence of zeros and ones be random? (Russian) Uspekhi Mat. Nauk 45: 105. English Translation: Russian Mathematical Surveys 45: 121.