Relative entropy: Free energy associated with equilibrium fluctuations and nonequilibrium deviations

Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics

Volumn 63, Issue 4, 2001, Pages

  Qian, Hong ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CHEMICAL RELAXATION; FREE ENERGY; MACROMOLECULES; MARKOV PROCESSES; MATHEMATICAL MODELS; PHASE EQUILIBRIA; PROBABILITY DISTRIBUTIONS; SOLUTIONS; STATISTICAL MECHANICS;

EQUILIBRIUM FLUCTUATIONS;

ENTROPY;

EID: 0035305311     PISSN: 1063651X     EISSN: None     Source Type: Journal    
DOI: 10.1103/PhysRevE.63.042103     Document Type: Article

References (27)

1. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois Press, Urbana, IL, 1963);
2. A. I. Khinchin, Mathematical Foundations of Information Theory (Dover, New York, 1957).
3. J. Rothstein, Science 114, 171 (1951).
4. We provide this information entropy formula with a Boltzmannian interpretation based on his celebrated relation (Formula presented)This formula was originally proposed for a uniform probability distribution (microcanonical ensemble, Boltzmann’s framework for statistical mechanics). To generalize it to a situation with nonuniform probability distribution (canonical ensemble, Gibbs and Smoluchowski’s framework) (Formula presented), let us consider a sequence of N independent and identically distributed random variables (Formula presented) all with the distribution (Formula presented) Then the probability of a particular sequence is (Formula presented), where (Formula presented) is the number of occurrences of (Formula presented) in the sequence. For large N, there are essentially two types of sequences: a typical sequence has (Formula presented) and all the remaining sequences are rare. The probability of a typical sequence is (Formula presented) where S is given in Eq. (1). The number of the typical sequences is (Formula presented), which among all the possible sequences (Formula presented) is essentially zero: (Formula presented). Therefore, for a large N, most sequences are rare; however, almost certainly none will occur. The relevant probability is defined on the space of all typical sequences, with the number of equal possible outcomes being (Formula presented). Therefore, by Boltzmann’s relation we have (Formula presented)
5. K. Huang, Statistical Mechanics (Wiley, New York, 1963);
6. A. I. Khinchin, Mathematical Foundations of Statistical Mechanics (Dover, New York, 1960).
7. J. Schnakenberg, Rev. Mod. Phys. 48, 571 (1976).
8. H. Risken, The Fokker-Planck Equation: Methods of Solution and Applications (Springer-Verlag, New York, 1984);
9. A. Lasota and M. C. Mackey, Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, 2nd ed. (Springer-Verlag, New York, 1994).
10. H. Qian, LANL e-print Physics/0007017.
11. R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics (Springer-Verlag, New York, 1985).
12. C.-L. Bai, C. Wang, X. S. Xie, and P. G. Wolynes, Proc. Natl. Acad. Sci. U.S.A. 96, 11 075 (1999).
13. B. C. Eu, J. Chem. Phys. 106, 2388 (1997);Kinetic Theory and Irreversible Thermodynamics (Wiley, New York, 1979).
14. R. J. Dorfman, Introduction to Chaos in Nonequilibrium Statistical Mechanics (Cambridge University Press, New York, 1999).
15. An entropy production rate has been introduced in the study of fluid mechanics in terms of dynamical systems [D. Ruelle, J. Stat. Phys. 85, 1 (1996)].
16. It was recently shown that this entropy production is intimately related to the entropy production in the Smoluchowski’s framework, and relative entropy [D.-q. Jiang, M. Qian, and M.-p. Qian, Commun. Math. Phys. 214, 389 (2000)].
17. H. Qian and B. E. Shapiro, Proteins: Struct., Funct., Genet. 37, 576 (1999).
18. H. Takahashi, Proc. Phys. Math. Soc. Jpn. 24, 60 (1942).
19. M. Kac, Phys. Fluids 2, 8 (1959).
20. H. Qian and E. L. Elson, Biophys. J. 76, 1598 (1999);
21. Biophys. J.H. Qian, 79, 137 (2000);
22. H. QianJ. Math. Biol. 41, 331 (2000).
23. Mathematically speaking, both this result and that in Ref. 2 are parts of Sanov’s theorem on large deviations for finite alphabets [see Large Deviation Techniques and Applications (Ref. 17)]. Realizing the mathematical nature of and applying the probabilistic method to this problem, though in a less explicit manner, dates back to Khinchin [Mathematical Foundations of Statistical Mechanics (Ref. 3)].
24. A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, 2nd ed. (Springer-Verlag, New York, 1998).
25. M.-P. Qian, M. Qian, and G. L. Gong, Contemp. Math. 118, 255 (1991).
26. H. Qian, Phys. Rev. Lett. 81, 3063 (1998).
27. L. Onsager, Phys. Rev. 37, 405 (1931).