Data processing theorems and the second law of thermodynamics

IEEE Transactions on Information Theory

Volumn 57, Issue 8, 2011, Pages 4926-4939

  Merhav, Neri ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Convexity; data processing inequality; detailed balance; H theorem; perspective function; thermodynamics

Indexed keywords

BOLTZMANN; CONVEX FUNCTIONS; CONVEXITY; DETAILED BALANCE; FINITE-STATE; FUNCTION OF TIME; GENERALIZED INFORMATION MEASURES; GENERALIZED MUTUAL INFORMATION; H-THEOREM; ISOLATED SYSTEMS; JOINT SOURCE-CHANNEL CODING; LOWER BOUNDS; MONOTONICITY; PERSPECTIVE FUNCTION; SECOND LAW OF THERMODYNAMICS; SHANNON ENTROPY; STATE SPACE; STEADY-STATE DISTRIBUTIONS;

MARKOV PROCESSES; THERMODYNAMICS;

DATA HANDLING;

EID: 79960994624     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2159052     Document Type: Article

References (28)

1. M. S. Ali and D. Silvey, "A general class of coefficients of divergence of one distribution from another," J. Roy. Statist. Soc., Ser. B, no. 28, pp. 131-140, 1966.
2. D. Andelman, "Bounds According to a Generalized Data Processing Theorem,"M.Sc., Dept. Elect. Eng., Technion, Israel Institute of Technology, Haifa, Israel, 1974.
3. G. B. Baǧci, The Physical Meaning of Rěnyi Relative Entropies arXiv:cond-mat/0703008v1, 2007.
4. A. H. W. Beck, Statistical Mechanics, Fluctuations and Noise. London, U.K.: Edward Arnold Publishers, 1976.
5. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge University Press, 2004.
6. H. R. Brown, W. Myrvold, and J. Uffink, "Boltzmann's H-theorem, its discontents, and the birth of statistical mechanics," Studies in History and Philos. of Modern Phys., vol. 40, pp. 174-191, 2009.
7. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley, 2006.
8. I. Csiszár, "Eine informationstheoretische ungleichung und ihre anwendung auf den beweis der ergodizität von markoffschen ketten," Publ. Math. Inst. Hungar. Acad., vol. 8, pp. 95-108, 1963.
9. I. Csiszár, "Information-type measures of difference of probability distributions and indirect observation," Studia Scientiarum Mathematicarum Hungarica, vol. 2, pp. 229-318, 1967.
10. I. Csiszár, "A class of measures of informativity of observation channels," Periodica Mathematica Hungarica, vol. 2, no. 1-4, pp. 191-213, 1972.
11. I. Csiszár and P. Shields, "Information theory and statistics: A tutorial," Found. and Trends in Commun. and Inf. Theory, vol. 1, no. 4, pp. 417-528, 2004.
12. B. Dacorogna and P. Marěchal, The Role of Perspective Functions in Convexity, Polyconvexity, Rank-One Convexity and Separate Convexity [Online]. Available: http://caa.epfl.ch/publications/ 2008-The-role-of-perspective- functions-in-convexity.pdf
13. E. T. Jaynes, "Violation of Boltzmann's H-Theorem in real gases," Phys. Rev. A, vol. 4, no. 2, pp. 747-750, Aug. 1971.
14. M. Kardar, Statistical Physics of Particles. Cambridge, U.K.: Cambridge University Press, 2007.
15. F. P. Kelly, Reversibility and Stochastic Networks. New York:Wiley, 1979.
16. C. Kittel, Elementary Statistical Physics. New York: Wiley, 1958.
17. F. Liese and I. Vajda, Convex Statistical Distances. Leipzig, Germany: Teubner, 1987.
18. F. Liese and I. Vajda, "On divergence and informations in statistics and information theory," IEEE Trans. Inf. Theory, vol. 52, no. 10, pp. 4394-4412, Oct. 2006. (Pubitemid 46445281)
19. N. Merhav, "Statistical physics and information theory," Found. and Trends in Commun. and Inf. Theory, vol. 6, no. 1-2, pp. 1-212, 2009.
20. T. Morimoto, "Markov processes and the H-Theorem," J. Phys. Soc. Jap., vol. 12, pp. 328-331, 1963.
21. D. P. Palomar and S. Verdú, "Lautum information," IEEE Trans. Inf. Theory, vol. 54, no. 3, pp. 964-975, Mar. 2008. (Pubitemid 351410913)
22. J. M. R. Parrondo, C. Van den Broeck, and R. Kawai, "Entropy Production and the Arrow of Time," New Journal of Physics, Volume 11, Issue 7, pp. 073008 (2009)., vol. 11, no. 7, p. 073008.
23. A. Pěrez-Madrid, "Molecular theory of irreversibility," J. Chem. Phys., vol. 123, pp. 204148-204148, 2005.
24. F. Reif, Fundamentals of Statistical and Thermal Physics. New York: McGraw-Hill, 1965.
25. L. S. Schulman, Time-Related Issues in Statistical Mechanics [Online]. Available: http://people.clarkson.edu/~lschulma/2006TimeStat- MechLecturesSaclayTechnion.pdf
26. A. J. Viterbi and J. K. Omura, Principles of Digital Communication and Coding. New York: McGraw-Hill, 1979.
27. J. Ziv and M. Zakai, "On functionals satisfying a data-processing theorem," IEEE Trans. Inf. Theory, vol. IT-19, no. 3, pp. 275-283, May 1973.
28. M. Zakai and J. Ziv, "A generalization of the rate-distortion theory and applications," in Information Theory New Trends and Open Problems, G. Longo, Ed. New York: Springer-Verlag, 1975, pp. 87-123.