Entropy: The Markov ordering approach

Entropy

Volumn 12, Issue 5, 2010, Pages 1145-1193

  Gorban, Alexander N ^a   Gorban, Pavel A ^b   Judge, George ^c  

^a UNIVERSITY OF LEICESTER   (United Kingdom)

^b SIBERIAN FEDERAL UNIVERSITY   (Russian Federation)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Attainable region; Entropy functionals; Inference; Lyapunov function; Markov process; MaxEnt

Indexed keywords

EID: 77953492098     PISSN: None     EISSN: 10994300     Source Type: Journal    
DOI: 10.3390/e12051145     Document Type: Article

References (94)

1. Clausius, R. Über vershiedene für die Anwendungen bequeme Formen der Hauptgleichungen der Wärmetheorie. Poggendorffs Annalen der Physic und Chemie 1865, 125, 353-400.
2. Gibbs, J.W. On the equilibrium of heterogeneous substance. Trans. Connect. Acad. 1875-1876, 108-248; 1877-1878, 343-524.
3. Boltzmann, L. Weitere Studien über das Wärmegleichgewicht unter Gasmolekülen. Sitzungsberichte der keiserlichen Akademie der Wissenschaften 1872, 66, 275-370.
4. Translation: Further studies on the thermal equilibrium of gas molecules, In Kinetic Theory of Gases: An Anthology of Classic Papers With Historical Commentary; Brush, S.G.; Hall, N.S., Eds.; Imperial Colledge Press: London, UK, 2003; pp. 362-368.
5. Gibbs, J.W. Elementary Principles in Statistical Mechanics; Ox Bow Press: New York, NY, USA, 1981.
6. Villani, C. H-theorem and beyond: Boltzmann's entropy in today's mathematics. In Boltzmann's Legacy; Gallavotti, G., Reiter, W.L., Yngvason J., Eds.; EMS Publishing House: Zürich, Switzerland, 2008; pp. 129-145.
7. Jaynes, E.T. Gibbs versus Boltzmann entropy. Am. J. Phys. 1965, 33, 391-398.
8. Goldstein, S.; Lebowitz, J.L. On the (Boltzmann) entropy of non-equilibrium systems. Physica D 2004, 193, 53-66.
9. Grmela, M.; Öttinger, H.C. Dynamics and thermodynamics of complex fluids. I. Development of a general formalism. Phys. Rev. E 1997, 56, 6620-6632.
10. Öttinger, H.C. Beyond Equilibrium Thermodynamics; Wiley-Interscience: Hoboken, NJ, USA, 2005.
11. Hartley, R.V.L. Transmission of information. Bell Syst. Tech. J. 1928, July, 535-563.
12. Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27, 379-423, 623-656.
13. Kullback, S.; Leibler, R.A. On information and sufficiency. Ann. Math. Statist. 1951, 22, 79-86.
14. Zeldovich, Y.B. Proof of the uniqueness of the solution of the equations of the law of mass action. In Selected Works of Yakov Borisovich Zeldovich; Ostriker, J.P., Ed.; Princeton University Press: Princeton, NJ, USA, 1996; Volume 1, pp. 144-148.
15. Yablonskii, G.S.; Bykov, V.I.; Gorban, A.N.; Elokhin, V.I. Kinetic Models of Catalytic Reactions, Series: Comprehensive Chemical Kinetics; Elsevier: Amsterdam, The Netherlands, 1991; Volume 32.
16. Hangos, K.M. Engineering model reduction and entropy-based lyapunov functions in chemical reaction kinetics. Entropy 2010, 12, 772-797.
17. Burg, J.P. Maximum entropy spectral analysis. In Proceedings of the 37th meeting of the Society of Exploration Geophysicists, Oklahoma City, OK, USA, 1967; Reprinted in Modern Spectrum Analysis; Childers, D.G., Ed.; IEEE Press: New York, NY, USA,1978; pp. 34-39.
18. Burg, J.P. The relationship between maximum entropy spectra and maximum likelihood spectra. Geophysics 1972, 37, 375-376.
19. Jaynes, E.T. Information theory and statistical mechanics. Phys. Rev. 1957, 106, 620-630.
20. Jaynes, E.T. Information theory and statistical mechanics. II. Phys. Rev. 1957, 108, 171-190.
21. Harremoës, P.; Topsøe, F. Maximum entropy fundamentals. Entropy 2001, 3, 191-226.
22. Beck, C. Generalized information and entropy measures in physics. Contemp. Phys. 2009, 50, 495-510.
23. Mittelhammer, R.; Judge, G.; Miller, D. Econometric Foundations; Cambridge University Press: New York, NY, USA, 2000.
24. Van Akkeren, M.; Judge, G.; Mittelhammer, R. Generalized moment based estimation and inference. J. Econom. 2002, 107, 127-148.
25. Myers, T.S.; Daniel, N.; Osherson, D.N. On the psychology of ampliative inference. Psychol. Sci. 1992, 3, 131-135.
26. Esteban, M.D.; Morales, D. A summary of entropy statistics. Kybernetica 1995, 31, 337-346.
27. Rényi, A. On measures of entropy and information. In Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability 1960, Berkeley, CA, USA, 20 June-30 July 1960; University of California Press: Berkeley, CA, USA, 1961; Volume 1, pp. 547-561.
28. Tsallis, C. Possible generalization of Boltzmann-Gibbs statistics. J. Stat. Phys. 1988, 52, 479-487.
29. Abe, S., Okamoto, Y. Eds. Nonextensive Statistical Mechanics and its Applications; Springer: Heidelberg, Germany, 2001.
30. Cressie, N.; Read, T. Multinomial goodness of fit tests. J. R. Stat. Soc. Ser. B 1984, 46, 440-464.
31. Read, T.R., Cressie, N.A. Goodness of Fit Statistics for Discrete Multivariate Data; Springer: New York, NY, USA, 1988.
32. Bagci, G.B.; Tirnakli, U. On the way towards a generalized entropy maximization procedure. Phys. Lett. A 2009, 373, 3230-3234.
33. Cho, A. A fresh take on disorder, or disorderly science? Science 2002, 297, 1268-1269.
34. Lin, S-K. Diversity and entropy. Entropy 1999, 1, 1-3.
35. Gorban, A.N. Equilibrium Encircling. Equations of Chemical kinetics and their Thermodynamic Analysis; Nauka: Novosibirsk, Russia, 1984.
36. Gorban, A.N.; Karlin, I.V. Family of additive entropy functions out of thermodynamic limit. Phys. Rev. E 2003, 67, 016104; Available online: http://arxiv.org/abs/cond-mat/0205511 (accessed on 4 May 2010)
37. Gorban, A.N.; Karlin, I.V.; Öttinger, H.C. The additive generalization of the Boltzmann entropy. Phys. Rev. E 2003, 67, 067104; Available online: http://arxiv.org/abs/cond-mat/0209319 (accessed on 4 May 2010).
38. Nonextensive statistical mechanics and thermodynamics: bibliography. Available online: http://tsallis.cat.cbpf.br/TEMUCO.pdf (accessed on 4 May 2010).
39. Petz, D. From f-divergence to quantum quasi-entropies and their use. Entropy 2010, 12, 304-325.
40. Cachin, C. Smooth entropy and Rényi entropy. In Proceedings of EUROSCRIPT'97; Fumy, W. et al., Eds., LNCS; Springer: New York, NY, USA, 1997; Volume 1233, pp. 193-208.
41. Davis, J.V.; Kulis, B.; Jain, P.; Sra, S.; Dhillon, I.S. Information-theoretic metric learning. ICML 2007, 227, 209-216.
42. Rényi, A. Probability Theory; North-Holland: Amsterdam, The Netherlands, 1970.
43. Abe, S. Axioms and uniqness theorem for Tsallis entropy. Phys. Lett. A 2000, 271, 74-79.
44. Aczel, J. Measuring information beyond communication theory. Inf. Process. Manage. 1984, 20, 383-395.
45. Aczel, J.; Daroczy, Z. On Measures of Information and Their Characterizations; Academic Press: New York, NY, USA, 1975.
46. Csiszár, I. Information measures: A critical survey. In Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions and the Eighth European Meeting of Statisticians, Prague, Czech Republic, 18 August-23 August 1974; Academia: Prague, Czech Republic, 1978; Volume B, pp. 73-86.
47. Gorban, P. Monotonically equivalent entropies and solution of additivity equation. Physica A 2003, 328, 380-390; Available online: http://arxiv.org/abs/cond-mat/0304131 (accessed on 4 May 2010).
48. Wehrl, A. General properties of entropy. Rev. Mod. Phys. 1978, 50, 221-260.
49. Morimoto, T. Markov processes and the H-theorem. J. Phys. Soc. Jap. 1963, 12, 328-331.
50. Csiszár, I. Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizitat von Markoffschen Ketten. Magyar. Tud. Akad. Mat. Kutato Int. Kozl. 1963, 8, 85-108.
51. Rockafellar, R.T. Convex Analysis; Princeton University Press: Princeton, NJ, USA, 1970, reprint 1997.
52. Liese, F.; Vajda, I. Convex Statistical Distances; Teubner: Leipzig, Germany, 1987.
53. Dobrushin, R.L. Central limit theorem for non-stationary Markov chains I, II. Theory Probab. Appl. 1956, 1, 163-80, 329-383.
54. Seneta, E. Nonnegative Matrices and Markov Chains; Springer: New York, USA, 1981.
55. Cohen, J.E.; Derriennic, Y.; Zbaganu, G.H. Majorization, Monotonicity of relative entropy and stochastic matrices. Contemp. Math. 1993, 149, 251-259.
56. Cohen, J.E.; Iwasa, Y.; Rautu, G.; Ruskai, M.B.; Seneta, E.; Zbaganu, G. Relative entropy under mappings by stochastic matrices. Linear. Alg. Appl. 1993, 179, 211-235.
57. Del Moral, P.; Ledoux, M.; Miclo, L. On contraction properties of Markov kernels. Probab. Theory Relat. Field 2003, 126, 395-420.
58. Amari, S. Divergence, Optimization, Geometry; In Proceedings of the 16th International Conference on Neural Information Processing, Bangkok, Thailand, 1 December-5 December 2009; Leung, C.S., Lee, M., Chan, J.H., Eds.; Springer: Berlin, Germany, 2009; LNCS 5863, pp. 185-193.
59. Bregman, L.M. The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 1967, 7, 200-217.
60. Itakura, F.; Saito, S. Analysis synthesis telephony based on the maximum likelihood method. In Proceedings of the 6th International Congress on Acoustics, Tokyo, Japan, 1968; pp. C17-C20
61. Reprinted in Speech Synthesis; Flanagan, J.L., Rabiner, L.R., Eds.; Dowden, Hutchinson & Ross: Stroudsburg, PA, USA, 1973; pp. 289-292.
62. Gorban, A.N. Invariant sets for kinetic equations. React. Kinet. Catal. Lett. 1979, 10, 187-190.
63. Gorban, A.N.; Kaganovich, B.M.; Filippov, S.P.; Keiko, A.V.; Shamansky, V.A.; Shirkalin, I.A. Thermodynamic Equilibria and Extrema: Analysis of Attainability Regions and Partial Equilibria; Springer: New York, NY, USA, 2006.
64. Corless, R.M.; Gonnet, G.H.; Hare, D.E.G.; Jeffrey, D.J.; Knuth, D.E. On the Lambert W function. Adv. Comput. Math. 1996, 5, 329-359.
65. Edwards, R.E. Functional Analysis: Theory and Applications; Dover Publications: New York, NY, USA, 1995.
66. Kolmogorov, A.N. Sulla teoria di Volterra della lotta per l'esistenza. Giornale Instituto Ital. Attuari 1936, 7, 74-80.
67. May, R.M.; Leonard, W.J. Nonlinear aspects of competition between three species. SIAM J. Appli. Math. 1975, 29, 243-253.
68. Bazykin, A.D. Nonlinear Dynamics of Interacting Populations; World Scientific Publishing: Singapore, 1998.
69. Sigmung, K. Kolmogorov and population dynamics. In Kolmogorovs Heritage in Mathematics; Charpentier, É., Lesne, A., Nikolski, N.K., Eds.; Springer: Berlin, Germany, 2007; pp. 177-186.
70. Horn, F. Attainable regions in chemical reaction technique. In The Third European Symposium on Chemical Reaction Engineering; Pergamon Press: London, UK, 1964; pp. 1-10.
71. Glasser, D.; Hildebrandt, D.; Crowe, C. A geometric approach to steady flow reactors: the attainable region and optimisation in concentration space. Am. Chem. Soc. 1987, 26, 1803-1810.
72. Filippi-Bossy, C.; Bordet, J.; Villermaux, J.; Marchal-Brassely, S.; Georgakis C. Batch reactor optimization by use of tendency models. Comput. Chem. Eng. 1989, 13, 35-47
73. Hildebrandt, D.; Glasser, D. The attainable region and optimal reactor structures. Chem. Eng. Sci. 1990, 45, 2161-2168.
74. Feinberg, M.; Hildebrandt, D. Optimal reactor design from a geometric viewpoint-I. Universal properties of the attainable region. Chem. Eng. Sci. 1997, 52, 1637-1665.
75. Hill, M. Chemical product engineering-the third paradigm. Comput. Chem. Eng. 2009, 33, 947-953.
76. Smith, R.L.; Malone, M.F. Attainable regions for polymerization reaction systems. Ind. Eng. Chem. Res. 1997, 36, 1076-1084.
77. Metzger, M.J.; Glasser, D., Hausberger, B.; Hildebrandt, D.; Glasser, B.J. Use of the attainable region analysis to optimize particle breakage in a ball mill. Chem. Eng. Sci. 2009 64, 3766-3777.
78. McGregor, C; Glasser, D; Hildebrandt, D. The attainable region and Pontryagin's maximum principle. Ind. Eng. Chem. Res. 1999, 38, 652-659.
79. Kauchali, S.; Rooney, W.C.; Biegler, L.T.; Glasser, D.; Hildebrandt, D. Linear programming formulations for attainable region analysis. Chem. Eng. Sci. 2002, 57, 2015-2028.
80. Manousiouthakis, V.I.; Justanieah, A.M.; Taylor, L.A. The Shrink-Wrap algorithm for the construction of the attainable region: an application of the IDEAS framework. Comput. Chem. Eng. 2004, 28, 1563-1575.
81. Gorban, A.N. Methods for qualitative analysis of chemical kinetics equations. In Numerical Methods of Continuum Mechanics; Institute of Theoretical and Applied Mechanics: Novosibirsk, USSR, 1979; Volume 10, pp. 42-59.
82. Gorban, A.N.; Yablonskii, G.S.; Bykov, V.I. Path to equilibrium. In Mathematical Problems of Chemical Thermodynamics; Nauka: Novosibirsk, USSR, 1980; pp. 37-47 (in Russian)
83. English translation: Int. Chem. Eng. 1982, 22, 368-375.
84. Gorban, A.N.; Yablonskii, G.S. On one unused possibility in planning of kinetic experiment. Dokl. Akad. Nauk SSSR 1980, 250, 1171-1174.
85. Zylka, Ch. A note on the attainability of states by equalizing processes. Theor. Chim. Acta 1985, 68, 363-377.
86. Krambeck, F.J. Accessible composition domains for monomolecular systems. Chem. Eng. Sci. 1984, 39, 1181-1184.
87. Shinnar, R.; Feng, C.A. Structure of complex catalytic reactions: thermodynamic constraints in kinetic modeling and catalyst evaluation. Ind. Eng. Chem. Fundam. 1985, 24, 153-170.
88. Shinnar, R. Thermodynamic analysis in chemical process and reactor design. Chem. Eng. Sci. 1988, 43, 2303-2318.
89. Bykov, V.I. Comments on "Structure of complex catalytic reactions: thermodynamic constraints in kinetic modeling and catalyst evaluation". Ind. Eng. Chem. Res. 1987, 26, 1943-1944.
90. Alberti, P.M.; Uhlmann, A. Stochasticity and Partial Order-Doubly Stochastic Maps and Unitary Mixing (Mathematics and its Applications 9); D. Reidel: London, UK, 1982.
91. Alberti, P.M.; Crell, B.; Uhlmann, A.; Zylka, C. Order structure (majorization) and irreversible processes. In Vernetzte Wissenschaften-Crosslinks in Natural and Social Sciences; Plath, P.J., Hass, E.-Chr., Eds.; Logos Verlag: Berlin, Germany, 2008; pp. 281-290.
92. Harremoës, P. A new look on majorization. In Proceedings ISITA 2004, Parma, Italy, 2004; IEEE/SITA: Washington, DC, USA, 2004; pp. 1422-1425.
93. Harremoës P.; Tishby, N. The information bottleneck revisited or how to choose a good distortion measure. In Proceedings ISIT 2007, Nice, France, 2007; IEEE Information Theory Society: Washington, DC, USA, 2007; pp. 566-571.
94. Aczél, J. Lectures on Functional Equations and Their Applications; Academic Press: New York, NY, USA, 1966.