Properties of Cross-Entropy Minimization

IEEE Transactions on Information Theory

Volumn 27, Issue 4, 1981, Pages 472-482

  Shore, John E ^a   Johnson, Rodney W ^a  

^a NAVAL RESEARCH LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION THEORY;

EID: 0019586465     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/TIT.1981.1056373     Document Type: Article

References (21)

1. S. Kullback, Information Theory and Statistics. New York; Wiley, 1959.
2. E. T. Jaynes, “Information theory and statistical mechanics I,” Phys. Rev., vol. 106, pp. 620–630, 1957.
3. W. M. Elsasser, “On quantum measurements and the role of the uncertainty relations in statistical mechanics,” Phys. Rev., vol. 52, pp. 987–999, Nov. 1937.
4. J. E. Shore and R. W. Johnson, “Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy,” IEEE Trans. Inform. Theory, vol. IT-26, pp. 26–37, Jan. 1980.
5. J. E. Shore, “Minimum cross-entropy spectral analysis,” IEEE Trans. Acous. Speech Signal Processing, vol. ASSP-29, pp. 230–237, Apr. 1981.
6. R. M. Gray, A. H. Gray, Jr., G. Rebolledo, and J. E. Shore, “Rate-distortion speech coding with a minimum discrimination information distortion measure,” IEEE Trans. Inform. Theory, to be published.
7. J. E. Shore and R. M. Gray, “Minimum cross-entropy pattern classification and cluster analysis,” IEEE Trans. Pattern Anal. Mach. Intell. to be published.
8. R. W. Johnson, “Axiomatic characterization of the directed divergences and their linear combinations,” IEEE Trans. Inform. Theory, vol. IT-25, no. 6, pp. 709–716, Nov. 1979.
9. F. Itakura and S. Saito, “Analysis synthesis telephone based upon maximum likelihood method,” Reports of the 6th Int. Cong. Acoustics, Y. Yonasi, ed. Tokyo, 1968.
10. R. M. Gray, A. Buzo, A. H. Gray, Jr., and Y. Matsuyama, “Distortion measures for speech processing,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. ASSP-28, pp. 367–376, 1980.
11. A. Buzo, A. H. Gray, Jr., R. M. Gray, and J. D. Markel, “Speech coding based upon vector quantization,” IEEE Trans. Acoustics, Speech and Signal Processing, vol. ASSP-28, pp. 562–574, Oct. 1980.
12. I. Csiszar, “I-Divergence geometry of probability distributions and minimization problems,” Ann. Prob., vol. 3, pp. 146–158, 1975.
13. M. Tribus, Rational Descriptions, Decisions, and Designs. New York: Pergamon, 1969.
14. L. Geronimus, Orthogonal Polynomials. New York: Consultants Bureau, 1961.
15. J. D. Markel and A. H. Gray, Jr., Linear Prediction of Speech. New York: Springer—Verlag, 1976.
16. A. E. Albert, Regression and the Moore—PenrosePseudoinverse. New York: Academic, 1972.
17. R. W. Johnson, “Determining probability distributions by maximum entropy and minimum cross-entropy,” Proc. APL79, 24–29.
18. M. A. Jenkins, “The solution of linear systems of equations and linear least squares problems in APL,” Scientific Center Tech. Rep. No. 320–2989, IBM, NY, June 1970.
19. D. V. Gokhale and S. Kullback, The Information in Contingency Tables. New York: Marcel Dekher, 1978.
20. Y. Alhassid, N. Agmon, and R. D. Levine, “An upper bound for the entropy and its applications to the maximal entropy problem,” Chem. Phys. Lett., vol. 53, no. 1, pp. 22–26, 1978.
21. N. Agmon, Y. Alhassid, and R. D. Levine, “An algorithm for finding the distribution of maximal entropy,” J. Comput. Phys., vol. 30, pp. 250–258, 1979.