Some inequalities relating different measures of divergence between two probability distributions

IEEE Transactions on Information Theory

Volumn 45, Issue 5, 1999, Pages 1728-1735

  Withers Jr , Lang ^a,b  

^a IEEE

^b NAVAL RESEARCH LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; BOUNDARY CONDITIONS; FUNCTIONS; INTEGRAL EQUATIONS; INVERSE PROBLEMS; PROBABILITY DISTRIBUTIONS; SENSOR DATA FUSION; STATISTICAL METHODS;

BHATTACHARYYA ANGLE OF DIVERGENCE; CHI-SQUARED STATISTICS; CONVEX LIKELIHOOD RATION EXPECTATION; CRESSIE-READ-ANSCOMBE STATISTIC; CROSS ENTROPY; DIRECTED DIVERGENCE; DIVERGENCE MEASURES; INEQUALITY; RELATIVE INFORMATION;

INFORMATION THEORY;

EID: 0032679969     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.771256     Document Type: Article

References (30)

1. T. Kailath, "The divergence and Bhattacharyya distance measures in signal selection," IEEE Trans. Commun. Technol., vol. COM-15, pp. 52-60, Feb. 1967.
2. J. Richards, Remote Sensing Digital Image Analysis. New York: Springer-Verlag, 1993.
3. R. Duda and P. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
4. L. Jones and C. Byrne, "General entropy criteria for inverse problems, with applications to data compression, pattern classification, and cluster analysis," IEEE Trans. Inform. Theory, vol. 36, pp. 23-30, Jan. 1990.
5. W. Press, S. Teulkovsky, W. Vetterling, and B. Flannery, Numerical Recipes in C, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 1992.
6. D. Hall, Mathematical Methods in Multisensor Data Fusion. Norwood, MA: Artech House, 1992.
7. I. Csiszár, "Information-type measures of difference of probability distributions and indirect observations," Studio Scient. Math. Hung., vol. 2, pp. 299-318, 1967.
8. S. Ali and S. Silvey, "A general class of coefficients of divergence of one distribution from another," J. Roy. Stat. Soc., vol. 28, pp. 131-142, 1966.
9. F. Liese and I. Vajda, Convex Statistical Distances (Teubner-Texte zur Mathematik, vol. 95). Leipzig, Germany: Teubner, 1987.
10. S. Kullback, Information Theory and Statistics. New York: Wiley, 1959.
11. G. Hardy, J. Littlewood, and G. Pólya, Inequalities, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 1952.
12. T. Read and N. Cressie, Goodness of Fit Statistics for Discrete Multivariable Data. New York: Springer-Verlag, 1988.
13. E. Parzen, "Time series, statistics, and information," in New Directions in Times Series Analysis I, IMA, vol. 45. New York: Springer-Verlag, 1990, pp. 265-286.
14. L. Withers, Jr., "Computing measures of divergence between exact or approximated distributions: A case for avoiding infinities," in Proc. 1998 IRIS Nat. Symp. Sensor and Data Fusion (Marietta, GA).
15. A. Bhattacharyya, "On a measure of divergence between two statistical populations defined by their probability distributions," Bull. Calcutta Math. Soc., vol. 35, pp. 99-109, 1943.
16. H. Jeffreys, "An invariant form for the prior probability in estimation problems," Proc. Roy. Soc. A, vol. 186, pp. 453-461, 1946.
17. S. Kakutani, "On the equivalence of infinite product measures," Ann. Math., vol. 49, pp. 214-224, 1948.
18. K. Matusita, "Decision rules, based on the distance for problems of fit, two samples and estimation," Ann. Math. Statist., vol. 26, pp. 631-640, 1955.
19. E. Hellinger, "Die Orthogonalinvarianten quadratisher Formen von unendlichvielen Variablen," Inaugural-Dissertation, Georg-August-Universität, Göttingen, Germany, p. 86, 1907.
20. _, "Neue Begründung der Theorie quadratisher Formen von unendlichvielen Veränderlichen," J. für Reine und Angew. Math., vol. 36, pp. 210-271, 1909.
21. _, "Spectra of quadratic forms in infinitely many variables," in Northwestern Univ. Studies in Math, and the Physical Sciences, No. 1: Math. Monographs, vol. 1, Northwestern Univ., Evanston, IL, 1941, pp. 133-172.
22. L. de Branges, I. Gohberg, and J. Rovnyak, Eds., Topics in Operator Theory: Ernst D. Hellinger Memorial Volume, OT48. Basel, Switzerland: Birkhäuser, 1990.
23. S. Kullback and R. Leibler, "On information and sufficiency," Ann. Math. Statist., vol. 22, pp. 79-86, 1951.
24. L. Boltzmann, "Weitere Bemerkungen über einige Probleme der mechanishen Wärmetheorie," Weiner Berichte, vol. 76, pp. 7-46. 1878.
25. H. Akaike, "Prediction and entropy," Univ. of Wisconsin Math. Research Center, Tech. Rep. Summary 2397, June 1982.
26. R. Hogg and A. Craig, Intro. to Mathematical Statistics, 4th ed. New York: Macmillan, 1991.
27. D. McQuarrie and J. Simon, Physical Chemistry: A Molecular Approach. Univ. Sci. Books, 1997.
28. P. Billingsley, Probability and Measure, 2nd ed. New York: Wiley, 1986.
29. J. Conway, Functions of One Complex Variable. New York: Springer Verlag, 1973.
30. G. Toussaint, "Sharper lower bounds for discrimination information in terms of variation," IEEE Trans. Inform. Theory, vol. IT-21, pp. 99-100, 1975.