Uncertainty of discrete stochastic systems: General theory and statistical inference

IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans.

Volumn 26, Issue 6, 1996, Pages 681-697

  Morales, Domingo ^b   Pardo, Leandro ^b   Vajda, Igor ^a,c  

^a IEEE

^b UNIVERSIDAD COMPLUTENSE DE MADRID   (Spain)

^c INSTITUTE OF MATHEMATICS   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC STABILITY; DECISION MAKING; INFORMATION THEORY; PARAMETER ESTIMATION; STATISTICAL METHODS; STATISTICS; SYSTEMS ENGINEERING;

DISCRETE STOCHASTIC SYSTEMS; STATISTICAL INFERENCE; UNCERTAINTY;

PROBABILITY;

EID: 0030286376     PISSN: 10834427     EISSN: None     Source Type: Journal    
DOI: 10.1109/3468.541329     Document Type: Article

References (70)

1. J. Aczel and Z. Daróczy, On Measures of Information and Their Applications. New York: Academic, 1975.
2. S. Arimoto, "Information-theoretical considerations on estimation problems," Inform. Contr., vol. 19, pp. 181-194, 1971.
3. G. P. Basharin, "On the statistical estimate of the entropy of a sequence of independent random variables," Theory of Probability and Its Applications, vol. 4, pp. 361-364, 1959.
4. M. Ben Bassat, "f-entropies, probability of error and feature selection," Inform. Contr., vol. 39, pp. 227-242, 1978.
5. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. Pacific Grove, CA: Brooks/Cole, 1984.
6. J. M. Chambers and T. J. Hastie, Statistical Models. Pacific Grove, CA: Brooks/Cole, 1992.
7. J. E. Cohen, Y. Derriennic, and Gh. Zbaganu, "Majorization, monotonicity of relative entropy and stochastic matrices," Contemp. Math., vol. 149, pp. 251-259, 1993.
8. T. M. Cover and J. Thomas, Elements of Information Theory. New York: Wiley, 1991.
9. T. M. Cover and P. E. Hart, "Nearest neighbor pattern classification," IEEE Trans. Inform. Theory, vol. 13, pp. 21-27, 1967.
10. I. Csiszár, "Information measures: A critical survey," Trans. 7th Prague Conf. on Inform. Theory, Stat. Dec. Functions and Random Processes, Prague, 1978, vol. B, pp. 73-86.
11. I. Csiszár, "Generalized cutoff rates and Renyi's information measures," IEEE Trans. Inform. Theory, vol. 41, pp. 26-34, 1995.
12. H. Dalton, The Inequality of Incomes, 2nd ed. London: Routhledge et Keagan Paul, 1925.
13. M. H. De Groot, "Uncertainty, information and statistical experiments," Ann. Math. Statist., vol. 33, pp. 404-419, 1962.
14. _, Optimal Statistical Decisions. New York: McGraw-Hill, 1970.
15. P. A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach. New York: Academic, 1982.
16. J. J. Dik and M. C. M. Gunst "The distribution of general quadratic forms in normal variables," Statistica Neerlandica, vol. 39, pp. 14-26, 1985.
17. R. L. Dobrushin, "Central Limit Theorem for nonstationary Markov chains I, II," Theory of Probability and Its Applications, vol. 1, pp. 65-80 and 329-383, 1956.
18. J. M. Emlen, Ecology: An Evolutionary Approach. Reading, MA: Addison-Wesley, 1973.
19. M. D. Esteban, "Weighted entropies and divergences: Statistical applications," Ph.D Dissertation, Complutense University of Madrid, 1994, in Spanish.
20. D. K. Fadeev, "Zum Begriff der Entropie einer endlichen Wahrscheinlichkeitschemes," Arbeit zur Informationstheorie. Berlin: Deutscher Verlag der Wissenschaften, vol. 1, 1957.
21. J. Feistauerová and I. Vajda, "Testing system entropy and prediction error probability," IEEE Trans. Syst., Man Cybern., vol. 23, pp. 1352-1358.
22. C. Ferreri, "Hypoentropy and related heterogenity divergence measures," Statistica, vol. 40, pp. 55-118.
23. C. Gini, "Variabilitae mutabilita," Studi Economico - Giuridici della R. Universitá de Cagliari, vol. 3, part 2, p. 80, 1912.
24. P. K. Goel and M. H. De Groot, "Comparison of experiments and information measures," Ann. Statist., vol. 7, pp. 1066-1077, 1979.
25. E. L. Hall, Computer Image Processing and Recognition. New York: Academic, 1979.
26. G. H. Hardy, J. E. Littlewood, and G. Pólya, "Some simple inequalities satisfied by convex functions," Messenger Math., vol. 58. pp. 145-152, 1929.
27. R. V. L. Hartley, "Transmission of information," Bell Syst. Tech. J., vol. 7, pp. 535-545, 1928.
28. J. Havrda and F. Charvát, "The concept of structural a-entropy," Kybernetika, vol. 3, pp. 30-35, 1967.
29. L. K. Jones and C. L. Byrne, "Generalized entropy criteria for inverse problems, with applications to data compression, pattern classification and cluster analysis," IEEE Trans. Inform. Theory, vol. 36, pp. 23-30, 1990.
30. J. N. Kapur, "Some new nonadditive measures of entropy," Bull. U.M.I., pp. 253-266, 1988.
31. J. N. Kapur, P. K. Sahoo, and A. K. C. Wong, "A new method for gray level picture thresholding using the entropy histogram," Comput. Graphic, Vision, Image Processing, vol. 29, pp. 273-285, 1985.
32. G. Klir and T. A. Folger, Fuzzy Sets, Uncertainty and Information. Englewood Cliffs, NJ: Prentice-Hall, 1988.
33. V. A. Kovalevskij, "The problem of character recognition from the point of view of mathematical statistics," Character Readers and Pattern Recognition, V. A. Kovalevskij, Ed. New York: Spartan, 1968, pp. 3-30.
34. E. L. Lehmann, Testing Statistical Hypotheses New York: Wiley, 1959.
35. F. Liese and I. Vajda, Convex Statistical Distances. Leipzig: Teubner, 1987.
36. M. O. Lorenz, "Methods of measuring concentration of wealth," J. Amer. Statist. Assoc., vol. 9, pp. 209-219, 1905.
37. A. W. Marshal and I. Olkin, Inequalities: Theory of Majorization and Its Applications. New York: Academic, 1970.
38. R. P. McIntosh, "An index of diversity and the relation to certain concepts of diversity," Ecology, vol. 48, pp. 392 404, 1967.
39. M. L. Menéndez, I. J. Taneja, and J. A. Pardo, "A fuzzy probabilistic information system comparison criterion: Applications and properties," Fuzzy Sets Syst., vol. 58, pp. 171-183, 1993.
40. D. Morales, L. Pardo, and I. Taneja, "Hypoentropy as an index of diversity," Theory of Probability and Its Applications, vol. 37, pp. 191-194, 1992.
41. F. Österreicher and I. Vajda, "Existence, uniqueness and evaluation of log-optimal investment portfolio," Kybernetika, vol. 29, pp. 105-120, 1993.
42. S. K. Pal and B. Chakraborty, "Fuzzy set theoretic measure for automatic feature evaluation," IEEE Trans. Syst., Man, Cybern., vol. 16, pp. 754-760, 1986.
43. N. R. Pal and S. K. Pal, "Entropy: A new definition and its applications," IEEE Trans. Syst., Man, Cybern., vol. 21, pp. 1260-1270, 1991.
44. L. Pardo, M. L. Menéndez, and J. A. Pardo, "Unified (r, s)-entropy as an index of diversity," J. Franklin Inst., vol. 329, pp. 907-921, 1992.
45. A. C. Pigou, Wealth and Welfare. New York: Macmillan, New York, 1912.
46. C. R. Rao, "Diversity and dissimilarity coefficients: A unified approach," J. Theoret. Popul. Biol., vol. 21, pp. 24-43, 1982.
47. T. R. C. Read and N. A. C. Cressie Goodness-of-Fit Statistics for Discrete Multivariate Data. Berlin: Springer, 1988.
48. A. Rényi, "On the measures of entropy and information," in Proc. 4th Berkeley Symp. Math. Statist. and Prob., 1961, vol. 1, pp. 547-561.
49. _, "On the foundations of information theory," Rev. Int. Statist. Inst., vol. 33, pp. 1-14, 1965.
50. B. D. Ripley, "Statistical aspects of neural networks," Networks and Chaos, S. O. E. Barndorff-Nielsen, J. L. Jensen, and W. S. Kendall, Eds. London: Chapman and Hall, pp. 40-123, 1993.
51. M. Salicrú, M. L. Menéndez, D. Morales, and L. Pardo, "Asymptotic distributions of (h, φ) -entropies," Commun. Statist. (Theory and Methods), vol. 22, pp. 2015-2031, 1993.
52. A. P. Sant'anna and I. J. Taneja, "Trigonometric entropies, Jensen difference divergences and error bounds," Inform. Sci., vol. 35, pp. 145-156, 1985.
53. I. Schur, "Über eine Klasse von Mittelbildungen mit Adwendungen die Determinanten-Theorie," Sitzungsber, Berlin Math. Gesellschaft, vol. 22, pp. 9-20, 1923.
54. R. R. Schutz, "On the measurement of income inequality," Amer. Econom. Rev., vol. 41, pp. 107-122, 1951.
55. R. J. Serfling, Approximation Theorems of Mathematical Statistics. New York: Wiley, New York, 1980.
56. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 379-423 and 623-656, 1948.
57. B. D. Sharma and D. P. Mittal. "New nonadditive measures of relative information," J. Comb. Inform. Syst. Sci., vol. 2, pp. 122-133, 1975.
58. E. H. Simpson, "Measurement of diversity," Nature, vol. 163, p. 68, 1949.
59. I. J. Tancja, "A study of generalized measures in information theory," Ph.D. Dissertation, University of Delhi, Delhi, India, 1975.
60. D. L. Tebbe and S. J. Dwyer III, "Uncertainty and the probability of error," IEEE Trans. Inform. Theory, vol. 14, pp. 516-518, 1968.
61. I. Vajda, "Bounds on the minimal error probability in checking a finite or countable number of hypotheses," Inform. Transmission Problems, vol. 4, pp. 9-17, 1968.
62. _, "Axioms of a-entropy of a generalized probability scheme," Kybernetika, vol. 4, pp. 105-112, 1968.
63. _, "A contribution to informational analysis of patterns," Methodologies of Pattern Recognition, S. Watanabe, Ed. New York: Academic, 1969.
64. I. Vajda and K. Vašek, "Majorization, concave entropies and comparison of experiments," Prob. Contr. Inform. Theory, vol. 14, pp. 105-115, 1985.
65. I. Vajda, Theory of Statistical Inference and Information. Dordrecht: Kluwer, 1989.
66. J. Vejnarová and G. Klir, "Measure of strife in Dempster-Schafer theory," Int. J. General Syst., vol. 22, pp. 25-42, 1993.
67. P. Vosátka, "On some properties of conditional entropies," M.Sc. Thesis, Faculty of Mathematics and Physics, Charles University, Prague, 1969, in Czech.
68. A. K. C. Wong and M. You, "Entropy and distance of random graphs with application to structural pattern recognition," IEEE. Trans. Pattern Anal. Machine Intell., vol. 7, pp. 599-609, 1985.
69. J. Zvárová, "On asymptotic behavior of a sample estimator of Rényi's information of order α," Trans. 6th Prague Conf. Inform. Theory, Statist. Dec. Functions and Random Processes, Prague, pp. 919-924, 1973.
70. J. Zvárová, "On measures of statistical dependence," Čas. Pêst. Matematiky, vol. 99, pp. 15-29, 1974.