Large-sample asymptotic approximations for the sampling and posterior distributions of differential entropy for multivariate normal distributions

Entropy

Volumn 13, Issue 4, 2011, Pages 805-819

  Marrelec, Guillaume ^a,b,c   Benali, Habib ^a,b,c  

^a INSERM   (France)

^b SORBONNE UNIVERSITÉ   (France)

^c UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

Asymptotic approximation; Differential entropy; Large sample; Multiinformation; Multivariate constraint; Multivariate normal distribution; Mutual information; Posterior distribution; Sampling distribution; Total correlation

Indexed keywords

EID: 79955949561     PISSN: None     EISSN: 10994300     Source Type: Journal    
DOI: 10.3390/e13040805     Document Type: Article

References (20)

1. Ahmed, N.A.; Gokhale, D.V. Entropy expressions and their estimators for multivariate distributions. IEEE Trans. Inform. Theory 1989, 35, 688-692.
2. Misra, N.; Singh, H.; Demchuk, E. Estimation of the entropy of a multivariate normal distribution. J. Multivariate Anal. 2005, 92, 324-342.
3. Gupta, M.; Srivastava, S. Parametric Bayesian estimation od differential entropy and relative entropy. Entropy 2010, 12, 818-843.
4. Beirlant, J.; Dudewicz, E.J.; Gÿorfi, L.; van der Meulen, E.C. Nonparametric entropy estimation: An overview. Int. J. Math. Stastist. Sci. 1997, 6, 17-39.
5. Strong, S.P.; Koberle, R.; de Ruyter van Steveninck, R.R.; Bialek, W. Entropy and information in neural spike trains. Phys. Rev. Lett. 1998, 80, 197-200.
6. Antos, A.; Kontoyiannis, I. Convergence properties of functional estimates for discrete distributions. Random Struct. Algor. 2001, 19, 163-193.
7. Paninski, L. Estimation of entropy and mutual information. Neural Comput. 2003, 15, 1191-1253.
8. Wolpert, D.H.; Wolf, D.R. Estimating functions of probability distributions from a finite set of samples. Phys. Rev. E 1995, 52, 6841-6854.
9. Wolpert, D.H.; Wolf, D.R. Erratum: Estimating functions of probability distributions from a finite set of samples. Phys. Rev. E 1996, 54, 6973.
10. Anderson, T.W. An Introduction to Multivariate Statistical Analysis; John Wiley and Sons: New York, NY, USA, 1958.
11. Abramowitz, M.; Stegun, I.A. Handbook of Mathematical Functions; Applied Mathematics Series 55; National Bureau of Standards: Washington, DC, USA, 1972.
12. Anderson, T.W. An Introduction to Multivariate Statistical Analysis, 3rd ed.; Series in Probability and Mathematical Statistics; John Wiley and Sons: New York, NY, USA, 2003.
13. Gelman, A.; Carlin, J.B.; Stern, H.S.; Rubin, D.B. Bayesian Data Analysis; Texts in Statistical Science; Chapman & Hall: London, UK, 1998.
14. Watanabe, S. Information theoretical analysis of multivariate correlation. IBM J. Res. Dev. 1960, 4, 66-82.
15. Garner, W.R. Uncertainty and Structure as Psychological Concepts; John Wiley & Sons: New York, NY, USA, 1962.
16. Joe, H. Relative entropy measures of multivariate dependence. J. Am. Statist. Assoc. 1989, 84, 157-164.
17. Studeńy, M.; Vejnarova, J. The multiinformation function as a tool for measuring stochastic dependence. In Proceedings of the NATO Advanced Study Institute on Learning in Graphical Models; Jordan, M.I., Ed.; MIT Press: Cambridge, MA, USA, 1998; pp. 261-298.
18. Press, S.J. Applied Multivariate Analysis. Using Bayesian and Frequentist Methods of Inference, 2nd ed.; Dover: Mineola, NY, USA, 2005.
19. Kullback, S. Information Theory and Statistics; Dover: Mineola, NY, USA, 1968.
20. Chen, C.P. Inequalities for the polygamma functions with application. Gener. Math. 2005, 13, 65-72.