Fast multidimensional entropy estimation by κ-d partitioning

IEEE Signal Processing Letters

Volumn 16, Issue 6, 2009, Pages 537-540

  Stowell, Dan ^a   Plumbley, Mark D ^a  

^a QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Entropy; Estimation; Multidimensional signal processing; Multidimensional systems

Indexed keywords

ADAPTIVE PARTITIONING; BIAS AND VARIANCE; DATA POINTS; DIFFERENTIAL ENTROPY; ENTROPY ESTIMATION; LOW MEMORY; MULTIDIMENSIONAL DISTRIBUTIONS; MULTIDIMENSIONAL SIGNAL PROCESSING; MULTIDIMENSIONAL SYSTEMS; NONPARAMETRIC ESTIMATORS; ORDERS OF MAGNITUDE;

ENTROPY; SIGNAL PROCESSING; SYSTEM THEORY;

ESTIMATION;

EID: 77955978925     PISSN: 10709908     EISSN: None     Source Type: Journal    
DOI: 10.1109/LSP.2009.2017346     Document Type: Article

References (22)

1. C. Arndt, Information Measures. Hong Kong: Springer, 2001.
2. C.-I. Chang, Y. Du, J. Wang, S.-M. Guo, and P. D. Thouin, "Survey and comparative analysis of entropy and relative entropy thresholding techniques," Proc. bat. Elect. Eng., Vis., Image Signal Process., vol. 153, no. 6, pp. 837-850, Dec. 2006.
3. D. C. Martins, U. M. Braga-Neto, R. F. Hashimoto, M. L. Bittner, and E. R. Dougherty, "Intrinsically multivariate predictive genes," J. Sel. Topics Signal Process., vol. 2, no. 3, pp. 424-439, Jun. 2008.
4. J. Beirlant, E. J. Dudewicz, L. Gyorfi, and E. C. van der Meulen, "Nonparametric entropy estimation: An overview," Int. J. Math. Statist. Sci, vol. 6, pp. 17-39, 1997.
5. J. Victor, "Binless strategies for estimation of information from neural data," Phys. Rev. E, vol. 66, no. 5, p. 51903, 2002.
6. J. L. Bentley, "Multidimensional binary search trees used for associative searching," Commun. ACM, vol. 18, no. 9, pp. 509-517, 1975.
7. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. London, U.K.: Chapman & Hall/CRC, 1984.
8. r-norm convergence for data-based histogram density estimates," Theory Probabil Applicat., vol. 35, no. 2, pp. 396-403, 1990.
9. G. Lugosi and A. Nobel, "Consistency of data-driven histogram methods for density estimation and classification," Ann. Statist., vol. 24, no. 2, pp. 687-706, 1996.
10. O. Vasicek, "A test for normality based on sample entropy," J. Royal Statist. Soc. B (Methodological), vol. 38, no. 1, pp. 54-59, 1976.
11. E. G. Learned-Miller and J. W. Fisher, III, "ICA using spacings estimates of entropy," J. Mach. Learn. Res., vol. 4, pp. 1271-1295, 2003.
12. G. A. Darbellay and I. Vajda, "Estimation of the information by an adaptive partitioning of the observation space," IEEE Trans. Inf. Theory, vol. 45, no. 4, pp. 1315-1321, 1999.
13. H. Edelsbmnner, Algorithms in Computational Geometry, ser. EATCS Monographs on Theoretical Computer Science. Berlin, Germany: Springer, 1987, vol. 10.
14. E. G. Learned-Miller, "A new class of entropy estimators for multi-dimensional densities," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP"03), Apr. 2003, vol. 3, no. III, pp. 297-300.
15. C P. Quesenberry and F. L. Miller, "Power studies of some tests for uniformity,"J. Statist. Comput. Simul., vol. 5, no. 3, pp. 169-191, 1977.
16. J. T. Chu, "On the distribution of the sample median," Ann. Math. Statist., vol. 26, pp. 112-116, 1955.
17. J. Kybic, "Incremental updating of nearest neighbor-based high-dimensional entropy estimation," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP'06), 2006, vol. 3, pp. 804-807.
18. G. Navarro and R. Baeza-Yates, "Searching in metric spaces," ACM Comput. Surv., vol. 33, pp. 273-321, 2001.
19. A. Beygelzimer, S. Kakadě, and J. Langford, "Cover trees for nearest neighbor," in Proc. 23rd Int. Conf. Machine Learning, New York, 2006, pp. 97-104, ACM Press.
20. P. B. Callahan, "Optimal parallel all-nearest-neighbors using the wellseparated pair decomposition," in Proc. 34th IEEE Symp. Foundations of Computer Science, 1993, pp. 332-340.
21. D. Lang, M. Klaas, and N. de Freitas, "Insights on Fast Kernel Density Estimation Algorithms," Dept. Comput. Sci., Univ. Toronto, Toronto, ON, Canada, 2004, Tech. Rep.
22. C. A. R. Hoare, "Algorithm. 63 (partition) and algorithm 65 (find)," Commun. ACM, vol. 4, pp. 321-322, 1961.