Sample-spacings-based density and entropy estimators for spherically invariant multidimensional data

Neural Computation

Volumn 22, Issue 8, 2010, Pages 2208-2227

  Lee, Intae ^a  

^a UNIVERSITY OF CALIFORNIA SAN DIEGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77956003201     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2010.02-09-972     Document Type: Letter

References (29)

1. Ahmad, I. A., & Lin, P. E. (1976). A nonparametric estimation of the entropy for absolutely continuous distributions. IEEE Trans. on Information Theory, 22, 372-375.
2. Amari, S.-I., Cichocki, A., & Yang, H. H. (1996). A new learning algorithm for blind signal separation. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 757-763). Cambridge, MA: MIT Press.
3. Bach, F. R., & Jordan, M. I. (2002). Kernel independent component analysis. J. Machine Learning Research, 3, 1-48.
4. Beirlant, J., Dudewicz, E.J., Györfi, L., & van der Meulen, E.C.(1997). Nonparametric entropy estimation: An overview. Int. J. Mathematical Statistical Sciences, 6, 17-39.
5. Bell, A. J., & Sejnowski, T. J. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6), 1129-1159.
6. Bingham, E., & Hyvärinen, A. (2000). A fast fixed-point algorithm for independent component analysis of complex-valued signals. Int. J. Neural Systems, 10(1), 1-8.
7. Boscolo, R., Pan, H., & Roychowdhury, V. P. (2004). Independent component analysis based on nonparametric density estimation. IEEE Trans. on Neural Networks, 15(1), 55-65.
8. Cardoso, J.-F. (1999). High-order contrasts for independent component analysis. Neural Computation, 11(1), 157-192.
9. Chang, C.-I., Du, Y., Wang, J., Guo, S.-M., & Thouin, P. D. (2006). Survey and comparative analysis of entropy and relative entropy thresholding techniques. Vision, Image and Signal Processing, 153(6), 837-850.
10. Comon, P. (1994). Independent component analysis-a new concept? Signal Processing, 36, 287-314.
11. Costa, J. A., & Hero, A. O. (2004). Geodesic entropic graphs for dimension and entropy estimation in manifold learning. Signal Processing, 52(8), 2210-2221.
12. Hérault, J., & Jutten, C. (1986). Space or time processing by neural network models. In Proc. AIP Conf.: Neural Networks for Computing (Vol. 151, pp. 206-211). Melville, NY: American Institute of Physics.
13. Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent componenet analysis. IEEE Trans. Neural Networks, 10(3), 626-634.
14. Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley.
15. Hyvärinen, A., & Oja, E. (1997). A fast fixed-point algorithm for independent component analysis. Neural Computation, 9(7), 1483-1492.
16. Kozachenko, L. F., & Leonenko, N. N. (1987). Sample estimates of entropy of a random vector. Probl. Inform. Transmission, 23, 95-101.
17. Learned-Miller, E. G. (2003). A new class of entropy estimators for multi-dimensional densities. In Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (pp. 297-300). Piscataway, NJ: IEEE.
18. Learned-Miller, E. G., & Fisher III, J. W. (2003). ICA using spacings estimates of entropy. J. Machine Learning Research, 4, 1271-1295.
19. Lee, I., & Lee, T.-W. (2007). Nonparametric independent component analysis for circular complex variables. In Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (pp. 665-668). Piscataway, NJ: IEEE.
20. Lee, T.-W., Girolami, M., & Sejnowski, T. J. (1999). Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and super-gaussian sources. Neural Computation, 11(2), 417-441.
21. Lugosi, G., & Nobel, A. (1996). Consistency of data-driven histogram methods for density estimation and classification. Annals of Statistics, 24(2), 687-706.
22. Martins, D. C., Braga-Neto, U. M., Hashimoto, R. F., Bittner, M. L., & Dougherty, E. R. (2008). Intrinsically multivariate redictive genes. Selected Topics in Signal Processing, 2(3), 424-439.
23. Moddemeijer, R. (1989). On estimation of entropy and mutual information of continuous distributions. Signal Processing, 16(3), 233-246.
24. Pham, D.-T. (2000). Blind separation of instantaneous mixture of sources based on order statistics. IEEE Trans. on Signal Processing, 48(2), 363-375.
25. Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379-423.
26. Singh, H., Misra, N., Hnizdo, V., Fedorowicz, A., & Demchuk, F. (2003). Nearest neighbor estimates of entropy. Amer. J. Math. Management Sci., 23, 301-321.
27. van Es, B. (1992). Estimating functionals related to a density by a class of statistics based on spacings. Scand. J. Statist., 19, 61-72.
28. Vasicek, O. (1976). A test for normality based on sample entropy. J. Royal Statistical Society, Series B, 38(1), 54-59.
29. Wieczorkowski, R., & Grzegorzewski, P. (1999). Entropy estimators-improvements and comparisons. Communications in Statistics-Simulation and Computation, 28(2), 541-567.