An analysis of entropy estimators for blind source separation

Signal Processing

Volumn 86, Issue 1, 2006, Pages 182-194

  Hild II, Kenneth E ^a   Erdogmus, Deniz ^b   Principe, Jose C ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^c University of Florida   (United States)

Author keywords

Blind source separation; Independent component analysis; Information theoretic learning; Kurtosis; Renyi's quadratic entropy

Indexed keywords

COMPUTATIONAL COMPLEXITY; ENTROPY; GAUSSIAN NOISE (ELECTRONIC); INDEPENDENT COMPONENT ANALYSIS; INFORMATION THEORY; RANDOM PROCESSES; STATISTICAL METHODS;

GRADIENT ESTIMATOR; INFORMATION THEORETIC LEARNING; KURTOSIS; RENYI'S QUADRATIC THERAPY;

BLIND SOURCE SEPARATION;

EID: 27944489215     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2005.04.015     Document Type: Article

References (27)

1. J.F. Cardoso, Blind signal separation: statistical principles, Proc. IEEE 86(10) (October 1998) 2009-2025.
2. A. Hyvarinen, Survey on independent component analysis, Neural Comput. Surv. 2, (1999) 94-128, http://www.cse.ucsc.edu/NCS/.
3. K. Hild II, et al., Blind source separation using Renyi's mutual information, IEEE Signal Processing Lett. 8(6) (June 2001) 174-176.
4. A. Renyi Probability Theory, North-Holland Publishing Company 1970 Amsterdam Netherlands (Chapter 9)
5. E. Parzen On estimation of a probability function and mode Ann. Math. Statist. 33 3 1962 1065 1076
6. T. Cover, J. Thomas, Elements of Information Theory, Wiley, New York, NY, 1991, pp. 20-21 (Chapter 2).
7. C.E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J. (July 1948) 379-423; (October 1948) 623-656.
8. J.C. Principe, et al., Learning from examples with information theoretic criteria, J. VLSI Signal Processing Systems 26(1/2) (August 2000) 61-77.
9. D. Erdogmus, et al., Blind source separation using Renyi's alpha-marginal entropies, Neurocomputing 49(1) (December 2002) 25-38.
10. K. Hild II Blind separation of convolutive mixtures using Renyi's divergence 2003 Ph.D. Dissertation University of Florida pp. 3-39, 148-159
11. D. Erdogmus et al., On-line entropy manipulation: stochastic information gradient, IEEE Signal Processing Lett. 10(8) (August 2003) 242-245.
12. P. Viola, et al., Empirical entropy manipulation for real-world problems, Advances in Neural Information Processing Systems, Denver, CO, 27-30 November 1995, pp. 851-857.
13. K. Hild II, et al., Blind source separation of time-varying, instantaneous mixtures using an on-line algorithm, International Conference on Acoustics, Speech, and Signal Processing, Orlando, FL, 13-17 May 2002, pp. 993-996.
14. C. Bourin, P. Bondon, Efficiency of high-order moment estimators, IEEE Trans. Signal Process. 46(1) (January 1998) 255-258.
15. D. Erdogmus, J.C. Principe, Generalized information potential criterion for adaptive system training, IEEE Trans. Neural Networks 13(5) (September 2002) 1035-1044.
16. D. Pham, Contrast functions for blind separation and deconvolution of sources, International Workshop on Independent Component Analysis and Signal Separation, San Diego, CA, 9-12 December 2001, pp. 37-42
17. B.A. Pearlmutter, L.C. Parra, Maximum likelihood blind source separation: a context-sensitive generalization of ICA, Advances in Neural Information Processing Systems, vol. 9, MIT Press, Cambridge, MA, December 1996, pp. 613-619.
18. C.D. Giurcaneanu, I. Tabus, On the sign of kurtosis, International Conference on Acoustics, Speech, and Signal Processing, Helsinki, Finland, 19-22 June 2000, pp. 499-502.
19. J.F. Cardoso, A. Souloumiac, Blind beamforming for non-Gaussian signals, Radar Signal Processing IEE Proc F 140(6) (December 1993) 362-370.
20. A. Hyvarinen, Fast and robust fixed-point algorithms for independent component analysis, IEEE Trans Neural Networks 10(3) (May 1999) 626-634.
21. P. Comon, Independent component analysis, a new concept?, Signal Processing 36(3) (April 1994) 287-314.
22. A. Bell and T. Sejnowski, An information-maximization approach to blind separation and blind deconvolution, Neural Comput. 7(6) (November 1995) 1129-1159.
23. D. Pham, Fast algorithm for estimating mutual information, entropies, and score functions, International Workshop on Independent Component Analysis and Signal Separation, Nara, Japan, 1-4 April 2003, pp. 17-22.
24. H. Yang, S. Amari, Adaptive online learning algorithms for blind separation: maximum entropy and minimum mutual information, Neural Comput. 9(7) (October 1997) 1457-1482.
25. J.F. Cardoso, Infomax and maximum likelihood for blind source separation, IEEE Signal Processing Lett. 4(4) (April 1997) 112-114.
26. S. Amari, "Neural learning in structured parameter spaces - natural Riemannian gradient", Advances in Neural Information Proc. Systems, Denver, CO, 2-5 December 1996, pp. 127-133.
27. J.F. Cardoso, B. Laheld, Equivariant adaptive source separation, IEEE Trans. Signal Processing 44(12) (December 1996) 3017-3030.