Fast kernel entropy estimation and optimization

Signal Processing

Volumn 85, Issue 5 SPEC. ISS., 2005, Pages 1045-1058

  Shwartz, Sarit ^a   Zibulevsky, Michael ^a   Schechner, Yoav Y ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Automatic differentiation; Blind source separation; Independent component analysis; Mutual information; Non parametric entropy estimation; Parzen windows

Indexed keywords

ALGORITHMS; ASYMPTOTIC STABILITY; BLIND SOURCE SEPARATION; ENTROPY; ESTIMATION; INDEPENDENT COMPONENT ANALYSIS; INFORMATION THEORY; ITERATIVE METHODS; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; SENSITIVITY ANALYSIS; STATISTICAL METHODS; VECTORS;

AUTOMATIC DIFFERENTIATION; MUTUAL INFORMATION; NON-PARAMETRIC ENTROPY ESTIMATION; PARZEN WINDOWS;

SIGNAL PROCESSING;

EID: 14844348800     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2004.11.022     Document Type: Article

References (39)

1. A.J. Bell, and T.J. Sejnowski An information-maximization approach to blind separation and blind deconvolution Neural Comput. 7 6 1995 1129 1159
2. R. Boscolo, H. Pan, V. P. Roychowdhury, Non-parametric ICA, in: Proceedings of the ICA2001, pp. 13-18.
3. V.M. Bove Jr. Entropy-based depth from focus J. Opt. Soc. Amer. A 10 1993 561 566
4. J.-F. Cardoso, and A. Souloumiac Blind beamforming for non Gaussian signals IEE Proc. F Radar Signal Process. 140 6 1993 362 370
5. J.-F. Cardoso Blind signal separation statistical principles Proc. IEEE 9 10 1998 2009 2025
6. P. Comon Independent component analysis a new concept Signal Process. 36 1994 287 314
7. T.M. Cover, and J.A. Thomas Elements of Information Theory 1991 Wiley New York
8. J.I. de la Rosa, G. Fleury, On the kernel selection for minimum-entropy estimation, in: Proceedings of the IEEE Instrumentation and Measurement Technology, vol. 2, 2002, pp. 1205-1210.
9. R.O. Duda, P.E. Hart, and D.G. Stock Pattern Classification 2001 Wiley New York
10. D. Erdogmus, and J.C. Principe An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems IEEE Trans. Signal Process. 50 7 2002 1780 1786
11. D. Erdogmus, J.C. Principe, and K.E. Hild II On-line entropy manipulation Stochastic information gradient IEEE Signal Process. Lett. 10 8 2003 242 245
12. J. Fan, and J. Marron Fast implementation of nonparametric curve estimators J. Comput. Graph. Statist. 3 1994 35 56
13. A. Gray, A. Moore, Nonparametric density estimation: toward computational tractability, SIAM Data Mining.
14. A. Gray, A. Moore, Rapid evaluation of multiple density models, in: AI and Statistics, 2003.
15. A. Griewank Evaluating Derivatives Principles and Techniques of Algorithmic Differentiation 2000 SIAM Philadelphia
16. A. Hyvärinen, The Fast-ICA MATLAB package http://www.cis.hut.fi/ ∼ aapo/.
17. A. Hyvärinen, J. Karhunen, and E. Oja Independent Component Analysis 2001 Wiley New York
18. E. Learned-Miller, and J. Fisher ICA using spacings estimates of entropy J. Mach. Learn. Res. 4 2003 1271 1295
19. Y. Lomnitz, A. Yeredor, A blind-ML scheme for blind source separation, in: Proceedings of the IEEE Workshop on Statistical Signal Processing, 2003.
20. D.G. Luenberger Linear and Non Linear Programming 1987 Addison-Wesley California
21. S. Makeig, A.J. Bell, T.-P. Jung, and T.J. Sejnowski Independent component analysis of electroencephalographic data Adv. Neural Inform. Process. Systems 8 1996 145 151
22. B.A. Pearlmutter Fast exact multiplication by the hessian Neural Comput. 6 1 1994 147 160
23. D.T. Pham, Fast algorithm for estimating mutual information, entropies and score functions, in: Proceedings of the ICA2003, pp. 17-22.
24. D.T. Pham, and P. Garrat Blind separation of a mixture of independent sources through a quasi-maximum likelihood approach IEEE Trans. Signal Process. 45 7 1997 1712 1725
25. J.C. Principe, D. Erdogmus, From adaptive linear to information filtering, in: Proceedings of the IEEE Symposium on Adaptive Systems for Signal Processing, Communications and Control, 2000, pp. 99-104.
26. L.B. Rall Automatic Differentiation Techniques and Applications 1981 Springer Berlin
27. I. Santamaria, D. Erdogmus, and J.C. Principe Entropy minimization for supervised digital communications channel equalization IEEE Trans. Signal Process. 50 5 2002 1184 1192
28. Y.Y. Schechner, N. Kiryati, and R. Basri Separation of transparent layers using focus Int. J. Comput. Vision 89 2000 25 39
29. Y.Y. Schechner, J. Shamir, and N. Kiryati Polarization and statistical analysis of scenes containing a semi-reflector J. Opt. Soc. Amer. A 17 2000 276 284
30. N.N. Schraudolph Gradient-based manipulation of nonparametric entropy estimates IEEE Trans. Neural Networks 15 4 2004 828 837
31. W. Schwartzkopf, B.L. Evans, A.C. Bovik, Entropy estimation for segmentation of multi-spectral chromosome images, in: Proceedings of the IEEE Southwest Symposium on Image Analysis and Interpretation, 2002, pp. 234-257.
32. B.W. Silverman Kernel density estimation using the fast Fourier transform J. Roy. Statist. Soc. Ser. C Appl. Statist. 31 1 1982 93 99
33. B.W. Silverman Density Estimation for Statistics and Data Analysis 1986 Chapman & Hall New York
34. S. Shwartz, M. Zibulevsky, Y.Y. Schechner, ICA using kernel entropy estimation with NlogN complexity, in: Proceedings of the ICA2004.
35. P. Thevenaz, and M. Unser Optimization of mutual information for multiresolution image registration IEEE Trans. Image Process. 9 12 2000 2083 2099
36. O. Vasicek A test for normality based on sample entropy J. Roy. Statist. Soc. Ser. B 38 1 1976 54 59
37. P.A. Viola, Alignment by maximization of mutual information, Ph.D. Thesis, Massachusetts Institute of Technology - Artificial Intelligence Laboratory, 1995.
38. M.P. Wand Fast computation of multivariate kernel estimators J. Comput. Graph. Statist. 3 1994 433 445
39. R.R. Wang, T. Huang, Jialin-Zhong, Generative and discriminative face modelling for detection, in: Proceedings of the IEEE International Conference Automatic Face Gesture Recognition, 2002, pp. 281-286.