Complex independent component analysis by entropy bound minimization

IEEE Transactions on Circuits and Systems I: Regular Papers

Volumn 57, Issue 7, 2010, Pages 1417-1430

  Li, Xi Lin ^a   Adali, Tülay ^a  

^a University of Marylandw Baltimore County   (United States)

Author keywords

Complex optimization; complex random variable; differential entropy; independent component analysis (ICA); neural networks; principle of maximum entropy

Indexed keywords

COMPLEX NETWORKS; COMPUTATIONAL EFFICIENCY; INDEPENDENT COMPONENT ANALYSIS; MAXIMUM ENTROPY METHODS; NEURAL NETWORKS; OPTIMIZATION; RANDOM VARIABLES;

COMPLEX OPTIMIZATION; COMPLEX RANDOM VARIABLES; DIFFERENTIAL ENTROPY; INDEPENDENT COMPONENT ANALYSIS(ICA); PRINCIPLE OF MAXIMUM ENTROPY;

MINIMUM ENTROPY METHODS;

EID: 77954865269     PISSN: 15498328     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSI.2010.2046207     Document Type: Article

References (53)

1. T. Adal?, H. Li, M. Novey, and J. F. Cardoso, "Complex ICA using nonlinear functions," IEEE Trans. Signal Process., vol.56, no.9, pp. 4536-4544, Sep. 2008.
2. L. D. Lathauwer and B. D. Moor, "On the blind separation of noncircular sources," in Proc. EUSIPCO, Toulouse, France, 2002.
3. J. Eriksson and V. Koivunen, "Complex-valued ICA using second order statistics," in Proc. IEEE Int. Workshop MLSP, Sao Luis, Brazil, 2004, pp. 183-192.
4. E. Ollila and V. Koivunen, "Complex ICA using generalized uncorrelating transform," Signal Process., vol.39, no.4, pp. 365-377, Apr. 2009.
5. J. F. Cardoso and A. Souloumiac, "Blind beamforming for non-Gaussian signals," Proc. Inst. Elect. Eng.-F, vol.140, no.6, pp. 362-370, Dec. 1993.
6. S. C. Douglas, "Fixed-point algorithms for the blind separation of arbitrary complex-valued non-Gaussian signal mixtures," EURASIP J. Appl. Signal Process., vol.2007, no.1, p. 83, Jan. 2007.
7. H. Li and T. Adal?, "A class of complex ICA algorithms based on the kurtosis cost function," IEEE Trans. Neural Netw., vol.19, no.3, pp. 408-420, Mar. 2008.
8. P. Smaragdis, "Blind separation of convolved mixtures in the frequency domain," Neurocomputing, vol.22, no.1-3, pp. 21-34, Nov. 1998.
9. E. Bingham and A. Hyvärinen, "A fast fixed-point algorithm for independent component analysis of complex valued signals," Int. J. Neural Syst., vol.10, no.1, pp. 1-8, Feb. 2000.
10. H. Sawada, R. Mukai, S. Araki, and S. Makino, "A polar-coordinate based activation function for frequency domain blind source separation," in Proc. ICASSP, May 2002, pp. 1001-1004.
11. J. Anemüller, T. J. Sejnowski, and S. Makeig, "Complex independent component analysis of frequency-domain electroencephalographic data," Neural Netw., vol.16, no.9, pp. 1311-1323, Nov. 2003.
12. J. F. Cardoso and T. Adal?, "The maximum likelihood approach to complex ICA," in Proc. ICASSP, Toulouse, France, May 2006, vol.5, pp. 673-676.
13. M. Novey and T. Adal?, "Adaptable nonlinearity for complex maximization of nonGaussianity and a fixed-point algorithm," in Proc. IEEE Int. Workshop MLSP, Maynooth, Ireland, Sep. 2006, pp. 79-84.
14. M. Novey and T. Adal?, "Complex ICA by negentropy maximization," IEEE Trans. Neural Netw., vol.19, no.4, pp. 596-609, Apr. 2008.
15. M. Novey and T. Adal?, "On extending the complex FastICA algorithm to noncircular sources," IEEE Trans. Signal Process., vol.56, no.5, pp. 2148-2154, May 2008.
16. T. Adal?, T. Kim, and V. Calhoun, "Independent component analysis by complex nonlinearities," in Proc. ICASSP, Montreal, QC, Canada, May 2004, vol.5, pp. 525-528.
17. P. Comon, "Independent component analysis: A new concept?," Signal Process., vol.36, no.3, pp. 287-314, Apr. 1994.
18. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York: Wiley, 2001.
19. A. Cichocki and S. Amari, Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. Chichester, U.K.: Wiley, 2002.
20. Ö A. Hyvärinen and E. Oja, "Independent component analysis: Algorithms and applications," Neural Netw., vol. 13, no. 4/5, pp. 411-430, May/Jun. 2000. (Pubitemid 30447427)
21. J. Karvanen, J. Eriksson, and V. Koivunen, "Pearson system based method for blind separation," in Proc. 2nd Int.Workshop Ind. Compon. Anal. Blind Signal Separation, Helsinki, Finland, 2000, pp. 585-590.
22. Z. Koldovsky, P. Tichavsky, and E. Oja, "Efficient variant of algorithm FastICA for independent component analysis attaining the Cramér-Rao lower bound," IEEE Trans. Neural Netw., vol.17, no.5, pp. 1265-1277, Sep. 2006.
23. D. T. Pham and P. Garat, "Blind separation of mixture of independent sources through a quasi-maximum likelihood approach," IEEE Trans. Signal Process., vol.45, no.7, pp. 1712-1725, Jul. 1997.
24. A. Bell and T. Sejnowski, "An information-maximization approach to blind separation and blind deconvolution," Neural Comput., vol.7, no.6, pp. 1129-1159, Nov. 1995.
25. T.-W. Lee, M. Girolami, and T. J. Sejnowski, "Independent component analysis using an extended infomax algorithm for mixed sub-Gaussian and super-Gaussian sources," Neural Comput., vol.11, no.2, pp. 417-441, Feb. 1999.
26. A. Hyvärinen, "New approximations of differential entropy for independent component analysis and projection pursuit," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1998, pp. 273-279.
27. D. Erdogmus, K. E. Hild, II, Y. N. Rao, and J. C. Principe, "Minimax mutual information approach for independent component analysis," Neural Comput., vol.16, no.6, pp. 1235-1252, Jun. 2004. (Pubitemid 38640623)
28. R. Boscolo, H. Pan, and V. P. Roychowdhury, "Independent component analysis based on nonparametric density estimation," IEEE Trans. Neural Netw., vol.15, no.1, pp. 55-65, Jan. 2004.
29. E. G. Learned-Miller, J.W. Fisher, III, and T.-W. Lee, "ICA using spacings estimates of entropy," J. Mach. Learn. Res., vol.4, pp. 1271-1295, 2003.
30. B. W. Silverman, Density Estimation for Statistics and Data Analysis. London, U.K.: Chapman & Hall, 1986.
31. J. Beirlant, E. Dudewicz, L. Gyorfi, and E. van der Meulen, "Nonparametric entropy estimation: An overview," Int. J. Math. Statist. Sci., vol.6, no.1, pp. 17-39, Jun. 1997.
32. S. Fiori, "A theory for learning by weight flow on Stiefel-Grassman manifold," Neural Comput., vol.13, no.7, pp. 1625-1647, Jul. 2001.
33. E. T. Jaynes, "Information theory and statistical mechanics," Phys. Rev., vol.106, no.4, pp. 620-630, May 1957.
34. J. F. Cardoso, "On the performance of orthogonal source separation algorithms," in Proc. Eur. Assoc. Signal Process. VII, Edinburgh, U.K., 1994, pp. 776-779.
35. L. De Lathauwer, B. De Moor, and J. Vandewalle, "A prewhiteninginduced bound on the identification error in independent component analysis," IEEE Trans. Circuits Syst. I, Reg. Papers, vol.52, no.3, pp. 546-554, Mar. 2005.
36. B. Picinbono, "On circularity," IEEE Trans. Signal Process., vol.42, no.12, pp. 3473-3482, Dec. 1994.
37. M. E. Johnson and A. Tenenbein, "A bivariate distribution family with specified marginals," J. Amer. Stat. Assoc., vol.76, no.373, pp. 198-201, Mar. 1981.
38. D. Long and R. Krzysztofowicz, "A family of bivariate densities constructed from marginals," J. Amer. Stat. Assoc., vol.90, no.430, pp. 739-746, Jun. 1995.
39. K. T. Fang, S.Kotz, and K. W. Ng, Symmetric Multivariate and Related Distributions. London, U.K.: Chapman & Hall, 1990.
40. A. van den Bos, "The multivariate complex normal distribution-A generalization," IEEE Trans. Inf. Theory, vol.41, no.2, pp. 537-539, Mar. 1995.
41. P. R. Krishnaiah and J. Lin, "Complex elliptically symmetric distributions," Commun. Statist.-Theory Methods, vol.15, no.12, pp. 3693-3718, 1986.
42. E. Ollila and V. Koivunen, "Generalized complex elliptical distributions," in Proc. Sensor Array Multichannel Signal Process. Workshop, 2004, pp. 460-464.
43. W. Wirtinger, "Zur formalen theorie der funktionen von mehr komplexen veränderlichen," Math. Ann., vol.97, no.1, pp. 357-375, Dec. 1927.
44. M. Jones and R. Sibson, "What is projection pursuit?," J. R. Stat. Soc. A, vol.150, no.1, pp. 1-36, 1987.
45. , S. Haykin, Ed., Unsupervised Adaptive Filtering, Volume 1, Blind Source Separation. New York: Wiley, 2000.
46. O. Shalvi and E. Weinstein, "Super-exponential method for blind deconvolution," IEEE Trans. Inf. Theory, vol.39, no.2, pp. 504-519, Mar. 1993.
47. X.-L. Li and T. Adal?, "A novel entropy estimator and its application to ICA," in Proc. IEEE Workshop Mach. Learn. Signal Process., Grenoble, France, Sep. 2009, pp. 1-6.
48. X.-L. Li, "A new gradient search interpretation of super-exponential algorithm," IEEE Signal Process. Lett., vol.13, no.3, pp. 173-176, Mar. 2006.
49. V. Zarzoso and P. Comon, "Comparative speed analysis of FastICA," in Proc. 7th Int. Conf. Ind. Compon. Anal. Signal Separation, 2007, pp. 293-300.
50. S. Amari, A. Cichocki, and H. H. Yang, "A new learning algorithm for blind signal separation," in Advances in Neural Information Processing Systems 1995. Cambridge, MA: MIT Press, 1996, pp. 752-763.
51. X.-L. Li and X.-D. Zhang, "Nonorthogonal joint diagonalization free of degenerate solution," IEEE Trans. Signal Process., vol.55, no.5, pp. 1803-1814, May 2007.
52. D. H. Brandwood, "A complex gradient operator and its application in adaptive array theory," Proc. Inst. Elect. Eng., vol.130, no.1, pt. F and H, pp. 11-16, Feb. 1983.
53. G. H. Golub and C. F. Van Loan, Matrix Computations. Bethesda, MD: Johns Hopkins Univ. Press, 1989.