Minimax mutual information approach for independent component analysis

Neural Computation

Volumn 16, Issue 6, 2004, Pages 1235-1252

  Erdogmus, Deniz ^a   Hild II, Kenneth E ^a   Rao, Yadunandana N ^a   Príncipe, José C ^a  

^a University of Florida   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPARATIVE STUDY; ENTROPY;

ALGORITHMS; ENTROPY; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 2442463272     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976604773717595     Document Type: Article

References (45)

1. Almeida, L. B. (2000). Linear and nonlinear ICA based on mutual information. In Proceedings of AS-SPCC'00 (pp. 117-122). Piscataway, NJ: IEEE Press.
2. Amari, S. I. (1985). Differential-geometrical methods in statistics. Berlin: Springer-Verlag.
3. Amari, S. I. (1997). Neural learning in structured parameter spaces-natural Riemmanian gradient. In M. Mozer, M. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 127-133). Cambridge, MA: MIT Press.
4. Amari, S. I., Cichocki, A., & Yang, H. H. (1996). A new learning algorithm for blind signal separation. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 757-763). Cambridge, MA: MIT Press.
5. Barndorff-Nielsen, O. E. (1978). Information and exponential families in statistical theory. New York: Wiley.
6. Bell, T., & Sejnowski, T. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129-1159.
7. Cardoso, J. F. (1994). On the performance of orthogonal source separation algorithms. In Proceedings of EUSIPCO'94 (pp. 776-779). London: EURASIP.
8. Cardoso, J. F. (1998). Blind signal separation: Statistical principles. In Proceedings of IEEE (pp. 2009-2025). Piscataway, NJ: IEEE Press.
9. Cardoso, J. F. (1999). High-order contrasts for independent component analysis. Neural Computation, 11, 157-192.
10. Cardoso, J. F., & Souloumiac, A. (1993). Blind beamforming for non-gaussian signals. IEE Proceedings F Radar and Signal Processing, 140, 362-370.
11. Comon, P. (1994). Independent component analysis, a new concept? Signal Processing, 36, 287-314.
12. Comon, P. (1996). Contrasts for multichannel blind deconvolution. IEEE Signal Processing Letters, 3, 209-211.
13. Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. New York: Wiley.
14. Crain, B. R. (1974). Estimation of distributions using orthogonal expansions. Annals of Statistics, 2, 454-463.
15. Erdogmus, D., Hild II, K. E., & Principe, J. C. (2001). Independent component analysis using Renyi's mutual information and Legendre density estimation. In Proceedings of IJCNN'01 (pp. 2762-2767). Piscataway, NJ: IEEE Press.
16. Erdogmus, D., Hild II, K. E., Rao, Y. N., & Principe, J. C. (2003). Independent component analysis using Jaynes' maximum entropy principle. In Proceedings of ICA'03 (pp. 385-390). Available on-line: http://www.kecl.ntt.co.jp/icl/signal/ica2003/cdrom/index.htm.
17. Girolami, M. (1997). Symmetric adaptive maximum likelihood estimation for noise cancellation and signal estimation. Electronic Letters, 33, 1437-1438.
18. Girolami, M. (2002). Orthogonal series density estimation and the kernel eigen-value problem. Neural Computation, 14, 669-688.
19. Girolami, M., & Fyfe, C. (1997). Kurtosis extrema and identification of independent components: A neural network approach. In Proceedings of ICASSP'97 (pp. 3329-3332). Piscataway, NJ: IEEE Press.
20. Golub, G., & Van Loan, C. (1993). Matrix computation. Baltimore, MD: Johns Hopkins University Press.
21. Hild II, K. E., Erdogmus, D., & Principe, J.C. (2001). Blind source separation using Renyi's mutual information. IEEE Signal Processing Letters, 8, 174-176.
22. Hyvarinen, A. (1998). New approximations of differential entropy for independent component analysis and projection pursuit. In M. Kearns, M. Jordan, & S. Solla (Eds.), Advances in neural processing systems, 10 (pp. 273-279). Cambridge, MA: MIT Press.
23. Hyvarinen, A. (1999a). Survey on independent component analysis. Neural Computing Surveys, 2, 94-128.
24. Hyvarinen, A. (1999b). Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10, 626-634.
25. Jaynes, E. T. (1957). Information theory and statistical mechanics. Physical Review, 106, 620-630.
26. Kapur, J., & Kesavan, H. (1992). Entropy optimization principles and applications. New York: Academic Press.
27. Karvanen, J., Eriksson, J., & Koivunen, V. (2000). Maximum likelihood estimation of ICA model for wide class of source distributions. In Proceedings of NNSP'00 (pp. 445-454). Piscataway, NJ: IEEE Press.
28. Oja, E. (1999). The nonlinear PCA learning rule in independent component analysis. In Proceedings of ICA'99 (pp. 143-148).
29. Parra, L., & Spence, C. (2000). Convolutive blind separation of non-stationary sources. IEEE Transactions on Speech Audio Processing, 46, 320-327.
30. Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065-1076.
31. Pham, D. T. (1996). Blind separation of instantaneous mixture sources via an independent component analysis. IEEE Transactions on Signal Processing, 44, 2768-2779.
32. Pham, D. T. (2001). Blind separation of instantaneous mixture of sources via the gaussian mutual information criterion. Signal Processing, 81, 855-870.
33. Principe, J. C., & Xu, D. (1999). Information theoretic learning using Renyi's quadratic entropy. In Proceedings of ICA'99 (pp. 407-412).
34. Renyi, A. (1970). Probability theory. Amsterdam: North-Holland.
35. Shore, J. E., & Johnson, R. W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Transactions on Information Theory, 26, 26-37.
36. Simon, C., Loubaton, P., Vignat, C., Jutten, C., & d'Urso, G. (1998). Blind source separation of convolutive mixtures by maximizing of fourth-order cumulants: The non-IID case. In Proceedings of ICASSIP'98 (pp. 1584-1588). Piscataway, NJ: IEEE Press.
37. Sun, X., Douglas, S. C. (2001). Adaptive paraunitary filter banks for contrast-based multichannel blind deconvolution. In Proceedings of ICASSP'01 (pp. 2753-2756). Piscataway, NJ: IEEE Press.
38. Torkkola, K. (1996). Blind separation of delayed sources based on information maximization. In Proceedings of NNSP'96 (pp. 1-10). Piscataway, NJ: IEEE Press.
39. Torkkola, K. (1999). Blind separation for audio signals-are we there yet? In Proceedings of ICA'99 (pp. 239-244).
40. Weinstein, E., Feder, M., & Oppenheim, A. (1993). Multi-channel signal separation by decorrelation. IEEE Transactions on Speech Audio Processing, 1, 405-413.
41. Wu, H. C., & Principe, J. C. (1997). A unifying criterion for blind source separation and decorrelation: Simultaneous diagonalization of correlation matrices. In Proceedings of NNSP'97 (pp. 465-505). Piscataway, NJ: IEEE Press.
42. Wu, H. C., & Principe, J. C. (1999a). A gaussianity measure for blind source separation insensitive to the sign of kurtosis. In Proceedings of NNSP'99 (pp. 58-66). Piscataway, NJ: IEEE Press.
43. Wu, H. C., & Principe, J. C. (1999b). Generalized anti-Hebbian learning for source separation. In Proceedings of ICASSP'99 (pp. 1073-1076). Piscataway, NJ: IEEE Press.
44. Xu, D., Principe, J. C., Fisher, J., & Wu, H. C. (1998). A novel measure for independent component analysis. In Proceedings of ICASSP'98 (pp. 1161-1164). Piscataway, NJ: IEEE Press.
45. Yang, H. H., & Amari, S. I. (1997). Adaptive online learning algorithms for blind separation: Maximum entropy and minimum mutual information. Neural Computation, 9, 1457-1482.