An information theory based learning paradigm for linear feature extraction

Neurocomputing

Volumn 12, Issue 2-3, 1996, Pages 203-221

  Obradovic, D ^a   Deco, G ^a  

^a SIEMENS AG   (Germany)

Author keywords

Cumulants; Feature extraction; Independent component analysis; Information theory; Neural networks; Unsupervised learning

Indexed keywords

CONFORMAL MAPPING; FEATURE EXTRACTION; INFORMATION THEORY; LEARNING SYSTEMS; LYAPUNOV METHODS; PROBABILITY DENSITY FUNCTION; STATISTICAL METHODS;

CUMULANTS; EDGEWORTH EXPANSION; GAUSSIAN INPUT DISTRIBUTION; INTERDEPENDENT COMPONENT ANALYSIS; LEARNING PARADIGM; LEARNING RULE; STATISTICAL DEPENDENCE; UNSUPERVISED LEARNING;

NEURAL NETWORKS;

ARTICLE; INFORMATION PROCESSING; LEARNING; LINEAR SYSTEM; MATHEMATICAL ANALYSIS; PRIORITY JOURNAL; PROBABILITY; THEORY;

EID: 0030608110     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/0925-2312(95)00119-0     Document Type: Article

References (25)

1. [1] M.G. Kendall and A. Stuart, The Advanced Theory of Statistics (Charles Griffin, London).
2. [2] G. Zipf, Human Behavior and the Principle of Least Effort (Addison-Wesley, Cambridge, MA, 1949).
3. [3] F. Attneave, Informational aspects of visual perception, Psychological Rev. 61 (1954) 183-193.
4. [4] H. Barlow, Unsupervised learning. Neural Computation 1 (1989) 295-311.
5. [5] H. Barlow, T. Kaushal and G. Mitchison, Finding minimum entropy codes, Neural Computation 1 (1989) 412-423.
6. [6] J. Atick and A. Redlich, Towards a theory of early visual processing, Neural Computation 2 (1990) 308-320.
7. [7] J. Atick and A. Redlich, What does the retina know about natural scenes, Neural Computation 4 (1992) 196-210.
8. [8] A.N. Redlich, Redundancy reduction as a strategy for unsupervised learning, Neural Computation 5 (1993) 289-304.
9. [9] A.N. Redlich, Supervised factorial learning, Neural Computation 5 (1993) 750-766.
10. [10] S. Haykin, Neural Networks; A Comprehensive Foundation (Macmillan College Publishing Company, New York, 1994).
11. [11] A.J. Bell and T.J. Sejnowsi, A non-linear information maximization algorithm that performs blind separation, Neural Information Processing Systems, Denver (1994).
12. [12] P. Comon, Independent component analysis, A new concept? Signal Processing 36 (1994) 287-314.
13. [13] B.C.Y. Wong and I.F. Blake, Detection in multivariate non-Gaussian noise, IEEE Trans. Communications 42 (2/3/4) (1994).
14. [14] A. Benveniste, M. Goursat and G. Ruget, Robust identification of a nonminimum phase system, IEEE Trans. Automatic Control 25 (3) (1980) 385-399.
15. [15] G. Giannakis and A. Swami, On estimating noncausal nonminimum phase ARMA models of non-Gaussian processes, IEEE Trans. Acoustics and Speech Signal Processes 38 (3) (1990) 478-495.
16. [16] G. Giannakis and M.K. Tsatsanis, A unifying maximum-likelihood view of polyspectral measures for non-Gaussian signal classification and estimation, IEEE Trans. Information Theory 38 (2) (1992) 386-406.
17. [17] G. Giannakis, Y. Inouye and J.M. Mendel, Cumulant-based identification of multichannel moving avarage models, IEEE Trans. Automatic Control 34 (1989) 783-787.
18. [18] A. Swami and J.M. Mendel, ARMA systems excited by non-Gaussian processes are not always identifiable, IEEE Trans. Automatic Control 34 (5) (1989) 572-573.
19. [19] A. Swami and J.M. Mendel, ARMA parameter estimation using only output cumulants, IEEE Trans. Acoustics and Speech Signal Processes 38 (1990) 1257-1265.
20. [20] G. Jumarie, Relative Information, Theories and Applications (Springer-Verlag, Berlin, 1990).
21. [21] A. Papoulis, Probability, Random Variables and Stochastic Processes third ed. (McGraw-Hill, New York, 1991).
22. [22] M. Koecher, Lineare Algebra und analytische Geometrie (Springer-Verlag, 1983).
23. [23] G.H. Golub and C.F. van Loan, Matrix Computations (North Oxford Academic, Oxford, England, 1983).
24. [24] J. Karhunen and J. Joutsensalo, Representation and separation of signals using nonlinear PCA type learning, Neural Networks 7 (1) (1994) 113-127.
25. [25] J.A. Neider and R. Mead, A simplex method for function minimization, Computer J. 7 308-313.