A class of neural networks for independent component analysis

IEEE Transactions on Neural Networks

Volumn 8, Issue 3, 1997, Pages 486-504

  Karhunen, Juha ^a,b   Oja, Erkki ^a,b   Wang, Liuyue ^b   Vigário, Ricardo ^b   Joutsensalo, Jyrki ^b  

^a IEEE   (Finland)

^b AALTO UNIVERSITY   (Finland)

Author keywords

Blind source separation; Independent component analysis; Neural networks; Principal component analysis; Signal processing; Unsupervised learning

Indexed keywords

IMAGE SEGMENTATION; LEARNING ALGORITHMS; STATISTICAL METHODS; VECTORS;

BLIND SOURCE SEPARATION; INDEPENDENT COMPONENT ANALYSIS (ICA); PRINCIPAL COMPONENT ANALYSIS (PCA);

FEEDFORWARD NEURAL NETWORKS;

EID: 0031139249     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.572090     Document Type: Article

References (53)

1. S. Amari, A. Cichocki, and H. Yang, "A new learning algorithm for blind signal separation," in Advances in Neural Information Processing Systems 8, D. S. Touretzky et al., Eds. Cambridge, MA: MIT Press, 1996, pp. 757-763.
2. P. Baldi and K. Hornik, "Neural networks for principal component analysis: Learning from examples without local minima," Neural Networks, vol. 2, pp. 53-58, 1989.
3. _, "Learning in linear neural networks: A survey," IEEE Trans. Neural Networks, vol. 6, pp. 837-858, 1995.
4. A. Bell and T. Sejnowski, "An information-maximization approach to blind separation and blind deconvolution," Neural Computa., vol. 7, pp. 1129-1159, 1995.
5. _, "Learning the higher order structure of a natural sound," Network: Computa. Neural Syst., vol. 7, pp. 261-267, 1996.
6. _, "Edges are the independent components of natural scenes," to appear in Advances in Neural Information Processing Systems 9. Cambridge, MA: MIT Press, 1997.
7. G. Burel, "Blind separation of sources: A nonlinear neural algorithm," Neural Networks, vol. 5, pp. 937-947, 1992.
8. G. Burel and N. Rondel, "Neural networks for array processing: From DOA estimation to blind separation of sources," in Proc. 1993 IEEE Int. Conf. Syst., Man, Cybern., Le Touquet, France, Oct. 1993, pp. 601-606.
9. J.-F. Cardoso and B. Laheld, "Equivariant adaptive source separation," IEEE Trans. Signal Processing, vol. 44, pp. 3017-3030, Dec. 1996.
10. A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing. New York: Wiley, 1993.
11. A. Cichocki, W. Kasprzak, and S. Amari, "Multilayer neural networks with a local adaptive learning rule for blind separation of source signals," in Proc. 1995 Int. Symp. Nonlinear Theory Applicat., NOLTA-95, Las Vegas, NV, vol. 1, Dec. 1995, pp. 61-66.
12. P. Comon, "Separation of stochastic processes," in Proc. Wkshp. Higher Order Spectral Analysis, Vail, CO, June 1989, pp. 174-179.
13. _, "Independent component analysis - A new concept?," Signal Processing, vol. 36, pp. 287-314, 1994.
14. G. Deco and D. Obradovic, An Information-Theoretic Approach to Neural Computing. New York: Springer-Verlag, 1996.
15. K. Diamantaras and S. Kung, Principal Component Networks - Theory and Applications. New York: Wiley, 1996.
16. J. Friedman, "Exploratory projection pursuit," J. Amer. Statist. Assoc., vol. 82, no. 397, pp. 249-266, Mar. 1987.
17. C. Fyfe and R. Baddeley, "Nonlinear data structure extraction using simple Hebbian networks," Biol. Cybern., vol. 72, pp. 533-541, 1995.
18. S. Haykin, Neural Networks: A Comprehensive Foundation. New York: IEEE Computer Soc. Press and Macmillan, 1994.
19. J. Herault, C. Jutten, and B. Ans, "Détection de grandeur primitives dans un message composite par une architecture de calcul neuromimétique en apprentissage nonsupervisé, " in Proc. GRETSI Conference, Nice, France, May 1985, pp. 1017-1022, in French.
20. P. Huber, "Projection pursuit," Ann. Statist., vol. 13, no. 2, pp. 435-475, 1985.
21. J. Hum, A. Hyvärinen, J. Karhunen, and E. Oja, "Image feature extraction using independent component analysis," in Proc. 1996 IEEE Nordic Signal Processing Symp., Espoo, Finland, Sept. 1996, pp. 475-478.
22. A. Hyvärinen and E. Oja, "One-unit learning rules for independent component analysis," to appear in Advances in Neural Information Processing Systems 9. Cambridge, MA: MIT Press, 1997.
23. A. Hyvärinen, "A family of fixed-point algorithms for independent component analysis," to appear in Proc. 1997 IEEE Int. Conf. Acoust., Speech, Signal Processing, ICASSP'97, Munich, Germany, Apr. 1997.
24. C. Jutten, "Calcul neuromimétique et traitement du signal, analyze en composantes indépendantes," Thèse d'Etat, INPG, Univ. Grenoble, France, 1987 (in French).
25. C. Jutten and J. Herault, "Independent components analysis (INCA) versus principal components analysis," in Signal Processing IV: Theories and Applications, Proc. EUSIPCO-88, J. Lacoume et al., Eds. Amsterdam, The Netherlands: Elsevier, 1988, pp. 643-646.
26. _, "Blind separation of sources, Part I: An adaptive algorithm based on neuromimetic architecture," Signal Processing, vol. 24, no. 1, pp. 1-10, July 1991.
27. J. Karhunen and J. Joutsensalo, "Representation and separation of signals using nonlinear PCA type learning," Neural Networks, vol. 7, no. 1, pp. 113-127, 1994.
28. J. Karhunen, "Optimization criteria and nonlinear PCA neural networks," in Proc. 1994 IEEE Int. Conf. Neural Networks, Orlando, FL, June 1994, pp. 1241-1246.
29. _, "Stability of Oja's PCA subspace rule," Neural Computa., vol. 6, pp. 739-747, 1994.
30. J. Karhunen and J. Joutsensalo, "Generalizations of principal component analysis, optimization problems, and neural networks," Neural Networks, vol. 8, no. 4, pp. 549-562, 1995.
31. J. Karhunen, "Neural approaches to independent component analysis and source separation," in Proc. 4th European Symp. Artificial Neural Networks, ESANN'96, Bruges, Belgium, Apr. 1996, pp. 249-266.
32. J. Karhunen, A. Hyvärinen, R. Vigário, J. Hurri, and E. Oja, "Applications of neural blind separation to signal and image processing," to appear in Proc. 1997 IEEE Int. Conf. Acoust., Speech, Signal Processing, ICASSP'97, Munich, Germany, Apr. 1997.
33. J. Karhunen and P. Pajunen, "Blind source separation using least-squares type adaptive algorithms," to appear in Proc. 1997 IEEE Int. Conf. Acoust., Speech, Signal Processing, ICASSP'97, Munich, Germany, Apr. 1997.
34. P. Kotilainen, J. Saarinen, and K. Kaski, "Hardware implementations of PCA neural networks," in Artificial Neural Networks, in Proc. ICANN-92, I. Aleksander and J. Taylor, Eds. Amsterdam, The Netherlands: North-Holland, vol. 2, 1992, pp. 1427-1430.
35. B. Laheld and J.-F. Cardoso, "Adaptive source separation with uniform performance," in Signal Processing VII: Theories and Applications Proc. EUSIPCO-94, M. Holt et al., Eds. Lausanne, Switzerland: EURASIP, vol. 2, 1994, pp. 183-186.
36. K. Matsuoka, M. Ohya, and M. Kawamoto, "A neural net for blind separation of nonstationary signals," Neural Networks, vol. 8, no. 3, pp. 411-419, 1995.
37. E. Moreau and O. Macchi, "New self-adaptive algorithms for source separation based on contrast functions," in Proc. IEEE Signal Proc. Wkshp. Higher Order Statistics, Lake Tahoe, NV, June 1993, pp. 215-219.
38. E. Oja, "A simplified neuron model as a principal components analyzer," J. Math. Biol., vol. 15, pp. 267-273, 1982.
39. _, Subspace Methods of Pattern Recognition. Letchworth, U.K.: Res. Studies Press, 1983.
40. E. Oja and J. Karhunen, "On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix," J. Math. Anal. Applicat., vol. 106, pp. 69-84, 1985.
41. E. Oja, H. Ogawa, and J. Wangviwattana, "Learning in nonlinear constrained Hebbian networks," in Artificial Neural Networks Proc. ICANN-91, T. Kohonen et al., Eds. Amsterdam, The Netherlands: North-Holland, 1991, pp. 385-390.
42. E. Oja, "Principal components, minor components, and linear neural networks," Neural Networks, vol. 5, pp. 927-935, 1992.
43. _, "The nonlinear PCA learning rule and signal separation - Mathematical analysis," Lab. Computer Inform. Sci., Helsinki Univ. Technology, Espoo, Finland, Tech. Rep. A26, Aug. 1995.
44. J. Platt and F. Faggin, "Networks for the separation of sources that are superimposed and delayed," in Advances in Neural Information Processing Systems 4, J. Moody et al., Eds. San Mateo, CA: Morgan Kaufmann, 1991, pp. 730-737.
45. M. Plumbley, "A Hebbian/anti-Hebbian network which optimizes information capacity by orthonormalizing the principal subspace," in Proc. IEE Conf. Artificial Neural Networks, Brighton, U.K., May 1993, pp. 86-90.
46. _, "Efficient information transfer and anti-Hebbian neural networks," Neural Networks, vol. 6, pp. 823-833, 1993.
47. F. Silva and L. Almeida, "A distributed solution for data orthonormalization," in Artificial Neural Networks Proc. ICANN-91, T. Kohonen et al., Eds. Amsterdam, The Netherlands: North-Holland, 1991, pp. 943-948.
48. J. Taylor and M. Plumbley, "Information theory and neural networks," in Mathematical Approaches to Neural Networks, J. Taylor, Ed. Ams-terdam, The Netherlands: Elsevier, 1993, pp. 307-340.
49. C. Themen, Discrete Random Signals and Statistical Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1992.
50. K. Torkkola, "Blind separation of convolved sources based on information maximization," in Proc. IEEE Wkshp. Neural Networks Signal Processing, Kyoto, Japan, Sept. 1996, pp. 423-432.
51. R. Vigário, A. Hyvärinen, and E. Oja, "ICA fixed-point algorithm in extraction of artifacts from EEG," in Proc. 1996 IEEE Nordic Signal Processing Symp., Espoo, Finland, Sept. 1996, pp. 383-386.
52. L. Wang and J. Karhunen, "A unified neural bigradient algorithm for robust PCA and MCA," Int. J. Neural Syst., vol. 7, pp. 53-67, 1996.
53. L. Wang, J. Karhunen, and E. Oja, "A bigradient optimization approach for robust PCA, MCA, and source separation," in Proc. 1995 IEEE Int. Conf. Neural Networks, Perth, Australia, Nov. 1995, pp. 1684-1689.