Nonlinear complex-valued extensions of hebbian learning: An essay

Neural Computation

Volumn 17, Issue 4, 2005, Pages 779-838

  Fiori, Simone ^a  

^a UNIVERSITY OF PERUGIA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; MATHEMATICS; NONLINEAR SYSTEM; SIGNAL PROCESSING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; MATHEMATICS; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 17044396848     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/0899766053429381     Document Type: Article

References (98)

1. Abbas, H. M., & Fahmy, M. M. (1994). Neural model for Karhunen-Loéve transform with application to adaptive image compression. IEE Proceedings I, Communications, Speech and Vision, 140(2), 135-143.
2. Amari, S. (1977). Neural theory of association and concept formation. Biological Cybernetics, 26, 175-185.
3. Amari, S. & Cichocki, A. (1998). Adaptive blind signal processing: Neural network approaches. Proceedings of the IEEE, 86(10), 2026-2048.
4. Atick, J. J., & Redlich, A. N. (1993). Convergent algorithm for sensory receptive field development. Neural Computation, 5(1), 45-60.
5. Baldi, P. F., & Hornik, K. (1995). Learning in linear neural networks: A survey. IEEE Transactions on Neural Networks, 6(4), 837-858.
6. Bannour, S., & Azimi-Sadjadi, M. R. (1995). Principal component extraction using recursive least squares learning. IEEE Transactions on Neural Networks, 6(2), 457-469.
7. Barlow, H. B. (1989). Unsupervised learning. Neural Computation, 1, 295-311.
8. Barlow, H. B. (1998). Guest editorial: Cerebral predictions. Perception, 27(8), 885-888.
9. Barlow, H. B. (2001). Redundancy reduction revisited. Network: Computation in Neural Systems, 12(3), 241-253.
10. Barlow, H. B., & Földiák, P. (1989). Adaptation and decorrelation in the cortex. In C. Miall, R. M. Durbin, & G. J. Mitchison (Eds.), The computing neuron, (pp. 54-72). Reading, MA: Addison-Wesley.
11. Bechtel, W., & Abrahamsen, A. (1993). Connectionism and the mind. Oxford: Blackwell.
12. Bell, A. J., & Sejnowski, T. J. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6), 1129-1159.
13. Benvenuto, N., & Piazza, F. (1992). On the complex back-propagation algorithm. IEEE Transactions on Signal Processing, 40(4), 967-969.
14. Bertsekas, D. P. (1996). Constrained optimization and Lagrange multiplier methods. Bedford, MA: Athena Scientific.
15. Bingham, E., & Hyvärinen, A. (2000). A fast fixed-point algorithm for independent component analysis of complex valued signals. International Journal of Neural Systems, 10(1), 1-8.
16. Celledoni, E., & Fiori, S. (2004). Neural learning by geometric integration of reduced "rigid-body" equations. Journal of Computational and Applied Mathematics, 172 (2), 247-269.
17. Chen, T.-P., & Amari, S. (2001). Unified stabilization approach to principal and minor components extraction algorithms. Neural Networks, 14(10), 1377-1387.
18. Chen, T.-P., Amari, S., & Lin, Q. (1998). A unified algorithm for principal and minor components extraction. Neural Networks, 11(3), 385-390.
19. Chen, Y., & Hou, C. (1992). High resolution adaptive bearing estimation using a complex-weighted neural network. Proc. of International Conference on Acoustics, Speech and Signal Processing, 2, 317-320.
20. Chow, T. W., & Fang, Y. (2001). Neural blind deconvolution of MIMO noisy channels. IEEE Transactions on Circuits and Systems - Part I,48(1), 116-120.
21. Cichocki, A., & Amari, S. (2002). Adaptive blind signal and image processing. New York: Wiley.
22. Costa, S., & Fiori, S. (2001). Image compression using principal component neural networks. Image and Vision Computing Journal [Special issue], 19(9-10), 649-668.
23. De Castro, M. C. F., De Castro, F. C. C., Amaral, J. N., & Franco, P. R. G. (1998). A complex valued Hebbian learning algorithm. Proc. of International Joint Conference on Neural Networks (pp. 1235-1238) Piscataway, NJ: IEEE.
24. Desodt, G., & Muller, D. (1990). Complex ICA applied to the separation of radar signals. Proc. of Signal Processing V: Theories and Applications (EUSIPCO), 1, 665-668.
25. Diamantaras, K. I., & Kung, S. Y. (1996). Principal component neural networks: Theory and applications. New York: Wiley.
26. Douglas, S. C., Amari, S. & Kung, S. Y. (2000). On gradient adaptation with unit-norm constraints. IEEE Transactions on Signal Processing, 48(6), 1843-1847.
27. Fiori S. (2000). Blind separation of circularly distributed source signals by the neural extended APEX algorithm. Neurocomputing, 34(1-4), 239-252.
28. Fiori S. (2001). A theory for learning by weight flow on Stiefel-Grassman manifold. Neural Computation, 13(7), 1625-1647.
29. Fiori S. (2002). A theory for learning based on rigid bodies dynamics. IEEE Transactions on Neural Networks, 13(3), 521-531.
30. Fiori S. (2003a). Extended Hebbian learning for blind separation of complex-valued sources. IEEE Transactions on Circuits and Systems - Part II, 50(4), 195-202.
31. Fiori S. (2003b). A neural minor component analysis approach to robust constrained beamforming. IEE Proceedings - Vision, Image and Signal Processing, 250(4), 205-218.
32. Fiori S. (2003c). Neural ICA by "maximum-mismatch" learning principle. Neural Networks, 26(8), 1201-1221.
33. Fiori S. (2003d). Fully-multiplicative orthogonal-group ICA neural algorithm. Electronics Letters, 39(24), 1737-1738.
34. Fiori S. (2004). A fast fixed-point neural blind deconvolution algorithm. IEEE Transactions on Neural Networks, 15(2), 455-459.
35. Fiori S. (in press). Fixed-point neural independent component analysis algorithms on the orthogonal group. Journal of Future Generation Computer Systems.
36. Fiori, S., Faba, A., Albini, L., Cardelli, E., & Burrascano, P. (2003). Numerical modeling for the localization and the assessment of electromagnetic field sources. IEEE Transactions on Magnetics, 39(3), 1638-1641.
37. Fiori, S., & Piazza, F. (2000). A general class of ψ-APEX PCA neural algorithms. IEEE Transactions on Circuits and Systems - Part I, 47(9), 1394-1398.
38. Földiák P. (1990). Forming sparse representations by local anti-Hebbian learning. Biological Cybernetics, 64, 165-170.
39. Fyfe, C., & MacDonald, D. (2002). e-insensitive Hebbian learning. Neurocomputing, 47(1-4), 35-57.
40. Gao, K., Ahmed, M. O., & Swamy, M. N. S. (1994). A constrained anti-Hebbian learning algorithm for total least-squares estimation with applications to adaptive FIR and IIR filtering. IEEE Transactions on Circuits and Systems - Part II, 41 (11), 718-729.
41. Georgiou, G. M., & Koutsougeras, C. (1992). Complex-domain backpropagation. IEEE Transactions on Circuits and Systems - Part II, 39(5), 330-334.
42. Hairer, E., Lubich, C., & Wanner (2002). Geometric numerical integration: Structure preserving algorithms for ordinary differential equations, Berlin: Springer-Verlag.
43. Hanna, A. I., & Mandic, D. P. (2003). A complex-valued nonlinear neural adaptive filter with a gradient adaptive amplitude of the activation function. Neural Networks, 16(2), 155-159.
44. Harpur, G. F. (1997). Low entropy coding with unsupervised neural networks, Unpublished doctoral disseratation, University of Cambridge.
45. Haykin, S. (1989). An introduction to analog and digital communications. New York: Wiley.
46. Haykin, S. (1998). Neural networks: A comprehensive foundation, (2nd eds). Upper Saddle River, NJ: Prentice Hall.
47. Hebb, D. (1949). The organization of behaviour. New York: Wiley.
48. Helmke, U., & Moore, J. B. (1993). Optimization and dynamical systems. Berlin: Springer-Verlag.
49. Higuchi, I., & Eguchi, S. (2004). Robust principal component analysis with adaptive selection for tuning parameters. Journal of Machine Learning Research, 5, 453-471.
50. Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10(3), 626-634.
51. Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley.
52. Iserles, A., Munthe-Kaas, H. Z., Nørsett, S. P., & Zanna, A. (2000). Lie-group methods. Acta Numerica, 9, 215-365.
53. Karhunen, J., & Joutsensalo, J. (1994). Representation and separation of signals using nonlinear PCA type learning. Neural Networks, 7(1), 113-127.
54. Karhunen, J., & Joutsensalo, J. (1995). Generalizations of PCA, optimization problems, and neural networks. Neural Networks, 8(4), 549-562.
55. Kates, J. M. (1993). Superdirective arrays for hearing aids. Journal of Acoustics Society of America, 94(4), 1930-1933.
56. Klemm, R. (1987). Adaptive airborne MTI: An auxiliary channel approach. IEE Proceedings F, 134, 269-276.
57. Kung, S.-I., Diamantaras, K. I., & Taur J. S. (1994). Adaptive principal component extraction (APEX) and applications. IEEE Transactions on Signal Processing, 42(5), 1202-1217.
58. Laheld, B., & Cardoso, J. F. (1994). Adaptive source separation with uniform performance. Proc. of Signal Processing VII: Theories and Applications (EUSIPCO), 1, 183-186.
59. Levine, D. S., Brown, V. R., & Shirey, T. V. (1999). Oscillations in neural systems. Mahwah, NJ: Erlbaum.
60. Liano, K. (1996). Robust error measure for supervised neural network learning with outliers. JEEE Transactions on Neural Networks, 7(1), 246-250.
61. Linsker, R. (1992), Local synaptic rules suffice to maximize mutual information in a linear network. Neural Computation, 4, 691-702.
62. Luo, F.-L., & Unbehauen R. (1997). Applied neural networks for signal processing. Cambridge: Cambridge University Press.
63. Mathew, G., & Reddy, V. U. (1994). Development and analysis of a neural network approach to Pisarenko's harmonic retrieval method. IEEE Transactions on Signal Processing, 42(3), 663-667.
64. Mathis, H., & Douglas, S. C. (2002). On the existence of universal non-linearities for blind source separation. IEEE Transactions on Signal Processing, 50(5), 1007-1016.
65. Miao, Y., & Hua, Y. (1998). Fast subspace tracking and neural network learning by a novel information criterion. IEEE Transactions on Signal Processing, 46(7), 1967-1979.
66. Michaels, R. B., & Upadhyaya, B. R. (1999). A complex valued neural network with local learning laws. In C. H. Dagli et al. (Eds.), Intelligent engineering systems through artificial neural networks (Vol. 9, pp. 101-109). New York: ASME Press.
67. Miller, K. D., & MacKay, D. J. C. (1994). The role of constraints in Hebbian learning. Neural Computation, 6(1), 100-126.
68. Miyauchi, M., Seki, M., Watanabe, A., & Miyauchi, A. (1993). Interpretation of optical flow through complex neural network. Proc. of the International Conference on Artificial Neural Networks, (pp. 645-650). New York: Springer.
69. Müezzinǒglu, M. K., Güzeliş, C., & Zurada, J. M. (2003). A new design method for the complex-valued multistate Hopfield associative memory. IEEE Transactions on Neural Networks, 14(4), 891-899.
70. Nitta, T. (1997). An extension of the back-propagation algorithm to complex numbers. Neural Networks, 10(8), 1391-1415.
71. Nitta, T. (2000). An analysis of the fundamental structure of complex-valued neurons. Neural Processing Letters, 12(3), 239-246.
72. Nitta, T. (2004). Orthogonality of decision boundaries in complex-valued neural networks. Neural Computation, 16, 73-97.
73. Oja, E. (1982). A simplified neuron model as a principal component analyzer. Journal of Mathematics and Biology, 15, 267-273.
74. Oja, E. (1989). Neural networks, principal components, and subspaces. International Journal of Neural System, 1, 61-68.
75. Oja, E. (1992). Principal components, minor components, and linear neural networks. Neural Networks, 5(6), 927-935.
76. Oja, E. (1994). Beyond PCA: Statistical expansions by nonlinear neural networks. Proc. of the International Conference on Artificial Neural Networks, 2, 1049-1054.
77. O'Leary, D. P. (1990). Robust regression computation using iteratively reweighted least squares. SIAM Journal of Matrix Analysis Application, 11(3), 466-480.
78. Palmieri, F., & Zhu, J. (1995). Self-association and Hebbian learning in linear neural networks. IEEE Transactions on Neural Networks, 6(5), 1165-1184.
79. Pandey, R. (2001). Blind equalization and signal separation using neural networks. Unpublished doctoral disseratation, Indian Institute of Technology, Roorkee, India.
80. Plumbley, M. D. (1992). Efficient information transfer and anti-Hebbian neural networks. Neural Networks, 6(6), 823-833.
81. Plumbley, M. D. (2003). Algorithms for non-negative independent component analysis. IEEE Transactions on Neural Networks, 14(3), 534-543.
82. Rubner, J., & Tavan, P. (1989). A self-organizing network for principal component analysis. Europhysics Letters, 10(7), 693-698.
83. Sanger, T. D. (1989). Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Networks, 2(6), 459-473.
84. Schmidt, R. (1986). Multiple emitter location and signal parameter estimation. IEEE Transactions on Antennas and Propagation, 34(3), 276-280.
85. Shirazi, M. N., Peper, F., & Sawai, H. (1999). Principal component analysis by entropy-likelihood optimization. Transactions of the Information Processing Society of Japan, 40(10), 3638-3644.
86. Song, W., Yilong, L., & Feng, M. (1998). An adaptive robust PCA neural network. In Proc. of International Joint Conference on Neural Networks (pp. 2288-2293). Piscataway, NJ: IEEE.
87. Spratling, M. W., & Johnson, M. H. (2002). Preintegration lateral inhibition enhances unsupervised learning. Neural Computation, 14(9), 2157-2179.
88. Sudjianto, A., & Hassoun, M. H. (1995). Statistical basis of non-linear Hebbian learning and application to clustering. Neural Networks, 8(5), 707-715.
89. Thirion Moreau, N., & Moreau, E. (2002). Generalized criterion for blind multivariate signal equalization. IEEE Signal Processing Letters, 9(2), 72-74.
90. Vegas, J. M., & Zufiria, P. J. (2004). Generalized neural networks for spectral analysis: Dynamics and Lyapunov functions. Neural Networks, 17(2), 233-245.
91. Weingessel, A., & Hornik, K. (2000). Local PCA algorithms. IEEE Transactions on Neural Networks, 11(6), 1242-1250.
92. Widrow, B., & Winter, R. (1988). Neural nets for adaptive filtering and adaptive pattern recognition. IEEE Computer Magazine, 21, 25-39.
93. Xu, L. (1992). Least mean square error reconstruction principle for self-organizing neural nets. Neural Networks, 6(5), 627-648.
94. Xu, L., Oja, E., & Suen, C. Y. (1992). Modified Hebbian learning for curve and surface fitting. Neural Networks, 5(3), 441-457.
95. Xu, L., & Yuille, A. L. (1995). Robust PCA by self-organizing rules based on statistical physics approach. IEEE Transactions Neural Networks, 6(1), 131-143.
96. Yan, W.-Y, Helmke, U., & Moore, J. B. (1994). Global analysis of Oja's flow for neural networks. IEEE Transactions on Neural Networks, 5(5), 674-683.
97. Yang, B. (1995). Projection approximation subspace tracking. IEEE Transactions on Signal Processing, 43(1), 1247-1252.
98. Zufiria, P.J. (2002). On the discrete-time dynamics of the basic Hebbian neural network node. IEEE Transactions on Neural Networks, 13(6), 1342-1352.