Online independent component analysis with local learning rate adaptation

Advances in Neural Information Processing Systems

Volumn , Issue , 2000, Pages 789-795

  Schraudolph, Nicol N ^a   Giannakopolos, Xavier ^a  

^a DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

INDEPENDENT COMPONENT ANALYSIS; MIXTURES;

BLIND SEPARATION; EFFICIENT COMPUTATION; HESSIAN-VECTOR PRODUCTS; INDIVIDUAL LEARNING; LOCAL LEARNING RATES; MIXTURE COEFFICIENTS; ON-LINE ADAPTATION; STOCHASTIC META DESCENTS;

ONLINE SYSTEMS;

EID: 84898942573     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. J. Karhunen and P. Pajunen, "Blind source separation and tracking using nonlinear PCA criterion: A least-squares approach ", in Proc. IEEE Int. Conf. on Neural Networks, Houston, Texas, 1997, pp. 2147-2152.
2. N. Murata, K.-R. Müller, A. Ziehe, and S.-i. Amari, "Adaptive on-line learning in changing environments ", in Advances in Neural Information Processing Systems, M. C. Mozer, M. I. Jordan, and T. Petsche, Eds. 1997, vol. 9, pp. 599-605, The MIT Press, Cambridge, MA.
3. N. N. Schraudolph, "Local gain adaptation in stochastic gradient descent ", in Proceedings of the 9th International Conference on Artificial Neural Networks, Edinburgh, Scotland, 1999, pp. 569-574, IEE, London, ftp://ftp.idsia.ch/pub/nic/smd.ps.gz
4. N. N. Schraudolph, "Online learning with adaptive local step sizes", in Neural Nets - WIRN Vietri-99: Proceedings of the 11th Italian Workshop on Neural Nets, M. Marinaro and R. Tagliaferri, Eds., Vietri sul Mare, Salerno, Italy, 1999, Perspectives in Neural Computing, pp. 151-156, Springer Verlag, Berlin.
5. A. J. Bell and T. J. Sejnowski, "An information-maximization approach to blind separation and blind deconvolution", Neural Computation, 7(6):1129-1159, 1995.
6. S.-i. Amari, A. Cichocki, and H. H. Yang, "A new learning algorithm for blind signal separation ", in Advances in Neural Information Processing Systems, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds. 1996, vol. 8, pp. 757-763, The MIT Press, Cambridge, MA.
7. M. Girolami and C. Fyfe, "Generalised independent component analysis through unsupervised learning with emergent bussgang properties ", in Proc. IEEE Int. Conf. on Neural Networks, Houston, Texas, 1997, pp. 1788-1791.
8. J. Kivinen and M. K. Warmuth, "Exponentiated gradient versus gradient descent for linear predictors ", Tech. Rep. UCSC-CRL-94-16, University of California, Santa Cruz, June 1994.
9. J. Kivinen and M. K. Warmuth, "Additive versus exponentiated gradient updates for linear prediction ", in Proc. 27th Annual ACM Symposium on Theory of Computing, New York, NY, May 1995, pp. 209-218, The Association for Computing Machinery.
10. N. N. Schraudolph, "A fast, compact approximation of the exponential function ", Neural Computation, ll(4):853-862, 1999.
11. R. Jacobs, "Increased rates of convergence through learning rate adaptation ", Neural Networks, 1:295-307, 1988.
12. T. Tollenaere, "SuperSAB: fast adaptive back propagation with good scaling properties ", Neural Networks, 3:561-573, 1990.
13. F. M. Silva and L. B. Almeida, "Speeding up back-propagation ", in Advanced Neural Computers, R. Eckmiller, Ed., Amsterdam, 1990, pp. 151-158, Elsevier.
14. M. Riedmiller and H. Braun, "A direct adaptive method for faster backprop-agation learning: The RPROP algorithm ", in Proc. International Conference on Neural Networks, San Francisco, CA, 1993, pp. 586-591, IEEE, New York.
15. L. B. Almeida, T. Langlois,\D. Amarai, and A. Plakhov, "Parameter adaptation in stochastic optimization ", in On-Line Learning in Neural Networks, D. Saad, Ed., Publications of the Newton Institute, chapter 6. Cambridge University Press, 1999, ftp://146.193.2.131/pub/lba/papers/adsteps.ps. gz
16. M. E. Harmon and L. C. Baird III, "Multi-player residual advantage learning with general function approximation ", Tech. Rep. WL-TR-1065, Wright Laboratory, WL/AACF, 2241 Avionics Circle, Wright-Patterson Air Force Base, OH 45433-7308, 1996, http://www.leemon.com/papers/sim-tech/sim-tech.ps.gz.
17. R. S. Sutton, "Adapting bias by gradient descent: an incremental version of delta-bar-delta ", in Proc. 10th National Conference on Artificial Intelligence. 1992, pp. 171-176, The MIT Press, Cambridge, MA, ftp://ftp.cs.umass.edu/pub/anw/pub/sutton/sutton-92a.ps.gz
18. R. S. Sutton, "Gain adaptation beats least squares? ", in Proc. 7th Yale Workshop on Adaptive and Learning Systems, 1992, pp. 161-166, ftp://ftp.es.umass.edu/pub/anw/pub/sutton/sutton-92b.ps.gz.
19. R. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks ", Neural Computation, 1:270-280, 1989.
20. B. A. Pearlmutter, "Fast exact multiplication by the Hessian ", Neural Computation, 6(1):147-160, 1994.
21. J. Karhunen, E. Oja, L. Wang, R. Vigario, and J. Joutsensalo, "A class of neural networks for independent component analysis ", IEEE Trans, on Neural Networks, 8(3):486-504, 1997.