Balancing plasticity and stability of on-line learning based on hierarchical Bayesian adaptation of forgetting factors

Neurocomputing

Volumn 69, Issue 16-18, 2006, Pages 1954-1961

  Hirayama, Junichiro ^a   Yoshimoto, Junichiro ^a,b   Ishii, Shin ^a  

^a NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

^b OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

Author keywords

Forgetting factor; Hierarchical Bayes; On line learning; Variational Bayes

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HIERARCHICAL SYSTEMS; MATHEMATICAL MODELS; PROBABILITY; VARIATIONAL TECHNIQUES;

FORGETTING FACTOR; HIERARCHICAL BAYES; ON-LINE LEARNING; VARIATIONAL BAYES;

LEARNING ALGORITHMS;

ARTICLE; BAYES THEOREM; COMPUTER SIMULATION; LEARNING; MATHEMATICAL COMPUTING; MOTIVATION; NERVE CELL PLASTICITY; ONLINE SYSTEM; PRIORITY JOURNAL; PROBABILITY;

EID: 33748416315     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2005.11.020     Document Type: Article

References (24)

1. Amari S. Theory of adaptive pattern classifiers. IEEE Trans. Electron. Comput. 16 3 (1967) 299-307
2. Attias H. Independent factor analysis. Neural Comput. 11 4 (1999) 803-851
3. H. Attias, Inferring parameters and structure of latent variable models by variational bayes, in: Proceedings of 15th Conference on Uncertainty in AI, 1999, pp. 21-30.
4. Blake D.T., Byl N.N., and Merzenich M.M. Representation of the hand in the cerebral cortex. Behav. Brain. Res. 135 (2002) 179-184
5. Calford M.B. Dynamic representational plasticity in sensory cortex. Neuroscience 111 4 (2002) 709-738
6. C. Darken, J.E. Moody, Note on learning rate schedules for stochastic optimization, in: Advances in Neural Information Processing Systems, vol. 3, 1991, pp. 832-838.
7. Doya K. Metalearning and neuromodulation. Neural Network 15 4-6 (2002) 495-506
8. Edeline J.-M. Learning-induced physiological plasticity in the thalamo-cortical sensory systems: a critical evaluation of receptive field plasticity, map changes and their potential mechanisms. Progr. Neurobiol. 57 (1999) 165-224
9. Gilbert C.D., Sigman M., and Crist R.E. The neural basis of perceptual learning. Neuron 31 (2001) 681-697
10. Grossberg S. Adaptive pattern classification and universal recoding I: Parallel development and coding of neural feature detectors. Biol. Cybern. 23 (1976) 121-134
11. Hasselmo M.E. Neuromodulation and cortical function: modeling the physiological basis of behavior. Behav. Brain Res. 67 (1995) 1-27
12. Hirayama J., Yoshimoto J., and Ishii S. Bayesian representation learning in the cortex regulated by acetylcholine. Neural Networks 17 (2004) 1391-1400
13. Hyvärinen A. Independent component analysis in the presence of Gaussian noise by maximizing joint likelihood. Neurocomputing 22 (1998) 49-67
14. Lewicki M.S., and Sejnowski T.J. Learning overcomplete representations. Neural Comput. 12 (2000) 337-365
15. T. Minka, Bayesian linear regression, Technical Report, MIT, Cambridge, 2000.
16. Murata N., Kawanabe M., Ziehe A., Müller K.-R., and i Amari S. On-line learning in changing environments with applications in supervised and unsupervised learning. Neural Networks 15 4 (2002) 743-760
17. Murata N., Müller K.-R., Ziehe A., and i Amari S. Adaptive on-line learning in changing environments. Advances in Neural Information Processing Systems vol. 9 (1997), MIT Press, Cambridge, MA 599-605
18. S. Oba, M. Sato, and S. Ishii, Variational Bayes method for mixture of principal component analyzers, in: Proceeding for 7th International Conference on Neural Information Processing, vol. 2, 2000, pp. 1416-1421.
19. Olshausen B., and Field D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381 (1996) 607-609
20. Olshausen B.A., and Field D.J. Sparse coding with an overcomplete basis set: A strategy employed by V1?. Vision Res. 37 (1997) 3311-3325
21. Sato M. Online model selection based on the varational Bayes. Neural Comput. 13 (2001) 1649-1681
22. Schraudolph N.N. Fast curvature matrix-vector products for second-order gradient descent. Neural Comput. 14 (2002) 1723-1738
23. Sompolinsky H., Barkai N., and Seung H.S. On-line learning of dichotomies: algorithms and learning curves. Neural Networks: The Statistical Mechanics Perspective (1995), World Scientific, Singapore 105-130
24. M. Tipping, C. Bishop, Probabilistic principal component analysis, Technical report, Neural Computing Research Group, Aston University, 1997.