Relative gradient speeding up additive updates for nonnegative matrix factorization

Neurocomputing

Volumn 57, Issue 1-4, 2004, Pages 493-499

  Liu, Weixiang ^a   Zheng, Nanning ^a   Li, Xi ^a  

^a XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

Additive updates; Multiplicative updates; Nonnegative matrix factorization; Relative gradient

Indexed keywords

BLIND SOURCE SEPARATION; CONVERGENCE OF NUMERICAL METHODS; ERROR ANALYSIS; INDEPENDENT COMPONENT ANALYSIS; OPTIMIZATION;

FACTORIZATION; NONNEGATIVE MATRIX;

NEURAL NETWORKS;

ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; CONTROLLED STUDY; DATA ANALYSIS; INFORMATION PROCESSING; INTERMETHOD COMPARISON; LEARNING; MATHEMATICAL COMPUTING; PRIORITY JOURNAL; PROCESS OPTIMIZATION;

EID: 1642488120     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2004.01.002     Document Type: Article

References (20)

1. Amari S. Natural gradient works efficiently in learning Neural Comput. 10 1998 251-276
2. Amari S.-I. Hyvarinen A. Lee S.-Y. et-al. Blind signal separation and independent component analysis Neurocomputing 49 2002 1-5
3. S. Amari, H. Nagaoka, Methods of information geometry, Translations of Mathematical Monographs, Vol. 191, American Mathematical Society and Oxford University Press, 2000.
4. J.-F. Cardoso, Blind signal separation: statistical principles, special issue on blind identification and estimation, in: R.-W. Liu, L. Tong (Eds.), Proc. IEEE 90 (1998) 2009-2026.
5. Cardoso J.-F. Laheld B. Equivariant adaptive source separation IEEE Trans. Signal Process. 45 1996 433-444
6. CBCL Face Database #1, MIT Center For Biological and Computation Learning, available online, see http://www.ai.mit.edu/projects/cbcl.
7. Comon P. Independent component analysis: a new concept? Signal Process. 36 1994 287-314
8. Douglas S.C. Amari S. Natural gradient adaptation, in: Haykin S. (Ed.), Unsupervised Adaptive Filtering Blind Source Separation Vol. 1 2000 13-61 Wiley New York
9. P.O. Hoyer, Non-negative sparse coding, in: Neural Networks for Signal Processing XII, Proceedings of IEEE Workshop on Neural Networks for Signal Processing 2002, Martigny, Switzerland, pp. 557-565.
10. Hoyer P.O. Modeling receptive fields with non-negative sparse coding Neurocomputing 52-54 2003 547-552
11. Hoyer P.O. Hyvärinen A. A multi-layer sparse coding network learns contour coding from natural images Vision Res. 42 2002 1593-1605
12. Lee D.D. Seung H.S. Learning the parts of objects with nonnegative matrix factorization Nature 401 1999 788-791
13. D.D. Lee, H.S. Seung, Algorithms for nonnegative matrix factorization, in: T. Leen, T. Dietterich, V. Tresp (Eds.), Advances in Neural Information Processing Systems, Vol. 13, MIT Press, Cambridge, MA, November 2001, pp. 556-562.
14. Olshausen B.A. Field D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 381 1996 607-609
15. Paatero P. Tapper U. Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values Environmetrics 5 1994 111-126
16. Plumbley M.D. Algorithms for non-negative independent component analysis IEEE Trans. Neural Networks 14 2003 534-543
17. L.K. Saul, D.D. Lee, Multiplicative updates for classification by mixture models, in: T.G. Dietterich, S. Becker, Z. Ghahramani (Eds.), Advances in Neural Information Processing Systems, Vol. 14, MIT Press, Cambridge, MA, December 2002, pp. 897-904.
18. Weixiang Liu, Nanning Zheng, Review on nonnegative matrix factorization, Technical Report, Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, 2003.
19. Liu Weixiang Zheng Nanning Learning sparse features for classification by mixture models Pattern Recognition Lett. 25 2004 155-161
20. H.H. Yang, Multiplicative updating rule for blind separation derived from the method of scoring, in: M.J. Jordan, M. Kearns, S.A. Solla (Eds.), Advances in Neural Information Processing Systems, Vol. 10, MIT Press, Cambridge, MA, December 1998, pp. 696-702.