Nonnegative matrix factorization with constrained second-order optimization

Signal Processing

Volumn 87, Issue 8, 2007, Pages 1904-1916

  Zdunek, Rafal ^a,b   Cichocki, Andrzej ^a,c  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b WROCLAW UNIVERSITY OF TECHNOLOGY   (Poland)

^c WARSAW UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

Blind source separation; Fixed point algorithm; GPCG; Nonnegative matrix factorization; Quasi Newton method; Second order optimization

Indexed keywords

BLIND SOURCE SEPARATION; FACTORIZATION; IMAGE SEGMENTATION; OPTIMIZATION; PATTERN RECOGNITION;

FIXED POINT ALGORITHM; NONNEGATIVE MATRIX FACTORIZATION; QUASI-NEWTON METHOD_; SECOND ORDER OPTIMIZATION;

MATRIX ALGEBRA;

EID: 34247173538     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2007.01.024     Document Type: Article

References (45)

1. Guillamet D., Vitrià J., and Schiele B. Introducing a weighted nonnegative matrix factorization for image classification. Pattern Recognition Lett. 24 14 (2003) 2447-2454
2. Guillamet D., Schiele B., and Vitrià J. Analyzing non-negative matrix factorization for image classification. 16th International Conference on Pattern Recognition (ICPR'02) vol. 2 (2002), Quebec City, Canada 116-119
3. D. Guillamet, J. Vitrià, Classifying faces with nonnegative matrix factorization, in: Proc. of the Fifth Catalan Conference for Artificial Intelligence, Castello de la Plana, Spain, 2002.
4. J.-H. Ahn, S. Kim, J.-H. Oh, S. Choi, Multiple nonnegative-matrix factorization of dynamic PET images, in: ACCV, 2004.
5. Lee J.S., Lee D.D., Choi S., and Lee D.S. Application of nonnegative matrix factorization to dynamic positron emission tomography. Third International Conference on Independent Component Analysis and Blind Signal Separation (2001), San Diego, CA 556-562
6. Lee D.D., and Seung H.S. Learning the parts of objects by nonnegative matrix factorization. Nature 401 (1999) 788-791
7. Li H., Adali T., and Wang D.E.W. Non-negative matrix factorization with orthogonality constraints for chemical agent detection in raman spectra. IEEE Workshop on Machine Learning for Signal Processing (2005), Mystic, USA
8. P. Carmona-Saez, R.D. Pascual-Marqui, F. Tirado, J.M. Carazo, A. Pascual-Montano, Biclustering of gene expression data by non-smooth non-negative matrix factorization, BMC Bioinformatics 7(78) (2006).
9. Pascual-Montano A., Carazo J.M., Kochi K., Lehmean D., and Pascual-Marqui R. Nonsmooth nonnegative matrix factorization (nsNMF). IEEE Trans. Pattern Anal. Mach. Intell. 28 3 (2006) 403-415
10. Shahnaz F., Berry M., Pauca P., and Plemmons R. Document clustering using non-negative matrix factorization. J. Inf. Process. Manage. 42 (2006) 373-386
11. O. Okun, H. Priisalu, Fast nonnegative matrix factorization and its application for protein fold recognition, EURASIP J. Appl. Signal Process. (2006) 8 c(Article ID 71817).
12. Wang Y., Jia Y., Hu C., and Turk M. Non-negative matrix factorization framework for face recognition. Int. J. Pattern Recognition Artif. Intell. 19 4 (2005) 495-511
13. Liu W., and Zheng N. Non-negative matrix factorization based methods for object recognition. Pattern Recognition Lett. 25 8 (2004) 893-897
14. Spratling M.W. Learning image components for object recognition. J. Mach. Learn. Res. 7 (2006) 793-815
15. Sajda P., Du S., Brown T.R., Shungu R.S.D.C., Mao X., and Parra L.C. Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain. IEEE Trans. Medical Imaging 23 12 (2004) 1453-1465
16. Sajda P., Du S., Brown T., Parra L., and Stoyanova R. Recovery of constituent spectra in 3d chemical shift imaging using nonnegative matrix factorization. Fourth International Symposium on Independent Component Analysis and Blind Signal Separation (2003), Nara, Japan 71-76
17. P. Sajda, S. Du, L. Parra, Recovery of constituent spectra using non-negative matrix factorization, in: Proceedings of SPIE-Wavelets: Applications in Signal and Image Processing, vol. 5207, 2003, pp. 321-331.
18. Dhillon I.S., and Modha D.M. Concept decompositions for large sparse text data using clustering. Mach. Learn. J. 42 (2001) 143-175
19. Cho Y.C., and Choi S. Nonnegative features of spectro-temporal sounds for classification. Pattern Recognition Lett. 26 (2005) 1327-1336
20. J.-P. Brunet, P. Tamayo, T.R. Golub, J.P. Mesirov, Metagenes and molecular pattern discovery using matrix factorization, PNAS, vol. 101, 2000, pp. 4164-4169.
21. N. Rao, S.J. Shepherd, D. Yao, Extracting characteristic patterns from genome-wide expression data by non-negative matrix factorization, in: Proceedings of the 2004 IEEE Computational Systems Bioinformatics Conference (CSB 2004), Stanford, CA, 2004.
22. A. Cichocki, R. Zdunek, S. Amari, Csiszar's divergences for non-negative matrix factorization: family of new algorithms, Lecture Notes in Computer Science, vol. 3889, 2006, pp. 32-39.
23. Richardson R. Bayesian-based iterative method of image restoration. J. Opt. Soc. Am. 62 1 (1972) 55-59
24. Lucy L. An iterative technique for the rectification of observed distributions. Astron. J. 79 6 (1974) 745-754
25. Lange K., and Carson R. EM reconstruction algorithms for emission and transmission tomography. J. Comput. Assisted Tomogr. 8 2 (1984) 306-316
26. Byrne C. Accelerating the EMML algorithm and related iterative algorithms by rescaled block-iterative (RBI) methods. IEEE Trans. Image Process. 7 (1998) 100-109
27. Pierro A.R.D. On the relation between the ISRA and the EM algorithm for positron emission tomography. IEEE Trans. Med. Imaging 12 2 (1993) 328-333
28. Hoyer P. Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5 (2004) 1457-1469
29. C.-J. Lin, Projected gradient methods for non-negative matrix factorization, Neural Comput., in press, URL 〈http://www.csie.ntu.edu.tw/∼cjlin〉.
30. M.T. Chu, F. Diele, R. Plemmons, S. Ragni, Optimality, computation, and interpretation of nonnegative matrix factorizations, unpublished report, October 18, 2004, URL 〈http://www.citeseer.ist.psu.edu/758183.html〉.
31. M. Berry, M. Browne, A. Langville, P. Pauca, R. Plemmons, Algorithms and applications for approximate nonnegative matrix factorization, Comput. Stat. Data Anal., in press, doi:10.1016/j.csda.2006.11.006.
32. Cichocki A., and Zdunek R. Multilayer nonnegative matrix factorization. Electron. Lett. 42 16 (2006) 947-948
33. A. Cichocki, R. Zdunek, Regularized alternating least squares algorithms for non-negative matrix/tensor factorization, in: Proceedings of the Fourth International Symposium on Neural Networks (ISNN), June 3-7, 2007, Nanjing, China.
34. A. Cichocki, R. Zdunek, Multilayer nonnegative matrix factorization using projected gradient approaches, 13th International Conference on Neural Information Processing, Hong Kong, 2006.
35. Merritt M., and Zhang Y. An interior-point gradient method for large-scale totally nonnegative least squares problems. J. Optim. Theory Appl. 126 1 (2005) 191-202
36. R. Zdunek, A. Cichocki, Non-negative matrix factorization with quasi-Newton optimization, Lecture Notes in Artificial Intelligence, vol. 4029, 2006, pp. 870-879.
37. More J.J., and Toraldo G. On the solution of large quadratic programming problems with bound constraints. SIAM J. Optim. 1 1 (1991) 93-113
38. Bardsley J.M., and Vogel C.R. Nonnegatively constrained convex programming method for image reconstruction. SIAM J. Sci. Comput. 4 (2004) 1326-1343
39. Bardsley J.M. A nonnegatively constrained trust region algorithm for the restoration of images with an unknown blur. Electron. Trans. Numer. Anal. 20 (2005) 139-153
40. Hestenes M.R., and Stiefel E. Method of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49 (1952) 409-436
41. Amari S. Differential-Geometrical Methods in Statistics (1985), Springer, Berlin
42. Cressie N.A., and Read T. Goodness-of-Fit Statistics for Discrete Multivariate Data (1988), Springer, New York
43. A. Cichocki, S. Amari, R. Zdunek, R. Kompass, G. Hori, Z. He, Extended SMART algorithms for non-negative matrix factorization, Lecture Notes in Artificial Intelligence, vol. 4029, 2006, pp. 548-562.
44. Cotter S.F., Rao B.D., Engan K., and Kreutz-Delgado K. Sparse solutions to linear inverse problems with multiple measurement vectors. IEEE Trans. Signal Process. 53 7 (2005) 2477-2488
45. A. Cichocki, R. Zdunek, NMFLAB for Signal and Image Processing, Technical Report, Laboratory for Advanced Brain Signal Processing, BSI RIKEN, Saitama, Japan, 2006. URL 〈http://www.bsp.brain.riken.jp〉.