Online nonnegative matrix factorization with robust stochastic approximation

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 7, 2012, Pages 1087-1099

  Guan, Naiyang ^a   Tao, Dacheng ^b   Luo, Zhigang ^a   Yuan, Bo ^c  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

^b UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

^c SHANGHAI JIAO TONG UNIVERSITY   (China)

Author keywords

Nonnegative matrix factorization (NMF); online NMF (ONMF); robust stochastic approximation

Indexed keywords

AVERAGING TECHNIQUE; DIMENSION-REDUCTION; LOCAL OPTIMAL SOLUTION; NONNEGATIVE MATRIX FACTORIZATION; ONLINE NMF (ONMF); PATTERN RECOGNITION PROBLEMS; STOCHASTIC APPROXIMATIONS; TIME AND SPACE COMPLEXITY;

ALGORITHMS; FACE RECOGNITION;

APPROXIMATION THEORY;

EID: 84865442402     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2197827     Document Type: Article

References (48)

1. D. D. Lee and H. S. Seung, "Learning the parts of objects by nonnegative matrix factorization," Nature, vol. 401, no. 21, pp. 788-791, 1999.
2. S. Zafeiriou, A. Tefas, I. Buciu, and I. Pitas, "Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 683-695, May 2006.
3. Z. Yang and E. Oja, "Linear and nonlinear projective nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 21, no. 5, pp. 734-749, May 2010.
4. J. Wang, S. Yan, Y. Fu, X. Li, and T. S. Huang, "Non-negative graph embedding," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 4. Anchorage, AK, Jun. 2008, pp. 1-8.
5. D. Cai, X. He, J. Han, and T. S. Huang, "Graph regularized nonnegative matrix factorization for data representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1548-1560, Aug. 2011.
6. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Manifold regularized discriminative non-negative matrix factorization with fast gradient descent," IEEE Trans. Imag. Process., vol. 20, no. 7, pp. 2030-2048, Jul. 2011.
7. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Non-negative patch alignment framework," IEEE Trans. Neural Netw., vol. 22, no. 8, pp. 1218-1230, Aug. 2011.
8. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in Advances in Neural Information Processing Systems, vol. 12. Cambridge, MA: MIT Press, 2000, pp. 556-562.
9. C. J. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1589-1596, Nov. 2007.
10. C. J. Lin, "Projected gradient methods for nonnegative matrix factorization," Neural Comput., vol. 19, no. 10, pp. 2756-2779, Oct. 2007.
11. M. W. Berry, M. Browne, A. Langville, P. Pauca, and R. J. Plemmons, "Algorithms and applications for approximate non-negative matrix factorization," Comput. Stat. Data Anal., vol. 52, no. 1, pp. 155-173, 2007.
12. J. Kim and H. Park, "Toward faster nonnegative matrix factorization: A new algorithm and comparisons," in Proc. IEEE Int. Conf. Data Min., Dec. 2008, pp. 353-362.
13. N. Guan, D. Tao, Z. Luo, and B. Yuan, "NeNMF: An optimal gradient method for non-negative matrix factorization," IEEE Trans. Signal Process., 2012, DOI: 10.1109/TSP.2012.2190406, to be published.
14. V. P. Pauca, F. Shahnaz, M. W. Berry, and R. J. Plemmons, "Text mining using non-negative matrix factorizations," in Proc. 4th SIAM Int. Conf. Data Min., 2004, pp. 452-456.
15. X. Wang, Z. Li, and D. Tao, "Subspaces indexing model on grassmann manifold for image search," IEEE Trans. Image Process., vol. 20, no. 9, pp. 2627-2635, Sep. 2011.
16. B. Xie, Y. Mu, D. Tao, and K. Huang, "m-SNE: Multiview stochastic neighbor embedding," IEEE Trans. Syst. Man Cybern. B, vol. 41, no. 4, pp. 1088-1096, Aug. 2011.
17. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro, "Robust stochastic approximation approach to stochastic programming," SIAM J. Optim., vol. 19, no. 4, pp. 1574-1609, 2009.
18. L. Bottou, O. Bousquet, and G. Zürich, "The trade-offs of large scale learning," in Proc. Adv. Neural Inf. Process. Syst., vol. 20. 2008, pp. 161-168.
19. V. Strassen, "The existence of probability measures with given marginals," Ann. Math. Stat., vol. 36, no. 2, pp. 423-439, 1965.
20. A. Korattikara, L. Boyles, M. Welling, J. Kim, and H. Park, "Statistical optimization of non-negative matrix factorization," in Proc. 14th Int. Conf. Artif. Intell. Stat., vol. 15. 2011, pp. 1-9.
21. B. A. Lefèvre, F. Bach, and C. Fèvotte, "Online algorithms for nonnegative matrix factorization with the Itakura-Saito divergence," in Proc. IEEE Workshop Appl. Signal Process. Audio Acoust., New Paltz, NY, Oct. 2011, pp. 313-316.
22. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., vol. 11, pp. 19-60, Mar. 2010.
23. A. W. Van der Vaart, Asymptotic Statistics. Cambridge, U.K.: Cambridge Univ. Press, 1998.
24. D. L. Fisk, "Quasi-martingales," Trans. Amer. Math. Soc., vol. 120, no. 3, pp. 359-388, 1965.
25. D. Bertsekas, Nonlinear Programming, 2nd ed. Nashua, NH: Athena Scientific, 1999.
26. P. O. Hoyer, "Non-negative sparse coding," in Proc. 12th IEEE Workshop Neural Netw. Signal Process., Nov. 2002, pp. 557-565.
27. X. He, D. Cai, Y. Shao, H. Bao, and J. Han, "Laplacian regularized Gaussian mixture model for data clustering," IEEE Trans. Knowl. Data Eng., vol. 23, no. 9, pp. 1406-1418, Sep. 2011.
28. B. Geng, D. Tao, C. Xu, L. Yang, and X. Hua, "Ensemble manifold regularization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 6, pp. 1227-1233, Jun. 2012.
29. S. Si, D. Tao, and B. Geng, "Bregman divergence based regularization for transfer subspace learning," IEEE Trans. Knowl. Data Eng., vol. 22, no. 7, pp. 929-942, Jul. 2010.
30. M. Song, D. Tao, C. Chen, X. Li, and C. Chen, "Color to gray: Visual cue preservation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1537-1552, Sep. 2010.
31. X. Tian, D. Tao, and Y. Rui, "Sparse transfer learning for interactive video search reranking," ACM Trans. Multimedia Comput. Commun. Appl., 2012, arXiv:1103.2756v3, to be published.
32. X. He, "Laplacian regularized D-optimal design for active learning and its application to image retrieval," IEEE Trans. Image Process., vol. 19, no. 1, pp. 254-263, Jan. 2010.
33. C. Fèvotte, N. Bertin, and J. L. Durrieu, "Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis," Neural Comput., vol. 21, no. 3, pp. 793-830, Mar. 2009.
34. B. Cao, D. Shen, S. J. Tao, X. Wang, Q. Yang, and Z. Chen, "Detect and track latent factors with online nonnegative matrix factorization," in Proc. 20th Int. Joint Conf. Artif. Intell., San Francisco, CA, 2007, pp. 2689-2694.
35. S. S. Bucak and B. Gunsel, "Incremental subspace learning via nonnegative matrix factorization," Pattern Recognit., vol. 42, no. 5, pp. 788-797, 2009.
36. G. Zhou, Z. Yang, and S. Xie, "Online blind source separation using incremental nonnegative matrix factorization with volume constraint," IEEE Trans. Neural Netw., vol. 22, no. 4, pp. 550-560, Apr. 2011.
37. F. Wang, C. Tan, A. C. Knoig, and P. Li, "Efficient document clustering via online nonnegative matrix factorization," in Proc. SIAM Int. Conf. Data Min., 2011, pp. 1-12.
38. B. Weyrauch, J. Huang, B. Heisele, and V. Blanz, "Component-based face recognition with 3-D morphable models," in Proc. IEEE Workshop Face Process. Video, Jun. 2004, p. 85.
39. F. Samaria and A. Harter, "Parameterisation of a stochastic model for human face identification," in Proc. IEEE Workshop Appl. Comput. Vis., Sarasota, FL, Dec. 1994, pp. 138-142.
40. M. Grubinger, P. D. Clough, M. Henning, and D. Thomas, "The IAPR benchmark: A new evaluation resource for visual information systems," in Proc. Int. Conf. Lang. Resour. Evaluat., Genoa, Italy, 2006, pp. 1-10.
41. J. Duchi, S. Shalev-Schwartz, Y. Singer, and T. Chandra, "Efficient projections onto the l1-ball for learning in high dimensions," in Proc. 25th Int. Conf. Mach. Learn., Helsinki, Finland, Jul. 2008, pp. 1-8.
42. P. O. Hoyer, "Non-negative matrix factorization with sparseness constraints," J. Mach. Learn. Res., vol. 5, pp. 1457-1469, Dec. 2004.
43. A. Makadia, V. Pavlovic, and S. Kumar, "A new baseline for image annotation," in Proc. 10th Eur. Conf. Comput. Vis., vol. 3. Berlin, Germany, 2008, pp. 316-329.
44. M. Song, D. Tao, C. Chen, X. Li, and C. Chen, "Color to gray: Visual cue preservation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1537-1552, Sep. 2010.
45. M. Song, D. Tao, C. Chen, J. Luo, and C. Zhang, "Probabilistic exposure fusion," IEEE Trans. Image Process., vol. 21, no. 1, pp. 341-357, Jan. 2012.
46. M. Song, D. Tao, X. Huang, C. Chen, and J. Bu, "Three-dimensional face reconstruction from a single image by a coupled RBF network," IEEE Trans. Image Process., vol. 21, no. 5, pp. 2887-2897, May 2012.
47. K. Barnard, P. Duygulu, N. Freitas, D. Forsyth, D. Blei, and M. Jordan, "Matching words and pictures," J. Mach. Learn. Res., vol. 3, pp. 1107-1135, Mar. 2003.
48. P. Duygulu, N. Freitas, K. Barnard, and D. A. Forsyth, "Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary," in Proc. 7th Eur. Conf. Comput. Vis., vol. 4. 2002, pp. 97-112.