Stability analysis of multiplicative update algorithms and application to nonnegative matrix factorization

IEEE Transactions on Neural Networks

Volumn 21, Issue 12, 2010, Pages 1869-1881

  Badeau, Roland ^a   Bertin, Nancy ^b   Vincent, Emmanuel ^b  

^a INSTITUT TELECOM   (France)

^b INRIA   (France)

Author keywords

Convergence of numerical methods; Lyapunov methods; multiplicative update algorithms; nonnegative matrix factorization; optimization methods; stability

Indexed keywords

CONVERGENCE PROPERTIES; CONVERGENCE SPEED; LYAPUNOV; LYAPUNOV'S STABILITY; MULTIPLICATIVE UPDATES; NONNEGATIVE MATRIX FACTORIZATION; NONNEGATIVITY CONSTRAINTS; NUMERICAL SIMULATION; OPTIMIZATION METHODS; OPTIMIZATION PROBLEMS; STABILITY ANALYSIS; THEORETICAL RESULT;

ALGORITHMS; ASYMPTOTIC STABILITY; FACTORIZATION; LYAPUNOV METHODS; MATRIX ALGEBRA; OPTIMIZATION; PROBLEM SOLVING;

CONVERGENCE OF NUMERICAL METHODS;

EID: 78650045636     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2076831     Document Type: Article

References (37)

1. D. D. Lee and H. S. Seung, "Learning the parts of objects by nonnegative matrix factorization," Nature, vol. 401, no. 6755, pp. 788-791, Oct. 1999.
2. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in Proc. Conf. Adv. Neural Inform. Process. Syst., vol. 13. Vancouver, BC, Canada, Dec. 2001, pp. 556-562.
3. F. Itakura and S. Saito, "Analysis synthesis telephony based on the maximum likelihood method," in Proc. 6th Int. Congr. Acoust., Tokyo, Japan, Aug. 1968, pp. C17-C20.
4. A. Cichocki, R. Zdunek, and S. Amari, "Csisźar's divergences for nonnegative matrix factorization: Family of new algorithms," in Proc. 6th Int. Conf. Indepen. Compon. Anal. Blind Signal Separation, Charleston, SC, Mar. 2006, pp. 32-39. (Pubitemid 43998396)
5. S. Eguchi and Y. Kano, "Robustifying maximum likelihood estimation," Tokyo Inst. Statist. Math., Tokyo, Japan, Tech. Rep., 2001.
6. A. Cont, "Realtime multiple pitch observation using sparse non-negative constraints," in Proc. Int. Symp. Music Inform. Retrieval, Victoria, BC, Canada, Oct. 2006, pp. 206-212.
7. D. Guillamet, J. Vitrià, and B. Schiele, "Introducing a weighted nonnegative matrix factorization for image classification," Pattern Recognit. Lett., vol. 24, no. 14, pp. 2447-2454, Oct. 2003.
8. G. Peyré, "Non-negative sparse modeling of textures," in Proc. Scale Space Variational Methods Comput. Vis., Ischia Naples, Italy, May 2007, pp. 628-639.
9. E. Vincent, "Musical source separation using time-frequency source priors," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 1, pp. 91-98, Jan. 2006.
10. R. Kompass, "A generalized divergence measure for nonnegative matrix factorization," Neural Comput., vol. 19, no. 3, pp. 780-791, Mar. 2007.
11. C.-J. Lin, "Projected gradient methods for nonnegative matrix factorization," Neural Comput., vol. 19, no. 10, pp. 2756-2779, Oct. 2007.
12. L. Finesso and P. Spreij, "Approximate nonnegative matrix factorization via alternating minimization," in Proc. 16th Int. Symp. Math. Theory Netw. Syst., Leuven, Belgium, pp. 1-10, Jul. 2004.
13. A. Cichocki, R. Zdunek, and S. Amari, "Nonnegative matrix and tensor factorization," IEEE Signal Process. Mag., vol. 25, no. 1, pp. 142-145, Jan. 2008.
14. A. Cichocki, S. Amari, R. Zdunek, R. Kompass, G. Hori, and Z. He, "Extended SMART algorithms for non-negative matrix factorization," in Artificial Intelligence and Soft Computing (Lecture Notes in Computer Science), vol. 4029. Berlin, Germany: Springer-Verlag, 2006, pp. 548-562. (Pubitemid 44089843)
15. C. Févotte, N. Bertin, and J.-L. Durrieu, "Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis," Neural Comput., vol. 21, no. 3, pp. 793-830, Mar. 2009.
16. M. W. Berry, M. Browne, A. N. Langville, V. P. Pauca, and R. J. Plemmons, "Algorithms and applications for approximate nonnegative matrix factorization," Comput. Statist. & Data Anal., vol. 52, no. 1, pp. 155-173, Sep. 2007.
17. M. Chu, F. Diele, R. Plemmons, and S. Ragni, "Optimality, computation, and interpretation of nonnegative matrix factorizations," Dept. Comput. Sci. & Math., Wake Forest Univ., Winston-Salem, NC, Tech. Rep., Oct. 2004.
18. E. Gonzalez and Y. Zhang, "Accelerating the Lee-Seung algorithm for nonnegative matrix factorization," Dept. Comput. & Appl. Math., Rice Univ., Houston, TX, Tech. Rep. TR-05-02, Mar. 2005.
19. S. Choi, "Algorithms for orthogonal nonnegative matrix factorization," in Proc. IEEE Int. Joint Conf. Neural Netw., Orlando, FL, Jun. 2008, pp. 1828-1832.
20. S. Li, X. Hou, H. Zhang, and Q. Cheng, "Learning spatially localized, parts-based representation," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Kauai, HI, Dec. 2001, pp. 207-212.
21. J. Eggert, H. Wersing, and E. Korner, "Transformation-invariant representation and NMF," in Proc. IEEE Int. Joint Conf. Neural Netw., Budapest, Hungary, Jul. 2004, pp. 2535-2539.
22. N. Bertin, R. Badeau, and E. Vincent, "Fast Bayesian NMF algorithms enforcing harmonicity and temporal continuity in polyphonic music transcription," in Proc. IEEE Workshop Appl. Signal Process. Audio Acoust., New Paltz, NY, Oct. 2009, pp. 29-32.
23. N. Bertin, R. Badeau, and E. Vincent, "Enforcing harmonicity and smoothness in Bayesian non-negative matrix factorization applied to polyphonic music transcription," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 3, pp. 538-549, Mar. 2010.
24. E. Vincent, N. Bertin, and R. Badeau, "Adaptive harmonic spectral decomposition for multiple pitch estimation," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 3, pp. 528-537, Mar. 2010.
25. T. Virtanen, "Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 3, pp. 1066-1074, Mar. 2007.
26. Y. Zhang and Y. Fang, "A NMF algorithm for blind separation of uncorrelated signals," in Proc. IEEE Int. Conf. Wavelet Anal. Pattern Recognit., Beijing, China, Nov. 2007, pp. 999-1003.
27. P. Smaragdis, "Non-negative matrix factor deconvolution; extraction of multiple sound sources from monophonic inputs," in Proc. Int. Conf. Indepen. Compon. Anal. Blind Signal Separation, Granada, Spain, Sep. 2004, pp. 494-499.
28. C.-J. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1589-1596, Nov. 2007.
29. N. Bertin and R. Badeau, "Initialization, distances and local minima in audio applications of the non-negative matrix factorization," in Proc. Acoust., Paris, France, Jul. 2008, no. 2518.
30. N. Bertin, C. Févotte, and R. Badeau, "A tempering approach for Itakura-Saito non-negative matrix factorization. With application to music transcription," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Taipei, Taiwan, Apr. 2009, pp. 1545-1548.
31. J. Lasalle, The Stability and Control of Discrete Processes. New York: Springer-Verlag, 1986.
32. R. Badeau, N. Bertin, and E. Vincent, "Supporting document for the paper Stability analysis of multiplicative update algorithms and application to non-negative matrix factorization," Dept. Signal Image Process., Télécom ParisTech, Paris, France, Tech. Rep. 2010D019, Sep. 2010.
33. J. Nocedal and S. J. Wright, Numerical Optimization (Springer Series in Operations Research and Financial Engineering), 2nd ed. New York: Springer-Verlag, 2006.
34. Z. Wu, H. Su, J. Chu, and W. Zhou, "Improved delay-dependent stability condition of discrete recurrent neural networks with time-varying delays," IEEE Trans. Neural Netw., vol. 21, no. 4, pp. 692-697, Apr. 2010.
35. C.-D. Zheng, H. Zhang, and Z. Wang, "An augmented LKF approach involving derivative information of both state and delay," IEEE Trans. Neural Netw., vol. 21, no. 7, pp. 1100-1109, Jul. 2010.
36. X. Kong, C. Hu, and C. Han, "On the discrete-time dynamics of a class of self-stabilizing MCA extraction algorithms," IEEE Trans. Neural Netw., vol. 21, no. 1, pp. 175-181, Jan. 2010.
37. H. Laurberg, "Uniqueness of non-negative matrix factorization," in Proc. IEEE Workshop Statist. Signal Process., Madison, WI, Aug. 2007, pp. 44-48.