Putting nonnegative matrix factorization to the test: A tutorial derivation of pertinent Cramer? Rao bounds and performance benchmarking

IEEE Signal Processing Magazine

Volumn 31, Issue 3, 2014, Pages 76-86

  Huang, Kejun ^a   Sidiropoulos, Nicholas D ^b  

^a UNIVERSITY OF MINNESOTA   (United States)

^b University of Minnesota   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; FACTORIZATION; FISHER INFORMATION MATRIX; LEARNING SYSTEMS;

ANALYSIS PROBLEMS; APPROXIMATE SOLUTION; COMPUTATIONAL PROBLEM; ITS APPLICATIONS; NONNEGATIVE MATRIX FACTORIZATION; PERFORMANCE BENCHMARKING; SIGNAL SEPARATION; SPARSE AND LOW RANKS;

MATRIX ALGEBRA;

EID: 85032751413     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2013.2296172     Document Type: Article

References (41)

1. P. Paatero and U. Tapper, "Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values," Environmetrics, vol. 5, no. 2, pp. 111-126, 1994.
2. D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization," Nature, vol. 401, no. 6755, pp. 788-791, 1999.
3. S. M. Kay, Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory. Englewood Cliffs, NJ: Prentice Hall, 1993.
4. J. D. Gorman and A. O. Hero, "Lower bounds for parametric estimation with constraints," IEEE Trans. Inform. Theory, vol. 36, no. 6, pp. 1285-1301, 1990.
5. P. Stoica and B. C. Ng, "On the Cramér-Rao bound under parametric constraints," IEEE Signal Processing Lett., vol. 5, no. 7, pp. 177-179, 1998.
6. P. Stoica and T. L. Marzetta, "Parameter estimation problems with singular information matrices," IEEE Trans. Signal Processing, vol. 49, no. 1, pp. 87-90, 2001. (Pubitemid 32137126)
7. Z. Ben-Haim and Y. C. Eldar, "On the constrained Cramér-Rao bound with a singular Fisher information matrix," IEEE Signal Processing Lett., vol. 16, no. 6, pp. 453-456, 2009.
8. P. Tichavský and Z. Koldovský, "Optimal pairing of signal components separated by blind techniques," IEEE Signal Processing Lett., vol. 11, no. 2, pp. 119-122, 2004.
9. H. W. Kuhn, "The Hungarian method for the assignment problem," Naval Res. Logist. Quart., vol. 2, no. 1-2, pp. 83-97, 1955.
10. R.E. Burkard, M. Dell'Amico, and S. Martello, Assignment Problems. Philadelphia, PA: SIAM, 2009.
11. D. L. Donoho and V. C. Stodden, "When does non-negative matrix factorization give a correct decomposition into parts?," in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2003, vol. 16, pp. 1141-1148.
12. H. Laurberg, M. G. Christensen, M. D. Plumbley, L. K. Hansen, and S. H. Jensen, "Theorems on positive data: On the uniqueness of NMF," Computat. Intell. Neurosci., vol. 2008, Article ID 764206, 9 pages, DOI: 10.1155/2008/764206.
13. N. Gillis, "Sparse and unique nonnegative matrix factorization through data preprocessing," J. Mach. Learn. Res., vol. 13, pp. 3349-3386, Nov. 2012.
14. K. Huang, N. D. Sidiropoulos, and A. Swami, "Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition," IEEE Trans. Signal Processing, vol. 62, no. 1, pp. 211-224, Jan. 2014.
15. S. A. Vavasis, "On the complexity of nonnegative matrix factorization," SIAM J. Optim., vol. 20, no. 3, pp. 1364-1377, 2009.
16. A. Cichocki, R. Zdunek, A. H. Phan, and S. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind Source Separation. Hoboken, NJ: Wiley, 2009.
17. Z. He, S. Xie, R. Zdunek, G. Zhou, and A. Cichocki, "Symmetric nonnegative matrix factorization: Algorithms and applications to probabilistic clustering," IEEE Trans. Neural Networks, vol. 22, no. 12, pp. 2117-2131, 2011.
18. A. Berman and N. Shaked-Monderer, Completely Positive Matrices. Singapore: World Scientific, 2003.
19. P. J. C. Dickinson and L. Gijben, "On the computational complexity of membership problems for the completely positive cone and its dual," Computat. Optim. Applicat., submitted for publication. DOI: 10.1007/s10589-013- 9594-z
20. Z. Yang and E. Oja, "Quadratic nonnegative matrix factorization," Pattern Recognit., vol. 45, no. 4, pp. 1500-1510, 2012.
21. C. Ding, X. He, and H. D. Simon, "On the equivalence of nonnegative matrix factorization and spectral clustering," in Proc. SIAM Int. Conf. Data Mining (SDM'05), 2005, vol. 5, pp. 606-610.
22. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," Adv. Neural Inform. Process. Syst. (NIPS), vol. 13, pp. 556-562, 2001.
23. A. Cichocki, S. Amari, R. Zdunek, R. Kompass, G. Hori, and Z. He, "Extended SMART algorithms for non-negative matrix factorization," in Artificial Intelligence and Soft Computing (ICAISC). New York: Springer, 2006, pp. 548-562. (Pubitemid 44089843)
24. C.-J. Lin, "Projected gradient methods for nonnegative matrix factorization," Neural Computat., vol. 19, no. 10, pp. 2756-2779, 2007.
25. R. Zdunek and A. Cichocki, "Nonnegative matrix factorization with constrained second-order optimization," Signal Process., vol. 87, no. 8, pp. 1904-1916, 2007. (Pubitemid 46590394)
26. M. W. Berry, M. Browne, A. N. Langville, V. P. Pauca, and R. J. Plemmons, "Algorithms and applications for approximate nonnegative matrix factorization," Computat. Stat. Data Anal., vol. 52, no. 1, pp. 155-173, 2007. (Pubitemid 47331703)
27. M. Heiler and C. Schnörr, "Learning sparse representations by non-negative matrix factorization and sequential cone programming," J. Mach. Learn. Res., vol. 7, pp. 1385-1407, July 2006. (Pubitemid 44024595)
28. H. Kim and H. Park, "Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method," SIAM J. Matrix Anal. Applicat., vol. 30, no. 2, pp. 713-730, 2008.
29. J. Kim and H. Park, "Fast nonnegative matrix factorization: An active-set-like method and comparisons," SIAM J. Sci. Comput., vol. 33, no. 6, pp. 3261-3281, 2011.
30. A. Cichocki and A.-H. Phan, "Fast local algorithms for large scale nonnegative matrix and tensor factorizations," IEICE Trans. Fundam. Electron., Commun. Comput. Sci., vol. 92, no. 3, pp. 708-721, 2009.
31. B. Klingenberg, J. Curry, and A. Dougherty, "Non-negative matrix factorization: Ill-posedness and a geometric algorithm," Pattern Recognit., vol. 42, no. 5, pp. 918-928, 2009.
32. R. Zdunek, "Initialization of nonnegative matrix factorization with vertices of convex polytope," in Artificial Intelligence and Soft Computing. New York: Springer, 2012, pp. 448-455.
33. J. Li and J. M. Bioucas-Dias, "Minimum volume simplex analysis: A fast algorithm to unmix hyperspectral data," in Proc. IEEE Int. Geoscience and Remote Sensing Symp. (IGARSS), 2008, vol. 3, pp. 250-253.
34. W. S. B. Ouedraogo, A. Souloumiac, M. Jaidane, and C. Jutten, "Simplicial cone shrinking algorithm for unmixing nonnegative sources," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP) 2012, pp. 2405-2408.
35. M. T. Chu and M. M. Lin, "Low-dimensional polytope approximation and its applications to nonnegative matrix factorization," SIAM J. Sci. Comput., vol. 30, no. 3, pp. 1131-1155, 2008.
36. S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge, UK: Cambridge Univ. Press, 2004.
37. A. Swami, "Cramér-Rao bounds for deterministic signals in additive and multiplicative noise," Signal Process., vol. 53, no. 2, pp. 231-244, 1996. (Pubitemid 126370908)
38. K. B. Petersen and M. S. Pedersen, The Matrix Cookbook. Kongens Lyngby, Denmark: Technical Univ. Denmark, 2006.
39. M. Chu, F. Diele, R. Plemmons, and S. Ragni. (2004). Optimality, computation, and interpretation of nonnegative matrix factorizations. [Online]. Available: http://www4.ncsu.edu/~mtchu/Research/Papers/nnmf.pdf.
40. C. Hung and T. L. Markham, "The Moore-Penrose inverse of a partitioned matrix M = BA DC j, " Linear Algebra Applica., vol. 11, no. 1, pp. 73-86, 1975.
41. C. Hung and T. L. Markham, "The Moore-Penrose inverse of a sum of matrices," J. Aust. Math. Soc., Ser. A, vol. 24, no. 4, pp. 385-392, 1977.