On the performance of Manhattan nonnegative matrix factorization

IEEE Transactions on Neural Networks and Learning Systems

Volumn 27, Issue 9, 2016, Pages 1851-1863

  Liu, Tongliang ^a   Tao, Dacheng ^a  

^a UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

Approximation error; Estimation Error; Manhattan Distance; Nonnegative Matrix Factorization (NMF); Statistical analysis

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPRESSED SENSING; ERROR ANALYSIS; ERRORS; FACTORIZATION; IMAGE RECONSTRUCTION; LEARNING SYSTEMS; SIGNAL PROCESSING; VECTOR QUANTIZATION;

APPROXIMATION ERRORS; BRAIN SIGNAL PROCESSING; GENERALIZATION ERROR BOUNDS; NONNEGATIVE MATRIX FACTORIZATION; RECONSTRUCTION ERROR; STATISTICAL LEARNING THEORY; STATISTICAL PERFORMANCE; STATISTICAL PERFORMANCE ANALYSIS;

MATRIX ALGEBRA;

EID: 84939174251     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2015.2458986     Document Type: Article

References (59)

1. C. Lynch, "Big data: How do your data grow?" Nature, vol. 455, no. 7209, pp. 28-29, 2008.
2. M. Piernik, D. Brzezinski, and T. Morzy, "Clustering XML documents by patterns, " Knowl. Inf. Syst., pp. 1-28, Jan. 2015, doi: 10.1007/s10115-015-0820-0.
3. D. Tao, L. Jin, Y. Wang, and X. Li, "Person reidentification by minimum classification error-based KISS metric learning, " IEEE Trans. Cybern., vol. 45, no. 2, pp. 242-252, Feb. 2015.
4. X. Wen, L. Shao, W. Fang, and Y. Xue, "Efficient feature selection and classification for vehicle detection, " IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 3, pp. 508-517, Mar. 2015.
5. Y. Luo, D. Tao, B. Geng, C. Xu, and S. J. Maybank, "Manifold regularized multitask learning for semi-supervised multilabel image classification, " IEEE Trans. Image Process., vol. 22, no. 2, pp. 523-536, Feb. 2013.
6. Z. Zhang and K. Zhao, "Low-rank matrix approximation with manifold regularization, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 7, pp. 1717-1729, Jul. 2013.
7. D. Tao, L. Jin, W. Liu, and X. Li, "Hessian regularized support vector machines for mobile image annotation on the cloud, " IEEE Trans. Multimedia, vol. 15, no. 4, pp. 833-844, Jun. 2013.
8. I. T. Jolliffe, Principal Component Analysis. New York, NY, USA: Wiley, 2005.
9. R. Yan, L. Shao, and Y. Liu, "Nonlocal hierarchical dictionary learning using wavelets for image denoising, " IEEE Trans. Image Process., vol. 22, no. 12, pp. 4689-4698, Dec. 2013.
10. D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization, " Nature, vol. 401, no. 6755, pp. 788-791, 1999.
11. V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky, "Rank-sparsity incoherence for matrix decomposition, " SIAM J. Optim., vol. 21, no. 2, pp. 572-596, 2011.
12. E. J. Candès, X. Li, Y. Ma, and J. Wright, "Robust principal component analysis?" J. ACM, vol. 58, no. 3, 2011, Art. ID 11.
13. T. Zhou and D. Tao, "GoDec: Randomized low-rank & sparse matrix decomposition in noisy case, " in Proc. 28th Int. Conf. Mach. Learn. (ICML), Bellevue, WA, USA, Jul. 2011, pp. 33-40.
14. N. Guan, D. Tao, Z. Luo, and J. Shawe-Taylor. (2012) "MahNMF: Manhattan non-negative matrix factorization." [Online]. Available: http://arxiv.org/abs/1207.3438
15. L. Liu, M. Yu, and L. Shao, "Multiview alignment hashing for efficient image search, " IEEE Trans. Image Process., vol. 24, no. 3, pp. 956-966, Mar. 2015.
16. C. Xu, D. Tao, and C. Xu, "Multi-view intact space learning, " IEEE Trans. Pattern Anal. Mach. Intell., doi: 10.1109/TPAMI.2015.2417578.
17. L. Zhang, X. Zhen, and L. Shao, "Learning object-To-class kernels for scene classification, " IEEE Trans. Image Process., vol. 23, no. 8, pp. 3241-3253, Aug. 2014.
18. A. Khan, M. A. Jaffar, and L. Shao, "A modified adaptive differential evolution algorithm for color image segmentation, " Knowl. Inf. Syst., vol. 43, no. 3, pp. 583-597, 2015.
19. M. N. Schmidt, J. Larsen, and F.-T. Hsiao, "Wind noise reduction using non-negative sparse coding, " in Proc. IEEE Int. Workshop Mach. Learn. Signal Process., Thessaloniki, Greece, Aug. 2007, pp. 431-436.
20. T. Zhang, B. Fang, Y. Y. Tang, G. He, and J. Wen, "Topology preserving non-negative matrix factorization for face recognition, " IEEE Trans. Image Process., vol. 17, no. 4, pp. 574-584, Apr. 2008.
21. A. Dessein, A. Cont, and G. Lemaitre, "Real-Time polyphonic music transcription with non-negative matrix factorization and betadivergence, " in Proc. 11th Int. Soc. Music Inf. Retr. Conf., Utrecht, The Netherlands, Aug. 2010, pp. 489-494.
22. V. P. Pauca, F. Shahnaz, M. W. Berry, and R. J. Plemmons, "Text mining using non-negative matrix factorizations, " in Proc. SIAM Int. Conf. Data Mining (SDM), Lake Buena Vista, FL, USA, Apr. 2004, pp. 452-456.
23. S. Arora, R. Ge, and A. Moitra, "Learning topic models-Going beyond SVD, " in Proc. 53rd IEEE Annu. Symp. Found. Comput. Sci. (FOCS), New Brunswick, NJ, USA, Oct. 2012, pp. 1-10.
24. J. M. Tomczak and A. Gonczarek, "Decision rules extraction from data stream in the presence of changing context for diabetes treatment, " Knowl. Inf. Syst., vol. 34, no. 3, pp. 521-546, 2013.
25. H. Kim and H. Park, "Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis, " Bioinformatics, vol. 23, no. 12, pp. 1495-1502, 2007.
26. K. Devarajan, "Nonnegative matrix factorization: An analytical and interpretive tool in computational biology, " PLoS Comput. Biol., vol. 4, no. 7, p. e1000029, 2008.
27. S. Zhang, Q. Li, J. Liu, and X. J. Zhou, "A novel computational framework for simultaneous integration of multiple types of genomic data to identify microRNA-gene regulatory modules, " Bioinformatics, vol. 27, no. 13, pp. i401-i409, 2011.
28. N. Guan, D. Tao, Z. Luo, and B. Yuan, "NeNMF: An optimal gradient method for nonnegative matrix factorization, " IEEE Trans. Signal Process., vol. 60, no. 6, pp. 2882-2898, Jun. 2012.
29. B. Recht, C. Re, J. Tropp, and V. Bittorf, "Factoring nonnegative matrices with linear programs, " in Proc. Adv. Neural Inf. Process. Syst. (NIPS), Lake Tahoe, NV, USA, Dec. 2012, pp. 1214-1222.
30. B. Frénay and M. Verleysen, "Classification in the presence of label noise: A survey, " IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 5, pp. 845-869, May 2014.
31. T. Liu and D. Tao. (2014). "Classification with noisy labels by importance reweighting." [Online]. Available: http://arxiv.org/abs/1411.7718
32. T. Yang, M. Mahdavi, R. Jin, L. Zhang, and Y. Zhou, "Multiple kernel learning from noisy labels by stochastic programming, " in Proc. 29th Int. Conf. Mach. Learn. (ICML), Edinburgh, U.K., Jun. 2012, pp. 233-240.
33. D. Donoho and V. Stodden, "When does non-negative matrix factorization give a correct decomposition into parts?" in Proc. Adv. Neural Inf. Process. Syst. (NIPS), Lake Tahoe, NV, USA, Dec. 2003, pp. 1141-1148.
34. L. B. Thomas, "Rank factorization of nonnegative matrices (A. Berman), " SIAM Rev., vol. 16, no. 3, pp. 393-394, 1974.
35. M. Kaykobad, "On nonnegative factorization of matrices, " Linear Algebra Appl., vol. 96, pp. 27-33, Nov. 1987.
36. A. Maurer and M. Pontil, "K-dimensional coding schemes in Hilbert spaces, " IEEE Trans. Inf. Theory, vol. 56, no. 11, pp. 5839-5846, Nov. 2010.
37. V. Vapnik, The Nature of Statistical Learning Theory. New York, NY, USA: Springer-Verlag, 2000.
38. T. Poggio and C. R. Shelton, "On the mathematical foundations of learning, " Amer. Math. Soc., vol. 39, no. 1, pp. 1-49, 2002.
39. P. L. Bartlett and S. Mendelson, "Rademacher and Gaussian complexities: Risk bounds and structural results, " J. Mach. Learn. Res., vol. 3, pp. 463-482, Mar. 2003.
40. S. Mendelson, "A few notes on statistical learning theory, " in Advanced Lectures on Machine Learning, S. Mendelson and A. J. Smola, Eds. Berlin, Germany: Springer-Verlag, 2003, pp. 1-40.
41. P. M. Gruber, "Optimum quantization and its applications, " Adv. Math., vol. 186, no. 2, pp. 456-497, 2004.
42. X. Zhang, N. Guan, L. Lan, D. Tao, and Z. Luo, "Box-constrained projective nonnegative matrix factorization via augmented Lagrangian method, " in Proc. IEEE Int. Joint Conf. Neural Netw. (IJCNN), Beijing, China, Jul. 2014, pp. 1900-1906.
43. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction, " Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
44. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables, " J. Roy. Statist. Soc. B (Statist. Methodol.), vol. 68, no. 1, pp. 49-67, 2006.
45. H. Zou and T. Hastie, "Regularization and variable selection via the elastic net, " J. Roy. Statist. Soc. B (Statist. Methodol.), vol. 67, no. 2, pp. 301-320, 2005.
46. D. Kuang, H. Park, and C. Ding, "Symmetric nonnegative matrix factorization for graph clustering, " in Proc. SIAM Int. Conf. Data Mining (SDM), Anaheim, CA, USA, Apr. 2012, pp. 106-117.
47. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Online nonnegative matrix factorization with robust stochastic approximation, " IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1087-1099, Jul. 2012.
48. F. Wang, N. Lee, J. Hu, J. Sun, and S. Ebadollahi, "Towards heterogeneous temporal clinical event pattern discovery: A convolutional approach, " in Proc. 18th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD), Beijing, China, Aug. 2012, pp. 453-461.
49. F. Wang, T. Li, and C. Zhang, "Semi-supervised clustering via matrix factorization, " in Proc. SIAM Conf. Data Mining, Atlanta, GA, USA, Apr. 2008, pp. 1-12.
50. O. Dekel, "From online to batch learning with cutoff-Averaging, " in Proc. Adv. Neural Inf. Process. Syst. (NIPS), Vancouver, BC, Canada, Dec. 2009, pp. 377-384.
51. V. P. Pauca, J. Piper, and R. J. Plemmons, "Nonnegative matrix factorization for spectral data analysis, " Linear Algebra Appl., vol. 416, no. 1, pp. 29-47, 2006.
52. W. Hoeffding, "Probability inequalities for sums of bounded random variables, " J. Amer. Statist. Assoc., vol. 58, no. 301, pp. 13-30, 1963.
53. C. McDiarmid, "Concentration, " in Probabilistic Methods for Algorithmic Discrete Mathematics, M. Habib, C. McDiarmid, J. Ramirez-Alfonsin, and B. Reed, Eds. Berlin, Germany: Springer-Verlag, 1998, pp. 195-248.
54. M. Mohri, A. Rostamizadeh, and A. Talwalkar, Foundations of Machine Learning. Cambridge, MA, USA: MIT Press, 2012.
55. T. Zhang, "Covering number bounds of certain regularized linear function classes, " J. Mach. Learn. Res., vol. 2, pp. 527-550, Mar. 2002.
56. D. Pollard, Convergence of Stochastic Processes. New York, NY, USA: Springer-Verlag, 1984.
57. M. Ledoux and M. Talagrand, Probability in Banach Spaces: Isoperimetry and Processes. Berlin, Germany: Springer-Verlag, 1991.
58. O. Bousquet, S. Boucheron, and G. Lugosi, "Introduction to statistical learning theory, " in Advanced Lectures on Machine Learning, O. Bousquet, U. von Luxburg, and G. Rätsch, Eds. Berlin, Germany: Springer-Verlag, 2004, pp. 169-207.
59. T. Sim, S. Baker, and M. Bsat, "The CMU pose, illumination, and expression database, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 12, pp. 1615-1618, Dec. 2003.