Low-rank structure learning via nonconvex heuristic recovery

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 3, 2013, Pages 383-396

  Deng, Yue ^a   Dai, Qionghai ^a   Liu, Risheng ^b   Zhang, Zengke ^a   Hu, Sanqing ^c  

^a TSINGHUA UNIVERSITY   (China)

^b DALIAN UNIVERSITY OF TECHNOLOGY   (China)

^c HANGZHOU DIANZI UNIVERSITY   (China)

Author keywords

Compressive sensing; Log sum heuristic; Matrix learning; Nuclear norm minimization; Sparse optimization

Indexed keywords

COMPRESSIVE SENSING; LOG-SUM HEURISTIC; LOW-RANK REPRESENTATIONS; NON-CONVEX OBJECTIVE FUNCTIONS; NUCLEAR NORM MINIMIZATIONS; ROBUST PRINCIPAL COMPONENT ANALYSIS; SPARSE OPTIMIZATIONS; TRADITIONAL APPROACHES;

ALGORITHMS; HEURISTIC METHODS; OPTIMIZATION; PRINCIPAL COMPONENT ANALYSIS;

ITERATIVE METHODS;

EID: 84878648508     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2235082     Document Type: Conference Paper

References (53)

1. X. Li and Y. Pang, "Deterministic column-based matrix decomposition," IEEE Trans. Knowl. Data Eng., vol. 22, no. 1, pp. 145-149, Jan. 2010.
2. Y. Yuan, X. Li, Y. Pang, X. Lu, and D. Tao, "Binary sparse nonnegative matrix factorization," IEEE Trans.Circuits Syst. Video Technol., vol. 19, no. 5, pp. 772-777, May 2009.
3. S. Hu and J. Wang, "Absolute exponential stability of a class of continuous-time recurrent neural networks," IEEE Trans. Neural Netw., vol. 14, no. 1, pp. 35-45, Jan. 2003.
4. A. Goldberg, X. J. Zhu, B. Recht, J. Sui, and R. Nowak, "Transduction with Matrix Completion: Three birds with one stone," in Proc. Neural Inform. Process. Syst., 2010, pp. 1-9.
5. G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma, "Robust recovery of subspace structures by low-rank representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 171-184, Jan. 2013.
6. S. Hu and J. Wang, "Quadratic stabilizability of a new class of linear systems with structural independent time-varying uncertainty," Automatica, vol. 37, no. 1, pp. 51-59, 2001.
7. J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma, "Robust principal component analysis: Exact recovery of corrupted low-rank matrices," in Proc. Neural Inform. Process. Syst., 2009, pp. 1-9.
8. Y. Deng, Y. Liu, Q. Dai, Z. Zhang, and Y. Wang, "Noisy depth maps fusion for multiview stereo via matrix completion," IEEE J. Select. Topics Signal Process., vol. 6, no. 5, pp. 566-582, Sep. 2012.
9. Y. Deng, Q. Dai, and Z. Zhang, "Graph laplace for occluded face completion and recognition," IEEE Trans. Image Process., vol. 20, no. 8, pp. 2329-2338, Aug. 2011.
10. Y. Deng, Q. Dai, R. Wang, and Z. Zhang, "Commute time guided transformation for feature extraction," Comput. Vis. Image Understand., vol. 116, no. 4, pp. 473-483, 2012.
11. R. Liu, Z. Lin, S. Wei, and Z. Su, "Feature extraction by learning lorentzian metric tensor and its extensions," Pattern Recognit., vol. 43, no. 10, pp. 3298-3306, 2010.
12. D. Donoho, "Compressed sensing," IEEE Trans. Inform. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
13. B. Recht, M. Fazel, and P. Parrilo, "Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization," SIAM Rev., vol. 52, no. 3, pp. 471-501, 2010.
14. E. J. Candes, X. Li, Y. Ma, and J. Wright, "Robust principal component analysis?" J. ACM, vol. 59, no. 3, pp. 1-37, May 2011.
15. V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky, "Rank-sparsity incoherence for matrix decomposition," Tech. Rep. 2220, Elect. Eng. Res. Lab., Univ. Texas, Austin, Jun. 2009.
16. D. Hsu, S. M. Kakade, and T. Zhang, "Robust matrix decomposition with outliers," Tech. Rep. 1011.1518, Dept. Stat., Univ. Pennsylvania, Philadelphia, PA, Dec. 2010.
17. G. Liu, Z. Lin, and Y. Yu, "Robust subspace segmentation by low-rank representation," in Proc. Int. Conf. Mach. Learn., 2010, pp. 663-670.
18. 1 minimization," J. Fourier Anal. Appl., vol. 14, no. 5, pp. 877-905, 2007.
19. Z. Yang, Y. Xiang, S. Xie, S. Ding, and Y. Rong, "Nonnegative blind source separation by sparse component analysis based on determinant measure," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 10, pp. 1601-1610, Oct. 2012.
20. Z. Lai, W. K. Wong, Z. Jin, J. Yang, and Y. Xu, "Sparse approximation to the eigensubspace for discrimination," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 12, pp. 1948-1960, Dec. 2012.
21. Z. Xu, X. Chang, F. Xu, and H. Zhang, "L1/2regularization: A thresholding representation theory and a fast solver," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1013-1027, Jul. 2012.
22. H. Li, N. Chen, and L. Li, "Error analysis for matrix elastic-net regularization algorithms," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 5, pp. 737-748, May 2012.
23. 1 filtering," Tech. Rep., 11085359, Elect. Eng. Res. Lab., Univ. Texas, Austin, Aug. 2011.
24. A. Ganesh, J. Wright, X. Li, E. J. Candes, and Y. Ma, "Dense error correction for low-rank matrices via principal component pursuit," in Proc. Int. Symp. Inform. Theory, Jun. 2010, pp. 1-5.
25. R. Vidal, "Subspace clustering," IEEE Signal Process. Mag., vol. 28, no. 2, pp. 52-68, Mar. 2011.
26. Z. Lin, R. Liu, and Z. Su, "Linearized alternating direction method with adaptive penalty for low-rank representation," in Proc. Neural Inform. Process. Syst., Sep. 2011, pp. 1-7.
27. C.-S. Foo, C. B. Do, and A. Y. Ng, "A majorization-minimization algorithm for (multiple) hyperparameter learning," in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 321-328.
28. M. Figueiredo, J. Bioucas-Dias, and R. Nowak, "Majorization minimization algorithms for wavelet-based image restoration," IEEE Trans. Image Process., vol. 16, no. 12, pp. 2980-2991, Dec. 2007. (Pubitemid 350201231)
29. J. Bioucas-Dias, M. Figueiredo, and J. Oliveira, "Total variation-based image deconvolution: A majorization-minimization approach," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., May 2006, p. 2.
30. M. Fazel, "Matrix rank minimization with applications," Ph.D. thesis, Stanford Univ., Stanford, CA, 2002.
31. M. Fazel, H. Hindi, and S. Boyd, "Log-det heuristic for matrix rank minimization with applications to hankel and euclidean distance matrices," in Proc. Amer. Control Conf., vol. 3. Jun. 2003, pp. 2156-2162.
32. K. Mohan and M. Fazel, "Reweighted nuclear norm minimization with application to system identification," in Proc. Amer. Control Conf., 2010, pp. 2953-2959.
33. H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," J. Royal Statist. Soc. Series B, vol. 67, no. 2, pp. 301-320, 2005. (Pubitemid 40465877)
34. R. Chartrand and W. Yin, "Iteratively reweighted algorithms for compressive sensing," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. Apr. 2008, pp. 3869-3872.
35. J. Cai, E. Candes, and Z. Shen, "A singular value thresholding algorithm for matrix completion," Preprint, Vol. 20, no. 4, pp. 1-27, 2008.
36. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Image Science, vol. 2, no. 1, pp. 183-202, 2009.
37. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed optimization and statistical learning via the alternating direction method of multipliers," Found. Trends Mach. Learn., vol. 3, no. 1, pp. 1-123, 2010.
38. Z. Lin, M. Chen, and Y. Ma, "The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices," Aerospace Corp., Los Angeles, CA, Tech. Rep. 10095055, Mar. 2011.
39. M. Lees, "A note on the convergence of alternating direction methods," Math. Comput., vol. 16, no. 77, pp. 70-75, 1963.
40. "Low-rank matrix recovery and completion via convex optimization." (2012) [Online]. Available: http://perception.csl.uiuc.edu/ matrix-rank/sample-code.html
41. J. Suo, S. Zhu, S. Shan, and X. Chen, "A compositional and dynamic model for face aging," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 3, pp. 385-401, Mar. 2010.
42. C. Benedek and T. Sziranyi, "Bayesian foreground and shadow detection in uncertain frame rate surveillance videos," IEEE Trans. Image Process., vol. 17, no. 4, pp. 608-621, Apr. 2008. (Pubitemid 351493883)
43. J. K. Suhr, H. G. Jung, G. Li, and J. Kim, "Mixture of gaussians-based background subtraction for bayer-pattern image sequences," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 3, pp. 365-370, Mar. 2011.
44. Type i and Type ii Errors. (2009) [Online]. Available: http://en.wikipedia.org/wiki/type-i-and-type-ii-errors
45. M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, pp. 381-395, Jun. 1981.
46. R. Vidal, Y. Ma, and S. Sastry, "Generalized principal component analysis (gpca)," in Proc. IEEE Comput. Soc. Conf., Comput. Vis. Pattern Recognit., Jun. 2003, pp. 621-628.
47. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
48. "Yahoo!finacial." (2012) [Online]. Available: http://finance.yahoo.com
49. "Google financial." (2012) [Online]. Available: http://www.google.com.hk/finance?q=
50. M. Gavrilov, D. Anguelov, P. Indyk, and R. Motwani, "Mining the stock market (extended abstract): Which measure is best?" in Proc. 6th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2000, pp. 487-496.
51. T. Wittman, "Time-series clustering and association analysis of financial data," Elect. Eng. Res. Lab., Univ. Texas, Austin, Tech. Rep. CS 8980, 2002.
52. "Bolzano-Weierstrass theorem." (2012) [Online]. Available: http://en.wikipedia.org/wiki/Bolzano%E2%80%93Weierstrass-theorem
53. "Squeeze theorem." (2012) [Online]. Available: http://en.wikipedia.org/wiki/Squeezetheorem