Matrix recipes for hard thresholding methods

Journal of Mathematical Imaging and Vision

Volumn 48, Issue 2, 2014, Pages 235-265

  Kyrillidis, Anastasios ^a   Cevher, Volkan ^a  

^a EPFL   (Switzerland)

Author keywords

Affine rank minimization; Hard thresholding; Randomized algorithms; approximation schemes

Indexed keywords

APPROXIMATION ALGORITHMS; MATRIX ALGEBRA;

APPROXIMATION SCHEME; HARD THRESHOLDING; LINEAR INVERSE PROBLEMS; MEMORY BASED TECHNIQUES; RANDOMIZED ALGORITHMS; RANK MINIMIZATIONS; RECONSTRUCTION ACCURACY; STATE-OF-THE-ART ALGORITHMS;

INVERSE PROBLEMS;

EID: 84898600635     PISSN: 09249907     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10851-013-0434-7     Document Type: Article

References (54)

1. Baraniuk, R.G., Cevher, V., Wakin, M.B.: Low-dimensional models for dimensionality reduction and signal recovery: a geometric perspective. Proc. IEEE 98(6), 959-971 (2010)
2. Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717-772 (2009)
3. Meka, R., Jain, P., Dhillon, I.S.: Guaranteed rank minimization via singular value projection. In: NIPS Workshop on Discrete Optimization in Machine Learning (2010)
4. Tyagi, H., Cevher, V.: Learning ridge functions with randomized sampling in high dimensions. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2025- 2028. IEEE Press, New York (2012)
5. Tyagi, H., Cevher, V.: Learning non-parametric basis independent models from point queries via low-rank methods. Technical report, EPFL (2012)
6. Liu, Y.K.: Universal low-rank matrix recovery from Pauli measurements (2011)
7. Tyagi, H., Cevher, V.: Active learning of multi-index function models. In: Advances in Neural Information Processing Systems, vol. 25, pp. 1475-1483 (2012)
8. Candes, E.J., Li, X.: Solving quadratic equations via phaselift when there are about as many equations as unknowns (2012). Preprint arXiv:1208.6247
9. Bennett, J., Lanning, S.: The netflix prize. In: KDD Cup and Workshop in Conjunction with KDD (2007)
10. Candes, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3) (2011)
11. Kyrillidis, A., Cevher, V.: Matrix alps: Accelerated low rank and sparse matrix reconstruction. Technical report, EPFL (2012)
12. Waters, A.E., Sankaranarayanan, A.C., Baraniuk, R.G.: Sparcs: recovering low-rank and sparse matrices from compressive measurements. In: NIPS (2011)
13. Fazel, M., Recht, B., Parrilo, P.A.: Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization. SIAM Rev. 52(3), 471-501 (2010)
14. Liu, Z., Vandenberghe, L.: Interior-point method for nuclear norm approximation with application to system identification. SIAM J. Matrix Anal. Appl. 31, 1235-1256 (2009)
15. Mohan, K., Fazel, M.: Reweighted nuclear norm minimization with application to system identification. In: American Control Conference (ACC). IEEE Press, New York (2010)
16. Cai, J.-F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20, 1956-1982 (2010)
17. Recht, B., Re, C.: Parallel stochastic gradient algorithms for largescale matrix completion. Preprint (2011)
18. Lin, Z., Chen, M., Ma, Y.: The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices (2010). preprint arXiv:1009.5055
19. Wright, J., Wu, L., Chen, M., Lin, Z., Ganesh, A., Ma, Y.: Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. UIUC Technical Report UILU-ENG-09-2214
20. Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227-234 (1995)
21. Lee, K., Bresler, Y.: Admira: atomic decomposition for minimum rank approximation. IEEE Trans. Inf. Theory 56(9), 4402-4416 (2010)
22. Goldfarb, D., Ma, S.: Convergence of fixed-point continuation algorithms for matrix rank minimization. Found. Comput.Math. 11, 183-210 (2011)
23. Beck, A., Teboulle, M.: A linearly convergent algorithm for solving a class of nonconvex/affine feasibility problems. In: Fixed- Point Algorithms for Inverse Problems in Science and Engineering, pp. 33-48 (2011)
24. Kyrillidis, A., Cevher, V.: Recipes on hard thresholding methods. In: Computational Advances in Multi-Sensor Adaptive Processing, Dec. 2011
25. Kyrillidis, A., Cevher, V.: Combinatorial selection and least absolute shrinkage via the Clash algorithm. In: IEEE International Symposium on Information Theory, July 2012
26. Halko, N., Martinsson, P.G., Tropp, J.A.: Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53, 217-288 (2011)
27. Bertsekas, D.: Nonlinear Programming. Athena Scientific, Nashua (1995)
28. Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge Univ. Press, Cambridge (1990)
29. Cohen, A., Dahmen,W., DeVore, R.: Compressed sensing and best k-term approximation. J. Am. Math. Soc. 22(1), 211-231 (2009)
30. Tropp, J.A.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231-2242 (2004)
31. Cevher, V.: An alps view of sparse recovery. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5808-5811. IEEE Press, New York (2011)
32. Needell, D., Tropp, J.A.: Cosamp: iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301-321 (2009)
33. Dai, W., Milenkovic, O.: Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans. Inf. Theory 55, 2230-2249 (2009)
34. Foucart, S.: Hard thresholding pursuit: an algorithm for compressed sensing. SIAM J. Numer. Anal. 49(6), 2543-2563 (2011)
35. Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265-274 (2009)
36. Garg, R., Khandekar, R.: Gradient descent with sparsification: an iterative algorithm for sparse recovery with restricted isometry property. In: ICML. ACM Press, New York (2009)
37. Blumensath, T., Davies, M.E.: Normalized iterative hard thresholding: guaranteed stability and performance. IEEE J. Sel. Top. Signal Process. 4(2), 298-309 (2010)
38. Blumensath, T.: Accelerated iterative hard thresholding. Signal Process. 92, 752-756 (2012)
39. Tanner, J.,Wei, K.: Normalized iterative hard thresholding for matrix completion. Preprint (2012)
40. Coifman, R., Geshwind, F., Meyer, Y.: Noiselets. Appl. Comput. Harmon. Anal. 10(1), 27-44 (2001)
41. Foucart, S.: Sparse recovery algorithms: sufficient conditions in terms of restricted isometry constants. In: Proceedings of the 13th International Conference on Approximation Theory (2010)
42. Nesterov, Y.: Gradient methods for minimizing composite objective function. core discussion papers 2007076, université catholique de louvain. Center for Operations Research and Econometrics (CORE) (2007)
43. Nesterov, Y.: Introductory Lectures on Convex Optimization. Kluwer Academic, Dordrecht (1996)
44. Drineas, P., Frieze, A., Kannan, R., Vempala, S., Vinay, V.: Clustering large graphs via the singular value decomposition. Mach. Learn. 56(1), 9-33 (2004)
45. Drineas, P., Kannan, R., Mahoney, M.W.: Fast Monte Carlo algorithms for matrices ii: computing a low-rank approximation to a matrix. SIAM J. Comput. 36, 158-183 (2006)
46. Deshpande, A., Rademacher, L., Vempala, S., Wang, G.: Matrix approximation and projective clustering via volume sampling. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm, SODA '06, New York, NY, USA, pp. 1117-1126. ACM Press, New York (2006)
47. Deshpande, A., Vempala, S.: Adaptive sampling and fast low-rank matrix approximation. Electron. Colloq. Comput. Complex. 13, 042 (2006)
48. Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from a few entries. IEEE Trans. Inf. Theory 56(6), 2980-2998 (2010)
49. Balzano, L., Nowak, R., Recht, B.: Online identification and tracking of subspaces from highly incomplete information. In: 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 704-711. IEEE Press, New York (2010)
50. He, J., Balzano, L., Lui, J.C.S.: Online robust subspace tracking from partial information (2011). arXiv:1109.3827
51. Boumal, N., Absil, P.A.: Rtrmc: a Riemannian trust-region method for low-rank matrix completion. In: NIPS (2011)
52. Wen, Z., Yin, W., Zhang, Y.: Solving a low-rank factorization model for matrix completion by a nonlinear successive overrelaxation algorithm. Rice University CAAM Technical Report TR10-07 (2010) Submitted
53. Larsen, R.M.: Propack: Software for large and sparse svd calculations. http://soi.stanford.edu/~rmunk/PROPACK
54. Shi, X., Yu, P.S.: Limitations of matrix completion via trace norm minimization. ACM SIGKDD Explor. Newsl. 12(2), 16-20 (2011)