The optimal hard threshold for singular values is 4/√3

IEEE Transactions on Information Theory

Volumn 60, Issue 8, 2014, Pages 5040-5053

  Gavish, Matan ^a   Donoho, David L ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

bulk edge; low rank matrix denoising; optimal threshold; quarter circle law; scree plot elbow truncation; Singular values shrinkage; unique admissible

Indexed keywords

RECOVERY; SHRINKAGE; WHITE NOISE;

BULK EDGE; DE-NOISING; OPTIMAL THRESHOLD; QUARTER CIRCLES; SCREE PLOT ELBOW TRUNCATION; SINGULAR VALUES; UNIQUE ADMISSIBLE;

MATRIX ALGEBRA;

EID: 84904607493     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2014.2323359     Document Type: Article

References (34)

1. D. L. Donoho and M. Gavish. (2014, Mar. 27). Code supplement to 'the optimal hard threshold for singular values is 4/√3' [Online]. Available: http://purl.stanford.edu/vg705qn9070
2. G. Golub and W. Kahan, "Calculating the singular values and pseudoinverse of a matrix," J. Soc. Ind. Appl. Math. B, vol. 2, no. 2, pp. 205-224, 1965.
3. C. Eckart and G. Young, "The approximation of one matrix by another of lower rank," Psychometrika, vol. 1, no. 3, pp. 211-218, 1936.
4. O. Alter, P. Brown, and D. Botstein, "Singular value decomposition for genome-wide expression data processing and modeling," Proc. Nat. Acad. Sci. United States Amer., vol. 97, no. 18, pp. 10101-10106, Aug. 2000.
5. D. Jackson. (1993). Stopping rules in principal components analysis: A comparison of heuristical and statistical approaches. Ecology [Online]. 74(8), pp. 2204-2214. Available: http://www.jstor.org/stable/10.2307/1939574
6. T. D. Lagerlund, F. W. Sharbrough, and N. E. Busacker, "Spatial filtering of multichannel electroencephalographic recordings through principal component analysis by singular value decomposition" J. Clin. Neurophysiol., vol. 14, no. 1, pp. 73-82, Jan. 1997. (Pubitemid 27220507)
7. A. L. Price, N. J. Patterson, R. M. Plenge, M. E. Weinblatt, N. A. Shadick, and D. Reich, "Principal components analysis corrects for stratification in genome-wide association studies," Nature Genet., vol. 38, no. 8, pp. 904-9, Aug. 2006. (Pubitemid 44141658)
8. O. Edfors, M. Sandell, J.-J. van de Beek, S. K. Wilson, and P. O. Borjesson, "OFDM channel estimation by singular value decomposition," IEEE Trans. Commun., vol. 46, no. 7, pp. 931-939, Jul. 1998. (Pubitemid 128739444)
9. R. B. Cattell, "The scree test for the number of factors," Multivariate Behavioral Res., vol. 1, no. 2, pp. 245-276, 1966.
10. S. Wold. (1978). Cross-validatory estimation of the number of components in factor and principal components models. Technometrics [Online]. 20(4), pp. 397-405. Available: http://www.tandfonline.com/doi/pdf/10.1080/00401706.1978. 10489693
11. P. D. Hoff. (2006, Sep.). Model averaging and dimension selection for the singular value decomposition [Online]. Available: http://arxiv.org/abs/math/ 0609042
12. A. B. Owen and P. O. Perry. (2009, Jun.). Bi-cross-validation of the SVD and the nonnegative matrix factorization. Ann. Appl. Statist. [Online]. 3(2), pp. 564-594. Available: http://projecteuclid.org/euclid.aoas/1245676186
13. P. O. Perry, "Cross validation for unsupervised learning," Ph.D. thesis, Dept. Statist., Stanford Univ., Stanford, CA, USA, 2009.
14. D. Achlioptas and F. McSherry, "Fast computation of low-rank matrix approximations," in Proc. 33rd Annu. ACM Symp. Theory Comput., 2001, pp. 611-618. (Pubitemid 32865778)
15. Y. Azar, A. Fiat, A. R. Karlin, F. McSherry, and J. Saia, "Spectral analysis of data," in Proc. 33rd Annu. ACM Symp. Theory Comput., 2001, pp. 619-626. (Pubitemid 32865779)
16. P. J. Bickel and E. Levina. (2008, Dec.). Covariance regularization by thresholding. Ann. Statist. [Online]. 36(6), pp. 2577-2604. Available: http://projecteuclid.org/euclid.aos/1231165180
17. S. Chatterjee. (2010). Matrix estimation by universal singular value thresholding [Online]. pp. 1-52. Available: http://arxiv.org/abs/1212.1247
18. R. H. Keshavan and S. Oh, "OptSpace: A gradient descent algorithm on the grassman manifold for matrix completion," pp. 1-26, 2009.
19. R. H. Keshavan, A. Montanari, and S. Oh, "Matrix completion from a few entries," IEEE Trans. Inf. Theory, vol. 56, no. 6, pp. 2980-2998, Jun. 2010.
20. J. Tanner and K. Wei. (2012). Normalized iterative hard thresholding for matrix completion [Online]. Available: http://people.maths.ox.ac.uk/tanner/ papers/TaWei-NIHT.pdf
21. Z. Bai and J. W. Silverstein, Spectral Analysis of Large Dimensional Random Matrices, 2nd ed. New York, NY, USA: Springer-Verlag, 2010.
22. A. A. Shabalin and A. B. Nobel, "Reconstruction of a low-rank matrix in the presence of Gaussian noise," J. Multivariate Anal., vol. 118, pp. 67-76, Jul. 2013.
23. I. M. Johnstone, "On the distribution of the largest eigenvalue in principal components analysis," Ann. Statist., vol. 29, no. 2, pp. 295-327, 2001.
24. D. L. Donoho, M. Gavish, and I. M. Johnstone, "Optimal shrinkage of eigenvalues in the spiked covariance model," Dept. Statist., Stanford Univ., Stanford, CA, USA, Tech. Rep. 2013-10, 2013.
25. F. Benaych-Georges and R. R. Nadakuditi, "The singular values and vectors of low rank perturbations of large rectangular random matrices," J. Multivariate Anal., vol. 111, pp. 120-135, Oct. 2012.
26. Y. Q. Yin, Z. D. Bai, and P. R. Krishnaiah, "On the limit of the largest eigenvalue of the large dimensional sample covariance matrix," Probab. Theory Rel. Fields, vol. 78, no. 4, pp. 509-521, 1988.
27. Y. Q. Yin and Z. D. Bai, "Limit of the smallest eigenvalue of a large dimensional sample covariance matrix," Ann. Probab., vol. 21, no. 3, pp. 1275-1294, 1993.
28. D. L. Donoho and M. Gavish, "Optimal shrinkage of singular values," Dept. Statist., Stanford Univ., Stanford, CA, USA, Tech. Rep. 2014-08, 2014.
29. J.-F. Cai, E. J. Candès, and Z. Shen. (2008). A singular value thresholding algorithm for matrix completion. SIAM J. Optim. [Online]. 20(4), pp. 1956-1982. Available: http://arxiv.org/abs/0810.3286
30. E. J. Candès, C. A. Sing-long, and J. D. Trzasko, "Unbiased risk estimates for singular value thresholding and spectral estimators," IEEE Trans. Signal Process., vol. 61, no. 19, pp. 4643-4657, Oct. 2012.
31. D. L. Donoho and M. Gavish, "Minimax risk of matrix denoising by singular value thresholding," Dept. Statist., Stanford Univ., Stanford, CA, USA, Tech. Rep. 2013-03, 2013.
32. S. Kritchman and B. Nadler, "Non-parametric detection of the number of signals: Hypothesis testing and random matrix theory," IEEE Trans. Signal Process., vol. 57, no. 10, pp. 3930-3941, Oct. 2009.
33. Z. Bai and J.-F. Yao, "Central limit theorems for eigenvalues in a spiked population model," Ann. l'Inst. Henri Poincare B, Probab. Statist., vol. 44, no. 3, pp. 447-474, Jun. 2008.
34. D. Shi. (2013, Apr.). Asymptotic joint distribution of extreme sample eigenvalues and eigenvectors in the spiked population model [Online]. Available: http://arxiv.org/abs/1304.6113