Unbiased estimation of a sparse vector in white Gaussian noise

IEEE Transactions on Information Theory

Volumn 57, Issue 12, 2011, Pages 7856-7876

  Jung, Alexander ^a   Ben Haim, Zvika ^b   Hlawatsch, Franz ^a   Eldar, Yonina C ^c  

^a VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

^b Google Inc   (Israel)

^c TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Barankin bound; Cram r Rao bound; denoising; Hammersley Chapman Robbins bound; locally minimum variance unbiased estimator; sparsity; unbiased estimation

Indexed keywords

BARANKIN BOUNDS; DENOISING; HAMMERSLEY-CHAPMAN-ROBBINS BOUND; MINIMUM VARIANCE UNBIASED ESTIMATOR; SPARSITY; UNBIASED ESTIMATION;

ASYMPTOTIC ANALYSIS; GAUSSIAN NOISE (ELECTRONIC); SIGNAL TO NOISE RATIO; WHITE NOISE;

ESTIMATION;

EID: 83255193537     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2170124     Document Type: Article

References (38)

1. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
2. E. J. Candés and T. Tao, "Near-optimal signal recovery from random projections: Universal encoding strategies?," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5406-5425, Dec. 2006.
3. D. L. Donoho and I. M. Johnstone, "Ideal spatial adaptation by wavelet shrinkage," Biometrika, vol. 81, pp. 425-455, 1994.
4. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image restoration by sparse 3D transform-domain collaborative filtering," in Proc. SPIE Electron. Imaging, San Jose, CA, Jan. 2008, DOI: 10.1117/12.766355.
5. M. Protter and M. Elad, "Image sequence denoising via sparse and redundant representations," IEEE Trans. Image Process., vol. 18, no. 1, pp. 27-35, Jan. 2009.
6. E. L. Lehmann and G. Casella, Theory of Point Estimation, 2nd ed. New York: Springer-Verlag, 1998.
7. E. G. Larsson and Y. Selen, "Linear regression with a sparse parameter vector," IEEE Trans. Signal Process., vol. 55, no. 2, pp. 451-460, Feb. 2007.
8. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
9. P. Schniter, L. Potter, and J. Ziniel, "Fast Bayesian matching pursuit," in Proc. Inf. Theory Appl. Workshop, San Diego, CA, Jan./Feb. 2008, pp. 326-333.
10. L. L. Scharf, Statistical Signal Processing. Reading, MA: Addison- Wesley, 1991.
11. S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993.
12. H. Cramér, "A contribution to the theory of statistical estimation," Skand. Akt. Tidskr., vol. 29, pp. 85-94, 1946.
13. J. Stein, "Inadmissibility of the usual estimator for the mean of a multivariate normal distribution," in Proc. 3rd Berkeley Symp. Math. Stat. Prob., 1956, pp. 197-206.
14. Z. Ben-Haim and Y. C. Eldar, "Blind minimax estimation," IEEE Trans. Inf. Theory, vol. 53, no. 9, pp. 3145-3157, Sep. 2007.
15. Y. C. Eldar, "Rethinking biased estimation: Improving maximum likelihood and the Cramér-Rao bound," in Foundations and Trends in Signal Processing. Hanover, MA: Now Publishers, 2008, vol. 1.
16. C. Carbonelli, S. Vedantam, and U. Mitra, "Sparse channel estimation with zero tap detection," IEEE Trans. Wireless Commun., vol. 6, no. 5, pp. 1743-1763, May 2007.
17. M. Dong and L. Tong, "Optimal design and placement of pilot symbols for channel estimation," IEEE Trans. Signal Process., vol. 50, no. 12, pp. 3055-3069, Dec. 2002.
18. D. L. Donoho and I. M. Johnstone, "Minimax risk over -balls for -error," Prob. Theory Relat. Fields, vol. 99, pp. 277-303, 1994.
19. D. L. Donoho and I. M. Johnstone, "Minimax estimation via wavelet shrinkage," Ann. Stat., vol. 26, pp. 879-921, 1998.
20. Z. Ben-Haim and Y. C. Eldar, "The Cramér-Rao bound for estimating a sparse parameter vector," IEEE Trans. Signal Process., vol. 58, no. 6, pp. 3384-3389, Jun. 2010.
21. Z. Ben-Haim and Y. C. Eldar, "Performance bounds for sparse estimation with random noise," in Proc. IEEE-SP Workshop Stat. Signal Process., Cardiff, Wales, U.K., Aug. 2009, pp. 225-228.
22. J. D. Gorman and A. O. Hero, "Lower bounds for parametric estimation with constraints," IEEE Trans. Inf. Theory, vol. 36, no. 6, pp. 1285-1301, Nov. 1990.
23. E. W. Barankin, "Locally best unbiased estimates," Ann. Math. Stat., vol. 20, no. 4, pp. 477-501, 1949.
24. A. Jung, Z. Ben-Haim, F. Hlawatsch, and Y. C. Eldar, "On unbiased estimation of sparse vectors corrupted by Gaussian noise," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Dallas,TX,Mar. 2010, pp. 3990-3993.
25. I. M. Johnstone and B.W. Silverman, "Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences," Ann. Stat., vol. 32, pp. 1594-1649, 2004.
26. G. Raskutti, M. J.Wainwright, and B. Yu, "Minimax rates of estimation for high-dimensional linear regression over -balls," IEEE Trans. Inf. Theory, vol. 57, no. 10, pp. 6976-6994, Oct. 2011.
27. G. Raskutti, M. J. Wainwright, and B. Yu, "Minimax rates of convergence for high-dimensional regression under -ball sparsity," in Proc. 47th Annu. Allerton Conf. Commun. Control Comput., Monticello, IL, Sep./Oct. 2009, pp. 251-257.
28. C. R. Rao, "Information and the accuracy attainable in the estimation of statistical parameters," Bull. CalcuttaMath. Soc., vol. 37, pp. 81-91, 1945.
29. J.M. Hammersley, "On estimating restricted parameters," J. Roy. Stat. Soc. B, vol. 12, no. 2, pp. 192-240, 1950.
30. D. G. Chapman and H. Robbins, "Minimum variance estimation without regularity assumptions," Ann. Math. Stat., vol. 22, no. 4, pp. 581-586, Dec. 1951.
31. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
32. J. S. Abel, "A bound on mean-square-estimate error," IEEE Trans. Inf. Theory, vol. 39, no. 5, pp. 1675-1680, Sep. 1993.
33. P. Forster and P. Larzabal, "On lower bounds for deterministic parameter estimation," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Orlando, FL, May 2002, vol. 2, pp. 1137-1140.
34. K. Todros and J. Tabrikian, "A new lower bound on the mean-square error of unbiased estimators," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Las Vegas, NV, Mar./Apr. 2008, pp. 3913-3916.
35. S. G. Mallat, A Wavelet Tour of Signal Processing-The Sparse Way, 3rd ed. San Diego, CA: Academic, 2009.
36. R. S. Strichartz, A Guide to Distribution Theory and Fourier Transforms, 1st ed. Hackensack, NJ: World Scientific, 1994.
37. D. S. Bernstein, MatrixMathematics. Princeton, NJ: Princeton Univ. Press, 2005.
38. A. Papoulis and S. U. Pillai, Probability, Random Variables and Stochastic Processes, 4th ed. New York: McGraw-Hill, 2002.