An empirical Bayesian strategy for solving the simultaneous sparse approximation problem

IEEE Transactions on Signal Processing

Volumn 55, Issue 7 II, 2007, Pages 3704-3716

  Wipf, David P ^a   Rao, Bhaskar D ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Automatic relevance determination; Empirical Bayes; Multiple response models; Simultaneous sparse approximation; Sparse Bayesian learning; Variable selection

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; MATHEMATICAL MODELS; VECTORS;

AUTOMATIC RELEVANCE DETERMINATION; MULTIPLE RESPONSE MODELS; SIMULTANEOUS SPARSE APPROXIMATION; VARIABLE SELECTION;

BAYESIAN NETWORKS;

EID: 34347400802     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.894265     Document Type: Article

References (57)

1. H. Attias, "A variational Bayesian framework for graphical models," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2000, vol. 12.
2. M. J. Beal and Z. Ghahrarmani, "The variational Bayesian EM algorithm for incomplete data: With application to scoring graphical model structures," in Bayesian Statistics. Oxford, U.K.: Oxford Univ. Press, 2002, vol. 7.
3. J. O. Berger, Statistical Decision Theory and Bayesian Analysis, 2nd ed. New York: Springer-Verlag, 1985.
4. L. Breiman, "Better subset regression using the nonnegative garrote," Technometrics, vol. 37, no. 4, pp. 373-384, 1995.
5. J. Chen and X. Huo, "Sparse representations for multiple measurement vectors (MMV) in an overcomplete dictionary," in Proc. IEEE Int. Conf. Accoustics, Speech, Signal Processing, Mar. 2005, vol. 4, pp. 257-260.
6. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by Basis Pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1999.
7. S. F. Cotter, B. D. Rao, K. Engan, and K. Kreutz-Delgado, "Sparse solutions to linear inverse problems with multiple measurement vectors," IEEE Trans. Signal Process., vol. 53, no. 7, pp. 2477-2488, Jul. 2005.
8. A. P. Dempster, D. B. Rubin, and R. K. Tsutakawa, "Estimation in covariance components models," J. Amer. Statist. Assoc., vol. 76, no. 374, pp. 341-353, Jun. 1981.
9. D. L. Donoho, "On minimum entropy segmentation," in Wavelets: Theory, Algorithms, and Applications. New York: Academic, 1994, pp. 233-269.
10. 1 -norm solution is also the sparsest solution," Stanford Univ. Tech. Rep., Stanford, CA, Sep. 2004.
11. 1 minimization," Proc. Nat. Acad. Sci., vol. 100, no. 5, pp. 2197-2202, Mar. 2003.
12. D. L. Donoho and X. Huo, "Uncertainty principles and ideal atomic decomposition," IEEE Trans. Inf. Theory, vol. 47, no. 7, pp. 2845-2862, Jul. 2001.
13. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
14. I. J. Fevrier, S. B. Gelfand, and M. P. Fitz, "Reduced complexity decision feedback equalization for multipath channels with large delay spreads," IEEE Trans. Commun., vol. 47, no. 6, pp. 927-937, Jun. 1999.
15. M. A. T. Figueired, "Adaptive sparseness using Jeffreys prior," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2002, vol. 14, pp. 697-704.
16. J. J. Fuchs, "On sparse representations in arbitrary redundant bases," IEEE Trans. Inf. Theory, vol. 50, no. 6, pp. 1341-1344, Jun. 2004.
17. H. Gao, "Wavelet shrinkage denoising using the nonnegative garrote," J. Comput. Graph. Statist., vol. 7, no. 4, pp. 469-488, 1998.
18. I. F. Gorodnitsky, J. S. George, and B. D. Rao, "Neuromagnetic source imaging with FOCUSS: A recursive weighted minimum norm algorithm," J. Electroencephalogr. Clin. Neurophysiol., vol. 95, no. 4, pp. 231-251, Oct. 1995.
19. I. F. Gorodnitsky and B. D. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm," IEEE Trans. Signal Process., vol. 45, no. 3, pp. 600-616, Mar. 1997.
20. R. Gribonval and M. Nielsen, "Sparse representations in unions of bases," IEEE Trans. Inf. Theory, vol. 49, no. 12, pp. 3320-3325, Dec. 2003.
21. D. A. Harville, "Bayesian inference for variance components using only error contrasts," Biometrika, vol. 61, pp. 383-385, 1974.
22. D. A. Harville, "Maximum likelihood approaches to variance component estimation and to related problems," J. Amer. Statist. Assoc., vol. 72, pp. 320-338, Jun. 1977.
23. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. New York: Springer, 2001.
24. A. Hillebrand, G. Barnes, I. Holliday, and G. Harding, "Comparison of a modified FOCUSS algorithm with constrained and unconstrained dipole solutions on a simulated cortical surface," in Proc. 12th Int. Conf. Biomagnetism, 2000, pp. 753-756.
25. B. D. Jeffs, "Sparse inverse solution methods for signal and image processing applications," in Proc. IEEE Int. Conf. Accoustics, Speech, Signal Processing, May 1998, vol. 3, pp. 1885-1888.
26. R. A. Johnson and D. W. Wichern, Applied Multivariate Statistical Analysis, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1992.
27. M. I. Jordan, Z. Ghahramani, T. Jaakkola, and L. K. Saul, "An introduction to variational methods for graphical models," Mach. Learn., vol. 37, no. 2, pp. 183-233, 1999.
28. S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993.
29. D. J. C. MacKay, "Bayesian interpolation," Neural Comput., vol. 4, no. 3, pp. 415-447, 1992.
30. D. J. C. MacKay, "Bayesian non-linear modelling for the energy prediction competition," ASHRAE Trans., vol. 100, no. 2, pp. 1053-1062, 1994.
31. D. M. Malioutov, M. Çetin, and A. S. Willsky, "Sparse signal reconstruction perspective for source localization with sensor arrays," IEEE Trans. Signal Process., vol. 53, no. 8, pp. 3010-3022, Aug. 2005.
32. S. G. Mallat and Z. Zhang, "Matching pursuits with time-frequency dictionaries," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
33. P. Moulin and J. Liu, "Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors," IEEE Trans. Inf. Theory, vol. 45, no. 3, Apr. 1999.
34. R. M. Neal, Bayesian Learning for Neural Networks. New York: Springer-Verlag, 1996.
35. J. A. Palmer, D. P. Wipf, K. Kreutz-Delgado, and B. D. Rao, "Variational EM algorithms for non-Gaussian latent variable models," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2006, vol. 18.
36. C. Phillips, J. Mattout, M. D. Rugg, P. Maquet, and K. J. Friston, "An empirical Bayesian solution to the source reconstruction problem in EEG," NeuroImage, vol. 24, pp. 997-1011, Jan. 2005.
37. J. W. Phillips, R. M. Leahy, and J. C. Mosher, "MEG-based imaging of focal neuronal current sources," IEEE Trans. Med. Imag., vol. 16, no. 3, pp. 338-348, Mar. 1997.
38. R. R. Ramírez, "Neuromagnetic source imaging of spontaneous and evoked human brain dynamics," Ph.D. dissertation, New York Univ., New York, May 2005.
39. R. R. Ramírez and S. Makeig, "Neuroelectromagnetic source imaging using multiscale geodesic neural bases and sparse Bayesian learning," presented at the Human Brain Mapping, 12th Annu. Meeting, Florence, Italy, Jun. 2006.
40. B. D. Rao and K. Kreutz-Delgado, "Basis selection in the presence of noise," in Proc. 32nd Asilomar Conf. Signals, Systems, Computers, Nov. 1998, vol. 1, pp. 752-756.
41. B. D. Rao, K. Engan, S. F. Cotter, J. Palmer, and K. Kreutz-Delgado, "Subset selection in noise based on diversity measure minimization," IEEE Trans. Signal Process.,vol. 51, no. 3, pp. 760-770, Mar. 2003.
42. C. E. Rasmussen, "Evaluation of Gaussian processes and other methods for non-linear regression," Ph.D. dissertation, Grad. Dept. Comput. Sci., Univ. Toronto, Toronto, ON, Canada, 1996.
43. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
44. R. T. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1970.
45. J. G. Silva, J. S. Marques, and J. M. Lemos, "Selecting landmark Points for sparse manifold learning," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2006, vol. 18.
46. P. Stoica and R. Moses, Introduction to Spectral Analysis. Upper Saddle River, NJ: Prentice-Hall, 1997.
47. R. Tibshirani, "Regression shrinkage and selection via the LASSO," J. Roy. Statist. Soc., vol. 58, no. 1, pp. 267-288, 1996.
48. M. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
49. J. A. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2231-2242, Oct. 2004.
50. J. A. Tropp, "Algorithms for simultaneous sparse approximation. Part II: Convex relaxation," EURASIP J. Signal Process., vol. 86, pp. 589-602, Apr. 2006.
51. J. A. Tropp, A. C. Gilbert, and M. J. Strauss, "Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit," EURASIP J. Signal Process., vol. 86, pp. 572-588, Apr. 2006.
52. M. B. Wakin, M. F. Duarte, S. Sarvotham, D. Baron, and R. G. Baraniuk, "Recovery of jointly sparse signals from a few random projections," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2006, vol. 18.
53. D. P. Wipf, J. A. Palmer, and B. D. Rao, "Perspectives on sparse Bayesian learning," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2004, vol. 16.
54. p diversity measures," presented at the IEEE Int. Conf. Accoustics, Speech, Signal Processing, May 2004.
55. 0-norm minimization for basis selection," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2005, vol. 17.
56. D. P. Wipf and B. D. Rao, "Finding sparse representations in multiple response models via Bayesian learning," presented at the Workshop on Signal Processing with Adaptive Sparse Structured Representations, Rennes, France, Nov. 2005.
57. D. P. Wipf, "Bayesian Methods for finding sparse representations," Ph.D. dissertation, Univ. California, San Diego, 2006.