Bayesian orthogonal component analysis for sparse representation

IEEE Transactions on Signal Processing

Volumn 58, Issue 5, 2010, Pages 2675-2685

  Dobigeon, Nicolas ^a   Tourneret, Jean Yves ^a  

^a UNIVERSITÉ DE TOULOUSE   (France)

Author keywords

Bayesian inference; Dictionary learning; Markov chain Monte Carlo (MCMC) methods; Sparse representation

Indexed keywords

BAYESIAN; BAYESIAN FRAMEWORKS; BAYESIAN INFERENCE; BERNOULLI; BLIND SEPARATION; DICTIONARY LEARNING; DIMENSIONAL SPACES; GAUSSIAN PROCESSES; GIBBS SAMPLERS; HYPERPARAMETERS; MARKOV CHAIN MONTE CARLO METHOD; MAXIMUM A POSTERIORI ESTIMATOR; MIXING MATRIX; MODEL PARAMETERS; NON-INFORMATIVE PRIOR; OBSERVED DATA; ORTHOGONAL COMPONENTS; POSTERIOR DISTRIBUTIONS; PRIOR DISTRIBUTION; SPARSE CODING; SPARSE REPRESENTATION; STIEFEL MANIFOLD; SYNTHETIC DATA; UNKNOWN PARAMETERS;

BAYESIAN NETWORKS; BLIND SOURCE SEPARATION; GAUSSIAN DISTRIBUTION; INFERENCE ENGINES; MARKOV PROCESSES; MATRIX ALGEBRA;

MONTE CARLO METHODS;

EID: 77951169269     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2010.2041594     Document Type: Article

References (49)

1. I. Gorodnitsky and B. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm," IEEE Trans. Signal Process., vol.45, no.3, pp. 600-616, Mar. 1997.
2. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol.52, no.4, pp. 1289-1306, Apr. 2006.
3. R. G. Baraniuk, "Compressive sensing," IEEE Signal Process. Mag., vol.24, no.4, pp. 118-121, Jul. 2007.
4. S. Mallat and Z. Zhang, "Matching pursuits with time-frequency dictionaries," IEEE Trans. Signal Process., vol.41, no.12, pp. 3397-3415, Dec. 1993.
5. J. A. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol.50, no.10, pp. 2231-2242, Oct. 2004.
6. D. Donoho and X. Huo, "Uncertainty principles and ideal atomic decompositions," IEEE Trans. Inf. Theory, vol.47, no.10, pp. 2845-2862, Nov. 2001.
7. S. Alliney and S. A. Ruzinsky, "An algorithm for the minimization of mixed l1 and l2 norms with application to Bayesian estimation," IEEE Trans. Signal Process., vol.42, no.3, pp. 618-627, Mar. 1994.
8. R. Gribonval and M. Nielsen, "Sparse representations in unions of bases," IEEE Trans. Inf. Theory, vol.49, no.12, pp. 3320-3325, Dec. 2003.
9. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process., vol.54, no.11, pp. 4311-4319, Nov. 2006.
10. M. Mishali and Y. C. Eldar, "Sparse source separation from orthogonal mixtures," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Apr. 2009, pp. 3145-3148.
11. H. Zou, T. Hastie, and R. Tibshirani, "Sparse principal component analysis," J. Comput. Graph. Stat., vol.15, no.2, pp. 262-286, 2006.
12. A. d'Aspremont, L. El Ghaoui, M. I. Jordan, and G. R. G. Lanckriet, "A direct formulation for sparse PCA using semidefinite programming," SIAM Rev., vol.49, no.3, pp. 434-448, 2007.
13. A. d'Aspremont, F. Bach, and L. El Ghaoui, "Optimal solutions for sparse principal component analysis," J. Mach. Learn. Res., vol.9, pp. 1269-1294, 2008.
14. P. O. Hoyer, "Non-negative matrix factorization with sparseness constraints," J. Mach. Learn. Res., vol.5, no.2, pp. 1457-1469, 2004.
15. J.-P. Brunet, P. Tamayo, T. R. Golub, and J. P. Mesirov, "Metagenes and molecular pattern discovery using matrix factorization," Proc. Nat. Academy Sci., vol.101, no.12, pp. 4164-4169, Mar. 2004.
16. S. Bourguignon and H. Carfantan, "Bernoulli-Gaussian spectral analysis of unevenly spaced astrophysical data," in Proc. IEEE Workshop Stat. Signal Process. (SSP), Bordeaux, France, Jul. 2005, pp. 811-816.
17. Q. Cheng, R. Chen, and T.-H. Li, "Simultaneous wavelet estimation and deconvolution of reflection seismic signals," IEEE Trans. Geosci. Remote Sens., vol.34, no.2, pp. 377-384, Mar. 1996.
18. C. Févotte and S. J. Godsill, "A Bayesian approach for blind separation of sparse sources," IEEE Trans. Audio, Speech, Lang. Process., vol.14, no.6, pp. 2174-2188, Nov. 2006.
19. N. Dobigeon, A. O. Hero, and J.-Y. Tourneret, "Hierarchical Bayesian sparse image reconstruction with application to MRFM," IEEE Trans. Image Process., vol.18, no.9, pp. 2059-2070, Sep. 2009.
20. J. J. Kormylo and J. M. Mendel, "Maximum likelihood detection and estimation of Bernoulli-Gaussian processes," IEEE Trans. Inf. Theory, vol.28, no.3, pp. 482-488, May 1982.
21. J. Idier and Y. Goussard, "Stack algorithm for recursive deconvolution of Bernoulli-Gaussian processes," IEEE Trans. Signal Process., vol.28, no.5, pp. 67-79, Sep. 1990.
22. M. Lavielle, "Bayesian deconvolution of Bernoulli-Gaussian processes," Signal Process., vol.33, no.1, pp. 67-79, Jul. 1993.
23. A. Doucet and P. Duvaut, "Bayesian estimation of state-space models applied to deconvolution of Bernoulli-Gaussian processes," Signal Process., vol.57, no.2, pp. 147-161, Mar. 1997.
24. C. Févotte, B. Torrésani, L. Daudet, and S. J. Godsill, "Sparse linear regression with structured priors and application to denoising of musical audio," IEEE Trans. Audio, Speech, Language Process., vol.16, no.1, pp. 174-185, Jan. 2008.
25. M. Ting, R. Raich, and A. O. Hero, "Sparse image reconstruction for molecular imaging," IEEE Trans. Image Process., vol.18, no.6, pp. 1215-1227, Jun. 2009.
26. M. Lavielle and E. Lebarbier, "An application of MCMC methods for the multiple change-points problem," Signal Process., vol.81, no.1, pp. 39-53, Jan. 2004.
27. E.Kuhn and M. Lavielle, "Coupling a stochastic approximation version of EM with an MCMC procedure," ESAIM Probab. Statist., vol.8, pp. 115-131, 2004.
28. C. P. Robert and G. Casella, Monte Carlo Statistical Methods, 2nd ed. New York: Springer, 2004.
29. E. Punskaya, C. Andrieu, A. Doucet, and W. Fitzgerald, "Bayesian curve fitting using MCMC with applications to signal segmentation," IEEE Trans. Signal Process., vol.50, no.3, pp. 747-758, Mar. 2002.
30. N. Dobigeon, J.-Y. Tourneret, and C.-I. Chang, "Semi-supervised linear spectral unmixing using a hierarchical Bayesian model for hyperspectral imagery," IEEE Trans. Signal Process., vol.56, no.7, pp. 2684-2695, Jul. 2008.
31. N. Dobigeon, S. Moussaoui, M. Coulon, J.-Y. Tourneret, and A. O. Hero, "Joint Bayesian endmember extraction and linear unmixing for hyperspectral imagery," IEEE Trans. Signal Process., vol.57, no.11, pp. 4355-4368, Nov. 2009.
32. D. Ge, J. Idier, and E. L. Carpentier, "A new MCMC algorithm for blind Bernoulli-Gaussian deconvolution," presented at the EURASIP Eur. Signal Process. Conf. (EUSIPCO), Lausanne, Switzerland, Aug. 2008.
33. D. A. van Dyk and T. Park, "Partially collapsed Gibbs samplers: Theory and methods," J. Amer. Stat. Soc., vol.103, no.482, pp. 790-796, Jun. 2008.
34. T. Park and D. A. van Dyk, "Partially collapsed Gibbs samplers: Illustrations and applications," J. Comput. and Graph. Stat., vol.18, no.2, pp. 283-305, Jun. 2009.
35. P. D. Hoff, "Model averaging and dimension selection for the singular value decomposition," J. Amer. Stat. Soc., vol.102, no.478, pp. 674-685, Jun. 2007.
36. N. Dobigeon, J.-Y. Tourneret, and M. Davy, "Joint segmentation of piecewise constant autoregressive processes by using a hierarchical model and a Bayesian sampling approach," IEEE Trans. Signal Process., vol.55, no.4, pp. 1251-1263, Apr. 2007.
37. Y. Chikuse, Statistics on Special Manifolds, ser. Lectures notes in statistics.. New York: Springer-Verlag, 2003, vol.174.
38. A. K. Gupta and D. K. Nagar, Matrix Variate Distributions, ser. Monographs and Surveys in Pure and Applied Mathematics. Boca Raton, FL: Chapman & Hall/CRC, 2000, vol.104.
39. R. J. Muirhead, Aspects of Multivariate Statistical Theory. Hoboken, NJ: Wiley, 2005.
40. B. Von Hohenbalken, "Uniform sampling on simplices and spheres," ACM SIGAPL APL Quote Quad, vol.11, no.1, pp. 16-18, Sep. 1980.
41. F. Champagnat, Y. Goussard, and J. Idier, "Unsupervised deconvolution of sparse spike trains using stochastic approximation," IEEE Trans. Signal Process., vol.44, no.12, pp. 2988-2998, Dec. 1996.
42. T. Blumensath and M. E. Davies, "Monte-Carlo methods for adaptive sparse approximations of time-series," IEEE Trans. Signal Process., vol.55, no.9, pp. 4474-4486, Sep. 2007.
43. C. Févotte, "Bayesian audio source separation," in Blind Speech Separation, S. Makino, T.-W. Lee, and H. Sawada, Eds. Dordrecht, The Netherlands: Springer, 2007, pp. 305-335.
44. N. Dobigeon and J.-Y. Tourneret, "Bayesian orthogonal component analysis for sparse representation. Extension to non-homogeneous sparsity level over times," Univ. of Toulouse, IRIT/INP-ENSEEIHT-TéSA, Toulouse, France, Tech. Rep., Nov. 2009 [Online]. Available: http://dobigeon.perso. enseeiht.fr/publis.html
45. A. T. A. Wood, "Simulation of the von Mises Fisher distribution," Comm. Statist. Simulation Comput., vol.23, pp. 157-164, 1994.
46. J.-M.Marin and C. P. Robert, Bayesian Core: A Practical Approach to Computational Bayesian Statistics. New York: Springer, 2007.
47. N. Bertin, R. Badeau, and G. Richard, "Blind signal decompositions for automatic transcription of polyphonic music: NMF and K-SVD on the benchmark," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Apr. 2007, vol.1, pp. I-65-I-68.
48. O. Bryt and M. Elad, "Compression of facial images using the K-SVD algorithm," J. Vis. Commun. Image Represent., vol.19, no.4, pp. 270-283, May 2008.
49. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," IEEE Trans. Image Process., vol.17, no.1, pp. 53-69, Jan. 2008.