Phase diagram and approximate message passing for blind calibration and dictionary learning

IEEE International Symposium on Information Theory - Proceedings

Volumn , Issue , 2013, Pages 659-663

  Krzakala, Florent ^a   Mezard, Marc ^b,c   Zdeborova, Lenka ^d  

^a Centre National de la Recherche Scientifique   (France)

^b ECOLE NORMALE SUPÉRIEURE   (France)

^c UNIVERSITÉ PARIS SUD   (France)

^d CNRS   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CALIBRATION PROBLEMS; DICTIONARY LEARNING; LARGE-SIGNALS; MEAN SQUARED ERROR; MESSAGE PASSING ALGORITHM; NUMERICAL TESTS; REPLICA METHOD; THEORETICAL PERFORMANCE;

INFORMATION THEORY;

CALIBRATION;

EID: 84890332329     PISSN: 21578095     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISIT.2013.6620308     Document Type: Conference Paper

References (27)

1. I. Carron, "The matrix factorization jungle," https://sites.google.com/site/igorcarron2/matrixfactorizations.
2. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by v1," Vision Research, vol. 37, p. 3311, 1997.
3. K. Engan, S. O. Aase, and H. J. Husoy, "Method of optimal directions for frame design," IEEE Acoustic, Speech and Signal Processing, vol. 5, p. 2443, 1999.
4. M. Aharon, M. Elad, and A. M. Bruckstein, "K-svd: An algorithm for designing overcomplete dictionaries for sparse representation," IEEE Transactions on Signal Processing, vol. 54, p. 4311, 2006.
5. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online dictionary learning for sparse coding," in Proceedings of the 26th Annual International Conference on Machine Learning, 2009, pp. 689-696.
6. P. Schniter, J. Parker, and V. Cevher, "Bilinear generalized approximate message passing (big-amp) for matrix recovery problem," in Workshop on Information Theory and Applications (ITA), San Diego CA, 2012.
7. A. M. B. Michal Aharon, Michael Elad, "On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them," Linear Algebra and its Applications, vol. 416, pp. 48-67, 2006.
8. R. Gribonval and K. Schnass, "Dictionary identification-sparse matrixfactorisation via -1-minimisation," IEEE Transactions on Information Theory, vol. 56, no. 7, pp. 3523-3539, 2010.
9. Q. Geng, H. Wang, and J. Wright, "On the local correctness of -1-minimization for dictionary learning," 2011, arXiv:1101.5672.
10. D. Vainsencher, S. Mannor, and A. M. Bruckstein, "The sample complexity of dictionary learning," Journal of Machine Learning Research, vol. 12, pp. 3259-3281, 2011.
11. R. Jenatton, R. Gribonval, and F. Bach, "Local stability and robustness of sparse dictionary learning in the presence of noise," arXiv:1210.0685, 2012.
12. A. Sakata and Y. Kabashima, "Statistical mechanics of dictionary learning," 2012, arXiv:1203.6178.
13. R. Gribonval, G. Chardon, and L. Daudet, "Blind calibration for compressed sensing by convex optimization," in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012, pp. 2713-2716.
14. R. Rubinstein, M. Zibulevsky, and M. Elad, "Double sparsity: Learning sparse dictionaries for sparse signal approximation," IEEE Transactions on Signal Processing, vol. 58, p. 1553, 2010.
15. M. Ḿezard, G. Parisi, and M. A. Virasoro, Spin-Glass Theory and Beyond. Singapore: World Scientific, 1987, vol. 9.
16. F. Krzakala, M. Ḿezard, F. Sausset, Y. Sun, and L. Zdeborov́a, "Statistical physics-based reconstruction in compressed sensing," Phys. Rev. X, vol. 2, p. 021005, 2012.
17. F. Krzakala, M. Ḿezard, F. Sausset, Y. Sun, and L. Zdeborov́a, "Probabilistic reconstruction in compressed sensing: Algorithms, phase diagrams, and threshold achieving matrices," J. Stat. Mech., 2012.
18. D. L. Donoho, A. Maleki, and A. Montanari, "Message-passing algorithms for compressed sensing," Proc. Natl. Acad. Sci., vol. 106, no. 45, pp. 18 914-18 919, 2009.
19. D. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing: I. motivation and construction," in IEEE Information Theory Workshop (ITW), 2010, pp. 1-5.
20. S. Rangan, "Generalized approximate message passing for estimation with random linear mixing," in IEEE International Symposium on Information Theory Proceedings (ISIT), 2011, pp. 2168-2172.
21. D. J. Thouless, P. W. Anderson, and R. G. Palmer, "Solution of 'solvable model of a spin-glass'," Phil. Mag., vol. 35, pp. 593-601, 1977.
22. F. Krzakala, M. Ḿezard, and L. Zdeborov́a, "Compressed sensing under matrix uncertainty: Optimum thresholds and robust approximate message passing," 2012, arXiv:1301.0901 [cs.IT], ICASSP 2013.
23. J. T. Parker, V. Cevher, and P. Schniter, "Compressive sensing under matrix uncertainties: An approximate message passing approach," in Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR), 2011, pp. 804-808.
24. M. Bayati and A. Montanari, "The dynamics of message passing on dense graphs, with applications to compressed sensing," IEEE Transactions on Information Theory, vol. 57, no. 2, pp. 764-785, 2011.
25. D. L. Donoho and J. Tanner, "Sparse nonnegative solution of underdetermined linear equations by linear programming," Proceedings of the National Academy of Sciences of the United States of America, vol. 102, no. 27, pp. 9446-9451, 2005.
26. Y. Wu and S. Verdu, "Optimal phase transitions in compressed sensing," 2011, arXiv:1111.6822v1 [cs.IT].
27. A. Sakata and Y. Kabashima, "Sample complexity of bayesian optimal dictionary learning," 2013, arXiv:1301.6199.