Sample complexity of Bayesian optimal dictionary learning

IEEE International Symposium on Information Theory - Proceedings

Volumn , Issue , 2013, Pages 669-673

  Sakata, Ayaka ^a   Kabashima, Yoshiyuki ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

BELIEF PROPAGATION; COMPRESSION RATES; DICTIONARY LEARNING; DIMENSIONAL VECTORS; LEARNING SCHEMES; LOW COMPUTATIONAL COMPLEXITY; POSTERIOR DISTRIBUTIONS; SAMPLE COMPLEXITY;

INFORMATION THEORY; SAMPLING; STATISTICAL MECHANICS;

OPTIMIZATION;

EID: 84890376112     PISSN: 21578095     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISIT.2013.6620310     Document Type: Conference Paper

References (19)

1. J.-L. Starck, F. Murtagh, and J. M. Fadili, Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity (Cambridge Univ. Press, New York, 2010).
2. H. Nyquist, Certain topics in telegraph transmission theory, Trans. AIEE 47 (2), pp. 617-644 (1928).
3. D. L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory 52 (4), pp. 1289-1306 (2006).
4. E. J. Candes, and T Tao, Decoding by Linear Programming, IEEE Trans. Inform. Theory 51 (12), pp. 4203-215 (2005).
5. B. A. Olshausen and D. J. Field, Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by VI?, Vision Res. 37 (23), pp. 33113325 (1997).
6. R. Rubinstein, A. M. Bruckstein, and M. Elad, Dictionaries for Sparse Representation Modeling,Froc.ofIEEE 98 (6), pp. 1045-1057 (2010).
7. M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing, (Springer-Verlag, New York, 2010).
8. S. Gleichman, and Y. C. Eldar, Blind Compressed Sensing, IEEE Inform. Theory 57, pp. 6958-6975 (2011).
9. M. Aharon, M. Elad, and A. M. Bruckstein, On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them, Linear Algebra and its Applications 416 (1), pp. 48-67 (2006).
10. D. Vainsencher, S. Mannor, and A. M. Bruckstein, The Sample Complexity of Dictionary Learning, Journal of Machine Learning Research 12, pp. 3259-3281 (2011).
11. A. Sakata, and Y. Kabashima, Statistical mechanics of dictionary learning, arXiv:1203.6178.
12. Y. Iba, The Nishimori line and Bayesian statistics, J. Phys. A: Math. Gen. 32 (21), 3875-3888 (1999)
13. M. Mezard, G. Parisi, and M. A. Virasoro, Spin Glass Theory and Beyond, (World Sci. Pub., 1987).
14. H. Nishimori, Statistical Physics of Spin Glasses and Information Processing: An Introduction, (Oxford Univ. Pr., 2001).
15. M. Mezard and A. Montanari, Information, Physics, and Computation, (Oxford Univ. Press, Oxford, UK, 2009).
16. F. Krzakala, M. Mezard, F. Sausset, Y. F. Sun, and L. Zdeborova Statistical-Physics-Based Reconstruction in Compressed Sensing,Vhys. Rev. X 2, pp. 021005-1-021005-18 (2012).
17. D. J. Thouless, P. W. Anderson, and R. G. Palmer, Solution of 'Solvable model of a spin glass, Phil. Mag. 35 (3), pp. 593-601 (1977).
18. K. Kabashima, A CDMA multiuser detection algorithm on the basis of belief propagation,3.?hys. A 36 (43), pp. 11111-11121 (2003).
19. F. Krzakala, M. Mezard, and L. Zdeborova, Phase Diagram and Approximate Message Passing for Blind Calibration and Dictionary Learning. Preprint received directly from the authors via private communication.