Approximate message passing with restricted Boltzmann machine priors

Journal of Statistical Mechanics: Theory and Experiment

Volumn 2016, Issue 7, 2016, Pages

  Tramel, Eric W ^b   Drémeau, Angélique ^c   Krzakala, Florent ^a,b  

^a SORBONNE UNIVERSITÉ   (France)

^b ECOLE NORMALE SUPÉRIEURE   (France)

^c LAB STICC   (France)

Author keywords

message passing algorithms; networks; random graphs; statistical inference

Indexed keywords

EID: 84973380162     PISSN: None     EISSN: 17425468     Source Type: Journal    
DOI: 10.1088/1742-5468/2016/07/073401     Document Type: Article

References (24)

1. Candès E J and Romberg J K 2005 Signal recovery from random projections Proc. SPIE 5674 76-86
2. Donoho D L, Maleki A and Montanari A 2009 Message-passing algorithms for compressed sensing Proc. Natl Acad. Sci. USA 106 18914
3. Borgerding M A, Schniter P and Rangan S 2013 Generalized approximate message passing for cosparse analysis compressive sensing arXiv preprint (1312.3968 [cs.IT])
4. Rangan S, Fletcher A K, Goyal V K and Schniter P 2011 Hybrid approximate message passing with applications to structured sparsity arXiv preprint (1111.2581 [cs.IT])
5. Smolensky P 1986 Information processing in dynamical systems: foundations of harmony theory Parallel Distributed Porcessing: Explorations in the Microsctructure of Cognition vol 1 ed D E Rumelhart, J L McClelland (Cambridge, MA: MIT Press) pp 194-281
6. Hinton G E 2002 Training products of experts by minimizing contrastive divergence Neural Comput. 14 1771-800
7. Bengio Y 2009 Learning deep architectures for AI Found. Trends. Mach. Learn. 2 1-127
8. Pearl J 1982 Reverend Bayes on inference engines: a distributed hierarchical approach Proc. National Conf. on Artificial Intelligence pp 133-6
9. Mézard M and Montanari A 2009 Information, Physics, and Computation (New York: OUP)
10. Donoho D L, Maleki A and Montanari A 2010 Message-passing algorithms for compressed sensing: I. motivation and construction Proc. IEEE Information Theory Workshop (Piscataway, NJ: IEEE) pp 1-5
11. Rangan S 2011 Generalized approximate message passing for estimation with random linear mixing Proc. IEEE Int. Symp. on Information Theory (Piscataway, NJ: IEEE) pp 2168-72
12. Vila J P and Schniter P 2013 Expectation-maximization gaussian-mixture approximate message passing IEEE Trans. Signal Process. 61 4658-72
13. Krzakala F, Mézard M, Sausset F, Sun Y F and Zdeborová L 2012 Statistical physics-based reconstruction in compressed sensing Phys. Rev. X 2 021005
14. Krzakala F, Mézard M, Sausset F, Sun Y and Zdeborová L 2012 Probabilistic reconstruction in compressed sensing: algorithms, phase diagrams, and threshold achieving matrices J. Stat. Mech. P08009
15. Schniter P 2010 Turbo reconstruction of structured sparse signals 44th Annu. Conf. on Information Sciences and Systems (CISS) (Piscataway, NJ: IEEE) pp 1-6
16. Drémeau A, Herzet C and Daudet L 2012 Boltzmann machine and mean-field approximation for structured sparse decompositions IEEE Trans. Signal Process. 60 3425-38
17. Thouless D J, Anderson P W and Palmer R G 1977 Solution of 'solvable model of a spin-glass' Phil. Mag. 35 593-601
18. Mézard M, Parisi G and Virasoro M A 1987 Spin Glass Theory and Beyond: An Introduction to the Replica Method and Its Applications (World Scientific Lecture Notes in Physics vol 9) (Singapore: World Scientific)
19. Yedidia J 2001 An idiosyncratic journey beyond mean field theory Advanced Mean Field Methods: Theory and Practice ed M Opper and D Saad (Cambridge, MA: MIT Press) pp 21-36
20. Gabrié M, Tramel E W and Krzakala F 2015 Training restricted Boltzmann machines via the Thouless-Andreson-Palmer free energy Advances in Neural Information Processing Systems vol 28 ed C Cortes, N D Lawrence, D D Lee, M Sugiyama and R Garnett (Red Hook, NY: Curran Associates)
21. Plefka T 1982 Convergence condition of the TAP equation for the infinite-ranged ising spin glass model J. Phys. A: Math. Gen. 15 1971
22. Tieleman T 2008 Training restricted Boltzmann machines using approximations to the likelihood gradient ICML '08: Proc. 25th Int. Conf. on Machine Learning (Helsinki, Finland,) (New York: ACM) pp 1064-71
23. Hinton G E 2012 A practical guide to training restricted Boltzmann machines Neural Networks: Tricks of the Trade 2nd edn ed G Montavon, G B Orr and K-B Müller (Berlin: Springer) pp 599-619
24. Fischer A and Igel C 2012 An introduction to restricted Boltzmann machines Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 17th Iberoamerican Congress, CIARP 2012 (Buenos Aires, Argentina, 3-6 September 2012) ed L Alvarez, M Mejail, L Gomez and J Jacobo (Berlin: Springer) pp 14-36