Statistical mechanics of lossy compression for nonmonotonic multilayer perceptrons

Physical Review E - Statistical, Nonlinear, and Soft Matter Physics

Volumn 78, Issue 2, 2008, Pages

  Cousseau, Florent ^a   Mimura, Kazushi ^b   Omori, Toshiaki ^c   Okada, Masato ^a,c  

^a UNIVERSITY OF TOKYO   (Japan)

^b HIROSHIMA CITY UNIVERSITY   (Japan)

^c RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; FUNCTIONS; MECHANICS; NEURAL NETWORKS; PATTERN RECOGNITION SYSTEMS; SPEED; STATISTICAL MECHANICS; STATISTICAL METHODS; TRANSFER FUNCTIONS;

BOOLEAN MESSAGES; CODE LENGTHS; COMMITTEE MACHINES; LOSSY COMPRESSIONS; LOSSY DATA COMPRESSION; MULTI-LAYER PERCEPTRONS; NON-MONOTONIC; REPLICA METHOD; SHANNON BOUND; TREE-LIKE NETWORKS;

DATA COMPRESSION;

EID: 50149097728     PISSN: 15393755     EISSN: 15502376     Source Type: Journal    
DOI: 10.1103/PhysRevE.78.021124     Document Type: Article

References (26)

1. N. Sourlas, Nature (London) NATUAS 0028-0836 10.1038/339693a0 339, 693 (1989).
2. Y. Kabashima, T. Murayama, and D. Saad, Phys. Rev. Lett. PRLTAO 0031-9007 10.1103/PhysRevLett.84.1355 84, 1355 (2000).
3. H. Nishimori and K. Y. Michael Wong, Phys. Rev. E PLEEE8 1063-651X 10.1103/PhysRevE.60.132 60, 132 (1999).
4. A. Montanari and N. Sourlas, Eur. Phys. J. B EPJBFY 1434-6028 10.1007/s100510070085 18, 107 (2000).
5. T. Tanaka, Europhys. Lett. EULEEJ 0295-5075 10.1209/epl/i2001-00306-3 54, 540 (2001).
6. T. Tanaka and M. Okada, IEEE Trans. Inf. Theory IETTAW 0018-9448 10.1109/TIT.2004.840887 51, 700 (2005).
7. T. Murayama and M. Okada, J. Phys. A JPHAC5 0305-4470 10.1088/0305-4470/36/43/031 36, 11123 (2003).
8. T. Murayama, Phys. Rev. E PLEEE8 1063-651X 10.1103/PhysRevE.69.035105 69, 035105 (R) (2004).
9. T. Hosaka, Y. Kabashima, and H. Nishimori, Phys. Rev. E PLEEE8 1063-651X 10.1103/PhysRevE.66.066126 66, 066126 (2002).
10. T. Hosaka and Y. Kabashima, J. Phys. Soc. Jpn. JUPSAU 0031-9015 10.1143/JPSJ.74.488 74, 488 (2005).
11. K. Mimura and M. Okada, Phys. Rev. E PLEEE8 1063-651X 10.1103/PhysRevE.74.026108 74, 026108 (2006).
12. C. E. Shannon, Bell Syst. Tech. J. BSTJAN 0005-8580 27, 379 (1948).
13. R. Gallager, IEEE Trans. Inf. Theory IETTAW 0018-9448 10.1109/TIT.1962.1057683 IT-8, 21 (1968).
14. D. J. C. MacKay and R. M. Neal, Electron. Lett. ELLEAK 0013-5194 10.1049/el:19970362 33, 457 (1997).
15. T. J. Richardson and R. L. Urbanke, IEEE Trans. Inf. Theory IETTAW 0018-9448 10.1109/18.910579 47, 638 (2001).
16. C. E. Shannon, IRE Natl. Conv. Rec. IRNCA8 0097-6318 4, 142 (1959).
17. Y. Matsunaga and H. Yamamoto, IEEE Trans. Inf. Theory IETTAW 0018-9448 10.1109/TIT.2003.815805 49, 2225 (2003).
18. E. Martinian and M. J. Wainwright, in Proceedings of the Data Compression Conference, 2006, edited by, IEEE Computer Society, (unpublished).
19. M. J. Wainwright, IEEE Signal Process. Mag. ISPRE6 1053-5888 24, 47 (2007).
20. T. Hosaka and Y. Kabashima, Physica A PHYADX 0378-4371 10.1016/j.physa.2006.01.013 365, 113 (2006).
21. T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, New York, 1991).
22. E. Gardner, J. Phys. A JPHAC5 0305-4470 10.1088/0305-4470/21/1/030 21, 257 (1988).
23. W. Krauth and M. Mézard, J. Phys. (Paris) JOPQAG 0302-0738 10.1051/jphys:0198900500200305700 50, 3057 (1989).
24. A. Engel and C. Van den Broeck, Statistical Mechanics of Learning (Cambridge University Press, Cambridge, U.K., 2001).
25. J. R. L. de Almeida and D. J. Thouless, J. Phys. A JPHAC5 0305-4470 10.1088/0305-4470/11/5/028 11, 983 (1978).
26. M. Mézard, G. Parisi, and R. Zecchina, Science SCIEAS 0036-8075 10.1126/science.1073287 297, 812 (2002).