Threshold effects in parameter estimation as phase transitions in statistical mechanics

IEEE Transactions on Information Theory

Volumn 57, Issue 10, 2011, Pages 7000-7010

  Merhav, Neri ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Additive white Gaussian noise channel; bandwidth; disordered systems; nonlinear modulation; parameter estimation; phase transitions; random energy model; statistical physics; threshold effect

Indexed keywords

ADDITIVE WHITE GAUSSIAN NOISE CHANNEL; DISORDERED SYSTEM; NONLINEAR MODULATION; RANDOM ENERGY MODELS; STATISTICAL PHYSICS; THRESHOLD EFFECT;

ESTIMATION; GAUSSIAN NOISE (ELECTRONIC); PHASE TRANSITIONS; PHYSICS; STATISTICAL MECHANICS; WHITE NOISE;

PARAMETER ESTIMATION;

EID: 80053959949     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2165824     Document Type: Article

References (27)

1. R. J. Adler, An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes, ser. Lecture Notes-Monograph Series. Japan: Institute of Mathematical Statistics, 1990, vol. 12.
2. A. Barg and G. D. Forney Jr., "Random codes: Minimum distances and error exponents," IEEE Trans. Inf. Theory, vol. 48, no. 9, pp. 2568-2573, Sep. 2002.
3. I. F. Blake and W. C. Lindsey, "Level-crossing problems for random processes," IEEE Trans. Inf. Theory, vol. IT-19, no. 3, pp. 295-315, May 1973.
4. B. Derrida, "Random-energy model: Limit of a family of disordered models," Phys. Rev. Lett., vol. 45, no. 2, pp. 79-82, Jul. 1980.
5. B. Derrida, "The random energy model," Phys. Rep. (Review Section of Physics Letters), vol. 67, no. 1, pp. 29-35, 1980.
6. B. Derrida, "Random-energy model: An exactly solvable model for disordered systems," Phys. Rev. B, vol. 24, no. 5, pp. 2613-2626, Sep. 1981.
7. P. A. Floor, "On the Theory of Shannon-Kotel'nikov Mappings in Joint Source-Channel Coding," Ph.D. dissertation, Norwegian University of Science and Engineering (NTNU), Trondheim, Norway, 2008.
8. P. A. Floor and T. A. Ramstad, "On the analysis of Shannon-Kotel'nikov mappings," in Proc. CoRR, 2009.
9. S. Franz, M. Leone, A. Montanari, and F. Ricci-Tersenghi, "Dynamic phase transition for decoding algorithms," Phys. Rev. E, 2002.
10. R. G. Gallager and C.W. Helstrom, "A bound on the probability that a Gaussian process exceeds a given function," IEEE Trans. Inf. Theory, vol. IT-15, no. 1, pp. 163-166, Jan. 1969.
11. F. Hekland, "On the Design and Analysis of Shannon-Kotel'nikov Mappings for Joint Source-Channel Coding," Ph.D. dissertation, Norwegian University of Science and Engineering (NTNU), Trondheim, Norway, 2007.
12. Y. Iba, "The Nishimori line and Bayesian statistics," J. Phys. A: Math. Gen., vol. 32, pp. 3875-3888, 1999.
13. M. B. Marcus and L. A. Shepp, "Sample behavior of Gaussian processes," in Proc. 6th Berkeley Symp. Math. Statist. Prob., 1971, vol. 2, pp. 423-442.
14. N. Merhav, "Relations between random coding exponents and the statistical physics of random codes," IEEE Trans. Inf. Theory, vol. 55, no. 1, pp. 83-92, Jan. 2009.
15. N. Merhav,D.Guo, and S. Shamai (Shitz), "Statistical physics of signal estimation in Gaussian noise: Theory and examples of phase transitions," IEEE Trans. Inf. Theory, vol. 56, no. 3, pp. 1400-1416, Mar. 2010.
16. M. Mézard and A. Montanari, Information, Physics and Computation. Oxford, U.K.: Oxford University Press, 2009.
17. A. Montanari, "The glassy phase of Gallager codes," Eur. Phys. J. B-Condensed Matter and Complex Systems, vol. 23, no. 1, pp. 121-136, 2001.
18. H. Nishimori, Statistical Physics of Spin Glasses and Information Processing: An Introduction. Oxford, U.K.: Clarendon Press, 2001.
19. T. A. Ramstad, "Shannon mappings for robust communication," Telektronikk, vol. 98, no. 1, pp. 114-128, 2002.
20. P. Ruján, "Finite temperature error-correcting codes," Phys. Rev. Let., vol. 70, no. 19, pp. 2968-2971, May 1993.
21. L. A. Shepp, "Radon-Nykodim derivatives of Gaussian measures," Ann. Math. Statist., vol. 37, pp. 321-354, Apr. 1966.
22. D. Slepian, "The one-sided barrier problem for Gaussian noise," Bell Syst. Tech. J., vol. 41, pp. 463-501, Mar. 1962.
23. N. Sourlas, "Spin glasses, error-correcting codes and finite-temperature decoding," Europhys. Lett., vol. 25, pp. 159-164, 1994.
24. A. J. Weiss, "Fundamental Bounds in Parameter Estimation," Ph.D. dissertation, Tel Aviv University, Tel Aviv, Israel, 1985.
25. A. J.Weiss and E.Weinstein, "A lower bound on the mean square error in random parameter estimation," IEEE Trans. Inf. Theory, vol. IT-31, no. 5, pp. 680-682, Sep. 1985.
26. J.M.Wozencraft and I.M. Jacobs, Principles of Communication Engineering. New York: Wiley 1965, reissued by Waveland Press, 1990.
27. M. Zakai and J. Ziv, "On the threshold effect in radar range estimation," IEEE Trans. Inf. Theory, vol. IT-15, no. 1, pp. 167-170, Jan. 1969.