Rate distortion and denoising of individual data using kolmogorov complexity

IEEE Transactions on Information Theory

Volumn 56, Issue 7, 2010, Pages 3438-3454

  Vereshchagin, Nikolai K ^a   Vitanyi, Paul M B ^b  

^a MOSCOW STATE UNIVERSITY   (Russian Federation)

^b CWI   (Netherlands)

Author keywords

Algorithmic rate distortion; Characterization; Denoising; Distortion families; Fitness of destination words; Individual data; Kolmogorov complexity; Rate distortion; Shapes of curves

Indexed keywords

CARDINALITIES; DE-NOISING; DISTORTION MEASURES; EUCLIDEAN; FITNESS OF DESTINATION WORDS; KOLMOGOROV COMPLEXITY; MUTUAL INFORMATIONS; RATE DISTORTIONS; RATE-DISTORTION FUNCTION; SHANNON'S THEORY;

COMPUTATIONAL COMPLEXITY; CURVE FITTING; ELECTRIC DISTORTION; PARALLEL PROCESSING SYSTEMS; SIGNAL DISTORTION;

ALGORITHMS;

EID: 77953789264     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2048491     Document Type: Article

References (27)

1. T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, NJ: Prentice-Hall, 1971.
2. T. Berger and J. D. Gibson, "Lossy source coding," IEEE Trans. Inf. Theory, vol.44, no.6, pp. 2693-2723, 1998.
3. M. Burrows and D. J. Wheeler, A Block-Sorting Lossless Data Compression Algorithm. Digital Equip. Corp., Syst. Res. Center, Tech. Rep. 124, May 1994.
4. S. C. Chang, B. Yu, and M. Vetterli, "Image denoising via lossy compression and wavelet thresholding," in Proc. Int. Conf. Image Process. (ICIP'97), 1997, vol.1, pp. 604-607.
5. D. Donoho, "The Kolmogorov sampler," Ann. Statist., submitted for publication.
6. P. Gács, J. Tromp, and P. M. B. Vitányi, "Algorithmic statistics," IEEE Trans. Inf. Theory, vol.47, no.6, pp. 2443-2463, 2001.
7. A. N. Kolmogorov, "Three approaches to the quantitative definition of information," Problems Inf. Transmiss., vol.1, no.1, pp. 1-7, 1965.
8. A. N. Kolmogorov, "Complexity of algorithms and objective definition of randomness," a talk at Moscow Math. Soc. meeting 4/16/1974.," Uspekhi Mat. Nauk, vol. 29, no. 4, p. 155, 1974, An abstract available in English translation in [22].
9. M. Li and P. M. B. Vitányi, An Introduction to Kolmogorov Complexity and Its Applications, 2nd, 3rd ed. New York: Springer-Verlag, 1997, 2008.
10. M. Li, J. H. Badger, X. Chen, S. Kwong, P. Kearney, and H. Zhang, "An information-based sequence distance and its application to whole mitochondrial genome phylogeny," Bioinformatics, vol.17, no.2, pp. 149-154, 2001.
11. M. Li, X. Chen, X. Li, B. Ma, and P. M. B. Vitanyi, "The similarity metric," IEEE Trans. Inf. Theory, vol.50, no.12, pp. 3250-3264, 2004.
12. B. K. Natarajan, "Filtering random noise from deterministic signals via data compression," IEEE Trans. Signal Process., vol.43, no.11, pp. 2595-2605, 1995.
13. J. Muramatsu and F. Kanaya, "Distortion-complexity and rate-distortion function," IEICE Trans. Fund., vol.E77-A, no.8, pp. 1224-1229, 1994.
14. A. Rumyantsev, "Transmission of information through a noisy channel in Kolmogorov complexity setting," Vestnik MGU, Seriya Matematika i Mechanika (Russian), to appear.
15. S. de Rooij and P. M. B.Vitanyi, "Approximating rate-distortion graphs of individual data: Experiments in lossy compression and denoising," IEEE Trans. Comput., Submitted. Also: Arxiv preprint cs.IT/0609121, 2006.
16. N. Saito, "Simultaneous noise suppression and signal compression using a library of orthonormal bases and the minimum description length criterion," in Wavelets in Geophys., E. Foufoula-Georgiou and P. Kumar, Eds. New York: Academic, 1994, pp. 299-324.
17. C. E. Shannon, "The mathematical theory of communication," Bell Syst. Tech. J., vol.27, no.379-423, pp. 623-656, 1948.
18. C. E. Shannon, "Coding theorems for a discrete source with a fidelity criterion," IRE Nat. Convent. Rec., Part 4, pp. 142-163, 1959.
19. A. Kh. Shen, "The concept of α,β-stochasticity in the Kolmogorov sense, and its properties," Soviet Math. Dokl., vol.28, no.1, pp. 295-299, 1983.
20. D. M. Sow and A. Eleftheriadis, "Complexity distortion theory," IEEE Trans. Inf. Theory, vol.49, no.3, pp. 604-608, 2003.
21. A. M. Turing, "On computable numbers, with an application to the Entscheidungsproblem," Proc. London Math. Soc., 42:2(1936), 230-265.
22. "Correction", Proc. London Math. Soc.,Ibid., 43(1937), 544-546.
23. N. K. Vereshchagin and P. M. B. Vitányi, "Kolmogorov's structure functions and model selection," IEEE Trans. Inf. Theory, vol.50, no.12, pp. 3265-3290, 2004.
24. n ," Discrete and Computational Geometry, vol.33, pp. 143-155, 2005.
25. V. V. V'yugin, "On the defect of randomness of a finite object with respect to measures with given complexity bounds," SIAM Theory Probab. Appl., vol.32, no.3, pp. 508-512, 1987.
26. E.-H. Yang and S.-Y. Shen, "Distortion program-size complexity with respect to a fidelity criterion and rate-distortion function," IEEE Trans. Inf. Theory, vol.39, no.1, pp. 288-292, 1993.
27. J. Ziv, "Distortion-rate theory for individual sequences," IEEE Trans. Inf. Theory, vol.IT-26, no.2, pp. 137-143, 1980.