Algorithmic relative complexity

Entropy

Volumn 13, Issue 4, 2011, Pages 902-914

  Cerra, Daniele ^a   Datcu, Mihai ^a,b  

^a GERMAN AEROSPACE CENTER DLR   (Germany)

^b TELECOM PARISTECH   (France)

Author keywords

Compression; Compression based distance; Kolmogorov complexity; Kullback Leibler divergence; Relative entropy; Similarity measure

Indexed keywords

EID: 79955956801     PISSN: None     EISSN: 10994300     Source Type: Journal    
DOI: 10.3390/e13040902     Document Type: Article

References (23)

1. Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27, 379-423, 623-656.
2. Li, M.; Vitányi, P.M.B. An Introduction to Kolmogorov Complexity and Its Applications, 3rd edition; Springer: New York, NY, USA, 2008; Chapters 2, 8.
3. Li, M.; Chen, X.; Li, X.; Ma,B.; Vitányi, P.M.B. The similarity metric. IEEE Trans. Inform. Theor. 2004, 50, 3250-3264.
4. Cilibrasi, R. Statistical Inference through Data Compression; Lulu.com Press: Raleigh, NC, USA, 2006.
5. Ziv, J.; Merhav, N. A Measure of relative entropy between individual sequences with application to universal classification. IEEE Trans. Inform. Theor. 1993, 39, 1270-1279.
6. Benedetto, D.; Caglioti, E.; Loreto, V. Language trees and zipping. Phys. Rev. Lett. 2002, 88, 048702.
7. Solomonoff, R. A formal theory of inductive inference. Inform. Contr. 1964, 7, 1-22.
8. Kolmogorov, A.N. Three approaches to the quantitative definition of information. Probl. Inform. Transm. 1965, 1, 1-7.
9. Chaitin, G.J. On the length of programs for computing finite binary sequences: Statistical considerations. J. Assoc. Comput. Mach. 1969, 16, 145-159.
10. Vereshchagin, N.K.; Vitányi, P.M.B. Rate distortion and denoising of individual data using Kolmogorov complexity. IEEE Trans. Inform. Theor. 2010, 56, 3438-3454.
11. Chen, X.; Francia, B.; Li, M.; McKinnon, B.; Seker, A. Shared Information and Program Plagiarism detection. IEEE Trans. Inform. Theor. 2004, 50, 1545-1551.
12. Cilibrasi, R.; Vitányi, P.M.B. Clustering by Compression. IEEE Trans. Inform. Theor. 2005, 51, 1523-1545.
13. Keogh, E.J.; Lonardi, S.; Ratanamahatana, C. Towards parameter-free data mining. Proc. SIGKDD 2004, 10, 206-215.
14. Hutter, M. Algorithmic complexity. Scholarpedia 2008, 3, 2573.
15. Kullback, S.; Leibler, R.A. On information and sufficiency. Ann. Math. Stat. 1951, 22, 79-86.
16. MacKay, D.J.C. Information Theory, Inference, and Learning Algorithms; Cambridge University Press: Cambridge, UK, 2003; Chapter 2.
17. Welch, T.A. A technique for high-performance data compression. Computer 1984, 17, 8-19.
18. Watanabe, T.; Sugawara, K.; Sugihara, H. A new pattern representation scheme using data compression. IEEE Trans. Patt. Anal. Machine Intell. 2002, 24, 579-590.
19. The Liber Liber dataset. Available online: http://www.liberliber.it (accessed on 31 March 2011).
20. Coutinho, D.P.; Figueiredo, M. Information theoretic text classification using the Ziv-Merhav method. Proc. IbPRIA 2005, 2, 355-362.
21. Parodos.it. Grazia Deledda's biography. Available online: http://www.parodos.it/books/grazia_deledda.htm (accessed on 18 April 2011) (In Italian).
22. Cerra, D.; Mallet, A.; Gueguen, L.; Datcu, M. Algorithmic information theory based analysis of earth observation images: An assessment. IEEE Geosci. Remote Sens. Lett. 2010, 7, 9-12.
23. Cerra, D.; Datcu, M. Algorithmic cross-complexity and conditional complexity. Proc. DCC 2009, 19, 342-351.