Entropy of english text: Experiments with humans and a machine learning system based on rough sets

Information Sciences

Volumn 104, Issue 1-2, 1998, Pages 31-47

  Moradi, Hamid ^a   Grzymala Busse, Jerzy W ^a   Roberts, James A ^a  

^a UNIVERSITY OF KANSAS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; DATA COMPRESSION; DATA STRUCTURES; LEARNING SYSTEMS; SET THEORY;

ROUGH SETS;

OPTICAL CHARACTER RECOGNITION;

EID: 0031635619     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0020-0255(97)00074-1     Document Type: Article

References (27)

1. G. A. Barnard, Statistical calculation of word entropies for four western languages, IRE Trans. Inform. Theory IT-1:49-53 (1955).
2. G. P. Basharin, On a statistical estimate for the entropy of a sequence of independent random variable, Theory Probab. Appl. 4:333-336 (1959).
3. N. M. Blackman, Prevarication versus redundancy, Proc. IRE 1711-1712 (1962).
4. C. R. Blyth, Note on estimating information, Technical Report 17, Dept. Statistics, Stanford University, Stanford, CA, 1958.
5. N. G. Burton and J. C. R. Licklider, Long-range constraints in the statistical structure of printed English, Amer. J. Psych. 68:650-653 (1955).
6. T. M. Cover, A convergent gambling estimate of the entropy of English, IEEE Trans. Inform. Theory IT-24:413-421 (1978).
7. G. Dewey, Relative Frequency of English Speech Sounds, Harvard University Press, 1923.
8. M. Grignetti, A note on the entropy of words in printed English, Inform. Contr. 7:304-306 (1964).
9. J. W. Grzymala-Busse, Knowledge acquisition under uncertainty - A rough set approach, J. Intell. Robotic Syst. 1:3-16 (1988).
10. J. W. Grzymala-Busse, Managing Uncertainty in Expert Systems, Kluwer Academic Publishers, Dordrecht, 1991.
11. J. W. Grzymala-Busse, LERS - A system for learning from examples based on rough sets, in: R. Slowinski (Ed.), Intelligent Decision Support. Handbook of Applications and Advances of the Rough Set Theory, Kluwer Academic Publishers, Dordrecht, 1992, pp. 3-18.
12. D. Jamison and K. Jamison, A note on the entropy of partially known languages, Inform. Contr. 12:164-167 (1968).
13. V. Maixner, Some remarks on entropy prediction of natural language texts, Inform. Stor. Retr. 7:293-295 (1971).
14. G. A. Miller and J. A. Selfridge, Verbal context and the recall of meaningful material, Amer. J. Psych. 63:176-185 (1950).
15. T. Nemetz, On the experimental determination of the entropy, Kybernetika 10:137-139 (1972).
16. W. J. Paisley, The effects of authorship, topic structure, and time of composition on letter redundancy in English text, J. Verbal Learn. Behav. 5:28-34 (1966).
17. Z. Pawlak, Rough sets, Int. J. Comput. Inform. Sci. 11:341-356 (1982).
18. Z. Pawlak, Rough classification, Int. J. Man-Mach. Stud. 20:469-483 (1984).
19. E. Pfaffelhuber, Error estimation for the determination of entropy and information rate from relative frequency, Kybernetika 8:50-51 (1971).
20. F. Pratt, Secret and Urgent, Blue Ribbon Books, 1942.
21. A. P. Savchuk, On the evaluation of the entropy of language using the method of Shannon, Theory Probab. Appl. 9:154-157 (1964).
22. C. E. Shannon, Prediction and entropy of printed English, Bell Syst. Tech. J. 30:50-64 (1951).
23. A. Treisman, Verbal responses and contextual constraints in language, J. Verbal Learn. Behav. 4:118-128 (1965).
24. M. A. Wanas, A. I. Zayed, M. M. Shaker, and E. H. Taha, First-second-and third-order entropies of Arabic text, IEEE Trans. Inform. Theory IT-13:123 (1967).
25. H. E. White, Printed English compression by dictionary encoding, Proc. IEEE 55:390-396 (1967).
26. A. M. Yaglom and J. M. Yaglom, Probability and Information, Science House, Leningrad, 3rd revised ed., 1973, Chap. IV, Part 3, pp. 236-329.
27. J. Ziv and A. Lempel, Compression of individual sequences by variable rate coding, IEEE Trans. Inform. Theory IT-24:530-536 (1978).