n-Gram Statistics for Natural Language Understanding and Text Processing

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn PAMI-1, Issue 2, 1979, Pages 164-172

  Suen, Ching Y ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Character recognition context languages understanding n gram statistics positional distributions of letters text processing word length analysis

Indexed keywords

TEXT PROCESSING;

CHARACTER RECOGNITION;

EID: 0017952955     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.1979.4766902     Document Type: Article

References (102)

1. M. Aborn, H. Rubenstein, and T. D. Sterling, “Sources of contextual constraint upon words in sentences,” J. Exp. Psych., vol. 57, pp. 171–180, 1959.
2. C. N. Alberga, “String similarity and misspellings,” Commun. Ass. Comput. Mach., vol. 10, pp. 302–313, May 1967.
3. R. Alter, “Utilization of contextual constraints in automatic speech recognition,” IEEE Trans. Audio Electroacoust., vol. AU-16, pp. 6–11, Mar. 1968.
4. R. N. Ascher, G. M. Koppelman, M. J. Miller, G. Nagy, and G. L. Shelton, Jr., “An interactive system for reading unformatted printed text,” IEEE Trans. Comput., vol. C-20, pp. 1527–1543, Dec. 1971.
5. R. Attar, Y. Choueka, N. Dershowitz, and A. S. Fraenkel, “KEDMA-Linguistic tools for retrieval systems,” J. Ass. Com put. put. Mach., vol. 25, pp. 52–66, Jan. 1978.
6. A. D. Baddeley and R. Conrad, “Letter structure of the English language,” Nature, vol. 186, pp. 414–416, Apr. 1960.
7. L. R. Bahl and F. Jelinek, “Decoding for channels with insertions, deletions, and substitutions with applications to speech recognition,” IEEE Trans. Inform. Theory, vol. IT-21, pp. 404–411, 411, July 1975.
8. L. Baker and J. L. Santa, “Context, integration, and retrieval,” Memory and Cognition, vol. 5, pp. 308–314, May 1977.
9. C. R. Blair, “A program for correcting spelling errors,” Inform. Contr., vol. 3, pp. 60–67, 1960.
10. W. W. Bledsoe and I. Browning, “Pattern recognition and reading by machine,” in Proc. Eastern Joint Comput. Conf., 1959, pp. 225–232.
11. C. P. Bourne and D. F. Ford, “A study of the statistics of letters in English words,” Inform. Contr., vol. 4, pp. 48–67, 1961.
12. G. Carlson, “Techniques for replacing characters that are garbled on input,” in Proc. Spring Joint Comput. Conf., 1966, pp. 189–192.
13. J. B. Carroll, “Word-frequency studies and the lognormal distribution,” in Lanugage and Language Behavior, E. M. Zale, Ed. New York: Appleton-Century-Crofts, 1968, pp. 213–235.
14. J. B. Carroll, P. Davis, and B. Richman, Word Frequency Book. New York: American Heritage, 1971.
15. E. C. Carterette and M. H. Jones, “Redundancy in children's texts,” Science, vol. 140, pp. 1309–1311, 1963.
16. R. G. Casey and G. Nagy, “An autonomous reading machine,” IEEE Trans. Comput., vol. C-17, pp. 492–503, May 1968.
17. C. S. Christensen, “An investigation of the use of context in character recognition using graph searching,” Center for Applied Math., Cornell Univ., Ithaca, NY, Tech. Rep. AFOSR 68–2470, Nov. 1968.
18. C. Conrad, “Context effects in sentence comprehension: A study of the subjective lexicon,” Memory and Cognition, vol. 2, pp. 130–138, 1974.
19. R. W. Cornew, “A statistical method of spelling correction,” Inform Contr., vol. 12, pp. 79–93, 1968.
20. F. J. Damerau, “A technique for computer detection and correction of spelling errors,” Commun. Ass. Comput. Mach., vol. 7, pp. 171–176, Mar. 1964.
21. L. Davidson, “Retrieval of misspelled names in an airlines passenger record system,” Commun. Ass. Comput. Mach., vol. 5, pp. 169–171, 1962.
22. G. Dewey, Relative Frequency of English Speech Sounds. Cambridge: Harvard Univ. Press, 1923.
23. R. W. Donaldson and G. T. Toussaint, “Use of contextual constraints in recognition of contourtraced handprinted characters,” IEEE Trans. Comput., vol. C-19, pp. 1096–1099, Nov. 1970.
24. W. Doster, “Contextual postprocessing system for cooperation with a multiple choice character recognition systems,” IEEE Trans. Comput., vol. C-26, pp. 1090–1101, Nov. 1977.
25. R. O. Duda and P. E. Hart, “Experiments in the recognition of hand-printed text: Part II-Context analysis,” in Proc. Fall Joint Comput. Conf., 1968, pp. 1139–1149.
26. A. W. Edwards and R. L. Chambers, “Can a priori probabilities help in character recognition?” J. Ass. Comput. Mach., vol. 11, pp. 465–470, Oct. 1964.
27. R. W. Ehrich and K. J. Koehler, “Experiments in the contextual recognition of cursive script,” IEEE Trans. Comput., vol. C-24, pp. 182–194, Feb. 1975.
28. R. W. Ehrich, “A contextual post-processor for cursive script recognition-A summary,” in Proc. 1st Int. Joint Conf. Pattern Recognition, 1973, pp. 169–171.
29. E. G. Fisher, “The use of context in character recognition,” Ph.D. dissertation, Dep. Comput. Sci., Univ. Massachusetts, Amherst, Mar. 1976.
30. G. D. Forney, Jr., “The Viterbi algorithm,” Proc. IEEE, vol. 61, pp. 268–278, Mar. 1973.
31. H. F. Gaines, Cryptanalysis. New York: Dover, 1956, pp. 218–221.
32. J. J. Giangardella, J. F. Hudson, and R. S. Roper, “Spelling correction by vector representation using a digital computer,” IEEE Trans. Eng., Writing, Speech, vol. EWS-10, pp. 57–62, Dec. 1967.
33. P. M. Hahn and N. C. Randall, “A best-match file search procedure for postal directions,” in Proc. Int. Conf. Cybern. Soc., Oct. 1972, pp. 512–518.
34. A. R. Hanson, E. M. Riseman, and E. Fisher, “Context in word recognition,” Pattern Recognition, vol. 8, 35–45, 1976.
35. L. D. Harmon and E. J. Sitar, “Method and apparatus for correcting errors in mutilated text,” U.S. Patent 3 188 609, June 1965.
36. G. Herdan, The Advanced Theory of Language as Choice and Chance. Berlin: Springer-Verlag, 1966.
37. Y. Hoshino, “Word recognition apparatus,” U.S. Patent 4 010 445, Mar. 1977.
38. A. B. S. Hussain and R. W. Donaldson, “Suboptimal sequential decision schemes with on-line feature ordering,” IEEE Trans. Comput., vol. C-23, pp. 582–590, June 1974.
39. E. B. James and D. P. Partridge, “Adaptive correction of program statements,” Commun. Ass. Comput. Mach., vol. 16, pp. 27–37, Jan. 1973.
40. P. A. Kolers and M. T. Katzman, “Naming sequentially presented letters and words,” Language and Speech, vol. 9, pp. 84–95, 1966.
41. E. Korolev, “On automatic recognition of context,” in Proc. Int. Conf. Comput. Ling., 1971.
42. H. Kucera and W. N. Francis, Computational Analysisof Present-Day Day American English. Providence: Brown Univ. Press, 1967.
43. V. I. Levenstein, “Binary codes capable of correcting deletions, insertions, and reversals,” Soviet Phys.-Doklady, vol. 10, pp. 707–710, 1966.
44. M. S. Mayzner and M. E. Tresselt, “Tables of single-letter and digram frequency counts for various word-length and letter position combinations,” Psychonomic Monograph Suppl., vol. 1, pp. 13–32, 1965.
45. M. S. Mayzner, M. E. Tresselt, and B. R. Wolin, “Tables of trigram frequency counts for various word-length and letter position combinations,” Psychonomic Monograph Suppl., vol. 1, pp. 33–78, 1965.
46. G. I. McCalla and J. R. Sampson, “MUSE: A model to understand simple English,” Commun. Ass. Comput. Mach., vol. 15, pp. 29–40, Jan. 1972.
47. C. K. McElwain and M. B. Evens, “The degarbler-A program for correcting machine-read Morse code,” Inform. Contr., vol. 5, pp. 368–384, 1962.
48. J.-G. Meunier, S. Rolland, and F. Daoust, “A system for text and content analysis,” Comput. and the Humanities, vol. 10, pp. 281–286, Sept./Oct. 1976.
49. G. A. Miller, G. A. Heise, and W. Lichten, “The intelligibility of speech as a function of the context of the test materials,” J. Exp. Psych., vol. 41, pp. 329–335, 1941.
50. G. A. Miller and E. A. Friedman, “The reconstruction of mutilated English texts,” Inform. Contr., vol. 1, pp. 38–55, 1957.
51. H. L. Morgan, “Spelling correction in systems programs,” Commun. Ass. Comput. Mach., vol. 13, pp. 90–94, Feb. 1970.
52. R. Morris and L. L. Cherry, “Computer detection of typographical errors,” IEEE Trans. Prof. Commun., vol. PC-18, pp. 54–64, Mar. 1975.
53. D. L. Neuhoff, “The Viterbi algorithm as an aid in text recognition,” IEEE Trans. Inform. Theory, vol. IT-21, pp. 222–226, Mar. 1975.
54. E. B. Newman and L. J. Gerstman, “A new method for analyzing printed English,” J. Exp. Psych., vol. 44, pp. 114–125, 1952.
55. E. B. Newman and N. C. Waugh, “The redundancy of texts in three languages,” Inform. Contr., vol. 3, pp. 141–153, 1960.
56. T. Okuda, E. Tanaka, and T. Kasai, “A method for the correction of garbled words based on the Levenshtein metric,” IEEE Trans. Comput., vol. C-25, pp. 172–178, Feb. 1976.
57. F. Pratt, Secret and Urgent-The Story of Codes and Ciphers. Indianapolis: Bobbs-Merril, 1939, pp. 252–278.
58. J. Raviv, “Decision making in Markov chains applied to the problem of pattern recognition,” IEEE Trans. Inform. Theory, vol. IT-3, pp. 536–55 Oct. 1967.
59. C. Rieger, “An organization of knowledge for problem solving and language comprehension,” Artificial Intelligence, vol. 7, pp. 89–127, 1976.
60. E. M. Riseman and R. W. Ehrich, “Contextual word recognition using binary bigrams,” IEEE Trans. Comput., vol. C-20, pp. 397–403, Apr. 1971.
61. E. M. Riseman and A. R. Hanson, “A contextual postprocessing system for error correction using binary n-grams,” IEEE Trans. Comput., vol. C-23, pp. 480–493, May 1974.
62. W. S. Rosenbaum and J. J. Hilliard, “Multifont OCR postprocessing systems,” IBM J. Res. Develop., vol. 19, pp. 398–421, July 1975.
63. R. Rustin, Ed., Natural Language Processing. New York: Algorithmics, 1973.
64. G. Salton and A. Wong, “On the role of words and phrases in automatic text analysis,” Comput. and the Humanities, vol. 10, pp. 69–87, Mar./Apr. 1976.
65. J. Schurmann, “Multifont word recognition system with application to postal address reading,” in Proc. 3rd Int. Joint Conf. Pattern Recognition, Nov. 1976, pp. 658–662.
66. V. Seth, “An approach to address identification from degraded address data,” in Proc. AFIPS, June 1977, pp. 779–783.
67. C. Shannon, “Prediction and entropy of printed English,” Bell Sys. Tech. J., vol. 30, pp. 50–64, 1951.
68. M. Shimura, “Recognition machines with parametric and nonparametric parametric learning methods using contextual information,” Pattern Recognition, vol. 5, pp. 149–168, 1973.
69. R. Shinghal, “Using contextual information to improve performance of character recognition machines,” Ph.D. dissertation, School of Comput. Sci., McGill Univ., Montreal, P.Q., Canada, Mar. 1977.
70. R. Shinghal, D. Rosenberg, and G. T. Toussaint, “A simplified heuristic version of Raviv's algorithm for using context in text recognition,” in Proc. 5th Int. Joint Conf. Artificial Intelligene, vol. 1, Aug. 1977, pp. 179–180.
71. R. Shinghal, D. Rosenberg, and G. T. Toussaint, “A simplified heuristic version of a recursive Bayes algorithm for using context in text recognition,” IEEE Trans. Syst., Man, Cybern., vol. SMC-8, pp. 412–414, May 1978.
72. G. Silva and H. Love, “The identification of variant texts by computer,” Inform. Storage and Retrieval, vol. 5, pp. 89–108, 1969.
73. F. Smith, “The use of featural dependencies across letters in the visual identification of words,” J. Verb. Learn. Verb. Behav., vol. 8, pp. 215–218, 1969.
74. R. L. Solso and J. King, “Frequency and versatility of letters in the English language,” Behav. Res. Meth. & Instrum., vol. 8, pp. 283–286, 1976.
75. C. Y. Suen and M. P. Beddoes, “Development of a digital spelled-speech speech reading machine for the blind,” IEEE Trans. Bio-Med. Eng., vol. BME-20, pp. 452–459, Nov. 1973.
76. C. Y. Suen, M. P. Beddoes, and J. C. Swail, “The Spellex system of speech aids for the blind,” in Proc. AFIPS, June 1976, pp. 217–220.
77. C. Y. Suen, M. Berthod, and S. Mori, “Advances in recognition of handprinted characters,” in Proc. 4th Int. Joint Conf. Pattern Recognition, Nov. 1978.
78. C. Y. Suen, C. Shiau, R. Shinghal, and C. C. Kwan, “Reliable recognition of handprinted characters,” in Proc. Joint Workshop Pat. Recog. & Art. Intel., June 1976, pp. 98–102.
79. C. Y. Suen and R. J. Shillman, “Low error rate optical character recognition of unconstrained handprinted characters based on a model of human perception,” IEEE Trans. Syst., Man, Cybern., vol. SMC-7, pp. 491–495, June 1977.
80. M. Symonds, “Computer detection of misspelled words,” Nat. Res. Council of Canada, Radio and Elec. Eng. Div., Ottawa, Tech. Rep. ERB-843, Aug. 1970.
81. A. J. Szanser, “Error-correcting methods in natural language processing,” in Proc. IFIP, vol. 2, 1969, pp. 1412–1416.
82. “Resolution of ambiguities by contextual word repetition,” Rev. Appl. Ling., vol. 7, pp. 49–56, 1970.
83. “Automatic error correction in natural languages,” Inform. Storage and Retrieval, vol. 5, pp. 169–174, 1970. Also, Statis. Meth. Ling., vol. 6, pp. 52–59, 1970.
84. “Automatic error correction in natural texts,” Nat. Physical Lab., England, Tech. Rep. Com Sci 53 and 63, Dec. 1971 and Jan. 1973.
85. E. Tanaka and T. Kasai, “A correcting method of garbled languages using ordered key letters,” Electron. Commun. Japan, vol. 55-D, pp. 127–133, 1972.
86. R. B. Thomas and M. Kassler, “Character recognition in context,” text,” Inform. Contr., vol. 10, pp. 43–64, 1967.
87. R. A. Thompson, “Language correction using probabilistic grammers,” IEEE Trans. Comput., vol. C-25, pp. 275–286, Mar. 1976.
88. L. E. Thorelli, “Automatic correction of errors in text,” BIT, vol. 2, pp. 45–62, 1962.
89. E. L. Thorndike and I. Lorge, The Teacher's Word Book of 30,000 Words. New York: Teachers College Press, 1944.
90. G. E. Topper, W. H. Macey, and R. L. Solso, “Bigram versatility and bigram frequency,” Behav. Res. Meth. & Instrum., vol. 5, pp. 1–53, 1973.
91. G. T. Toussaint and R. W. Donaldson, “Some simple contextual decoding algorithms applied to recognition of hand-printed printed text,” in Proc. Canadian Comput. Conf., 422102– 422116, 422116, 1972.
92. G. T. Toussaint, “Recent progress in statistical methods applied to pattern recognition,” in Proc. 2nd Int. Joint Conf. Pattern Recognition, Aug. 1974, pp. 479–488.
93. G. T. Toussaint, “The use of context in pattern recogntion,” in Proc. Conf. Pattern Recognition and Image Processing, June 1977, pp. 1–10.
94. E. Tulving and C. Gold, “Stimulus information and contextual information as determinants of tachistoscopic recognition of words,” J. Exp. Psych., vol. 66, pp. 319–327, 1963.
95. J. R. Ullmann, “A binary n-gram technique for automatic correction of substitution, deletion, insertion and reversal errors in words,” Comput. J., vol. 20, pp. 141–147, May 1977.
96. B. J. Underwood and R. W. Schulz, Meaningfulness and Verbal Learning. Chicago: Lippincott, 1960.
97. C. M. Vossler and N. M. Branston, “The use of context for correcting garbled English text,” in Proc. Nat. Ass. Comput. Mach. Conf., D2.4-1-D2.4-13, Aug. 1964.
98. R. A. Wagner and M. J. Fischer, “The string-to-string correction problem,” J. Ass. Comput. Mach., vol. 21, pp. 168–173, 1974.
99. J. Weizenbaum, “Contextual understanding by computer,” Commun. Ass. Comput. Mach., vol. 10, pp. 474–480, 1967.
100. Y. Wilks, “A preferential, pattern-seeking, semantics for natural language inference,” Artifical Intelligence, vol. 6, pp. 53–74, 1975.
101. T. Winograd, Understanding Natural Languages. New York: Academic, 1972.
102. G. K. Zipf, Human Behavior and the Principle of Least Effort. Cambridge, MA: Addison-Wesley, 1949.