An evaluation of retrieval effectiveness using spelling-correction and string-similarity matching methods on malay texts

Journal of the American Society for Information Science and Technology

Volumn 51, Issue 8, 2000, Pages 691-706

  Abu Bakar, Zainab ^a   Sembok, Tengku Mohd T ^a   Yusoff, Mohammed ^a  

^a UNIVERSITI KEBANGSAAN MALAYSIA   (Malaysia)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; CHARACTER RECOGNITION; COMPUTATIONAL METHODS; DYNAMIC PROGRAMMING; PATTERN MATCHING; PATTERN RECOGNITION SYSTEMS;

SPELLING-CORRECTION METHODS; STRING-SIMILARITY METHODS;

INFORMATION RETRIEVAL SYSTEMS;

EID: 0033721946     PISSN: 15322882     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (72)

1. Abu Bakar, Z., Sembok, T.M.T., & Yusoff, M. (1997). Experiments on conflation algorithm on Malay texts for document retrieval. In Proceedings of the 15th IASTED international conference on applied informatics (pp. 229-232). Innsbruck, Austria: IASTED-ACTA Press.
2. Adamson, G.W., & Boreham, J. (1974). The use of an association measure based on character structure to identify semantically related pairs of words and document titles. Information Storage and Retrieval, 10, 253-260.
3. Ahmad, F. (1995). A Malay language document retrieval system: An experimental approach and analysis. Ph.D. Thesis, Universiti Kebangsaan Malaysia.
4. Alberga, C.N. (1967). String similarity and misspellings. Communications of the ACM, 10(5), 302-313.
5. Allison, L., & Dix, T.I. (1986). A bit longest-common subsequence algorithm. Information Processing Letters, 23, 305-310.
6. Al-Kharashi, I.A., & Evens, M.W. (1994). Comparing words, stems and roots as index terms in an Arabic information retrieval system. Journal of the American Society for Information Science, 45(8), 548-560.
7. Angell, R.C., Freund, G.E., & Willett, P. (1983). Automatic spelling correction using a trigram similarity measure. Information Processing and Management, 19, 255-261.
8. Arcelus, A. (1990). Name management strategy for database integration. Alfred P. Sloan School of Management, WP # CIS-90-15. Cambridge, MA: Massachusetts Institute of Technology.
9. Bellman, R., & Dreyfus, S. (1962). Applied dynamic programming. Princeton, NJ: Princeton University Press.
10. Berghel, H.L. (1987). A logical framework for the correction of spelling errors in electronic journals. Information Processing and Management, 23, 477-494.
11. Bickel, M.A. (1987). Automatic correction to misspelled names: A fourth-generation language approach. Communications of the ACM, 30(3), 224-228.
12. Borgman, C.L., & Siegfried, S.L. (1992). Getty's syndrome and its cousins: A survey of personal name-matching. Journal of the American Society for Information Science, 43, 459-476.
13. Boyer, R.S., & Moore, J.S. (1977). A fast string searching algorithm. Communications of ACM 20(10), 762-772.
14. Damashek, M. (1995). Gauging similarity with n-grams: Language-independent categorization of text. Science, 267, 843-848.
15. Damerau, F.J. (1964). A technique for computer detection and correction of spelling errors. Communications of the ACM, 7, 171-176.
16. Davidson, L. (1962). Retrieval of misspelled names in an airline's passenger record system. Communications of the ACM, 5, 169-171.
17. Dunn, O.J. 1964. Multiple comparisons using rank sums. Technometrics, 6, 241-252.
18. Ehrenfeucht, A., & Haussler, D. (1988). A new distance metric on strings computable in linear time. Discrete Applied Mathematics, 20, 191-203.
19. Ekmekcioglu, F.C., Lynch, M.F., Robertson, A.M., Sembok, T.M.T., & Willett, P. (1996). Comparison of n-gram matching and stemming for terms conflation in English, Malay and Turkish texts. Text Technology, Wright State University, Lake Campus, 6(1), 1-14.
20. Faulk, R.D. (1964). An inductive approach to language translation. Communications of the ACM, 7, 647-653.
21. Frakes, W.B. 1992. Stemming algorithm. In W.B. Frakes, & R. BaezaYates (Eds.), Information retrieval: Data structures and algorithms (pp. 131-160), Englewood Cliffs, NJ: Prentice Hall.
22. Freund, G.E., Willett, P. (1982).Online identification of word variants and arbitrary truncation searching using a string similarity measure. Information Technology: Research and Development, 1, 177-187.
23. Gadd, T.N. (1988). Fisching fore werds: Phonetic retrieval of written text in information systems. Program, 22, 222-237.
24. Gadd, T.N. (1990). PHONIX: The algorithm. Program, 24, 363-366.
25. Graham, R.T., Rudorf, E.H. (1970). Dialect and spelling. Elementary English, 42, 363-376.
26. Hall, P.A.V., Dowling, G.R. (1980). Approximate string matching. Computing Surveys, 12, 381-402.
27. Hamming, R. (1982). Coding and information theory. Englewood Cliffs, NJ: Prentice Hall.
28. Harman, D. (1991). How effective is suffixing? Journal of the American Society for Information Science, 42, 7-15.
29. Hirschberg, D.S. (1975). A linear space algorithm for computing maximal common subsequences. Journal of the ACM 18(6), 341-343.
30. Hirschberg, D.S. (1977). Algorithm for the longest common subsequence problem. Journal of the ACM, 24, 664-675.
31. Keen, E.M. 1992. The effect of stemming strength on the effectiveness of output ranking. Proceedings of Informatics, 11.
32. Knuth, D.E., Morris, J.H., Pratt, V.R. (1977). Fast pattern matching in strings. Siam Journal on Computing 6(2), 323-350.
33. Kruskal, W.H., Wallis, W.A. (1952). Use of ranks in one-criterion variance analysis. Journal American Statistical Association, 47, 583-621.
34. Lennon, M., Peirce, D.S., Tarry, B.D., Willett, P. (1981). An evaluaton of some conflation algorithms for information retrieval. Journal of Information Science, 32, 177-183.
35. Levenshtein, V.I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Cybernetics and Control Theory, 10(8), 707-710.
36. Lovins, J.B. 1968. Development of a stemming algorithm. Mechanical Translation and Computational Linguistics, 11, 22-31.
37. Mitton, R. (1987). Spelling checkers, spelling correctors and the misspellings of poor spellers. Information Processing and Management, 23, 495-505.
38. Needleman, S.B., Wunsch, C.D. (1970). A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology, 48, 443-453.
39. Peterson, J.L. (1980). Computer programs for detecting and correcting spelling errors. Communications of the ACM, 23, 676-687.
40. Pollock, J.J. (1980). SPEEDCOP: Task A.I - Quantification. Chemical Abstracts Service Internal Report.
41. Pollock, J.J. (1981a). SPEEDCOP: Task B.2 - Automatic correction of misspellings. Chemical Abstracts Service Internal Report.
42. Pollock, J.J. (1981b). SPEEDCOP: Task final report. Chemical Abstracts Service Internal Report.
43. Pollock, J.J. (1982). Spelling error detection and correction by computer: some notes and a bibliography. Journal of Documentation, 38, 282-291.
44. Pollock, J.J., Zamora, A. (1983). Correction and characterization of spelling errors in scientific and scholarly text. Journal of the American Society for Information Science, 34, 282-291.
45. Pollock, J.J., Zamora, A. (1984). Automatic spelling correction in scientific and scholarly text. Communications of the ACM, 27,358-368.
46. Popovic, M., Willett, P. (1992). The effectiveness of stemming for natural-language access to Slovene textual data. Journal of the American Society for Information Science, 43(5), 384-390.
47. Porter, M.F. (1980). An algorithm for suffix stripping. Program, 14(3), 130-137.
48. Robertson, A.M. (1992). An evaluation of algorithmic techniques for the identification of word variants in historical text database. Ph.D. Thesis. Department of Information Studies, University of Sheffield.
49. Robertson, A.M., Willett, P. (1993). A comparison of spelling-correction methods for the identification of word forms in historical text databases. Literary and Linguistic Computing, 8(3), 143-152.
50. Rogers, H.J. (1990). Investigation of the use of automatic spelling correction algorithms for retrieval of old-fashioned spellings of words. MSc dissertation. University of Sheffield, Department of Information Studies.
51. Rogers, H.J., Willett, P. (1991). Searching for historical word forms in text databases using spelling correction methods: Reverse error and phonetic coding methods. Journal of Documentation, 47, 333-353.
52. Russell, R.C. (1918). United States patent 1261167. Washington, DC: United States Patent Office.
53. Russell, R.C. (1922). United States patent 1435663. Washington, DC: United States Patent Office.
54. Sandy, R. (1990). Statistics for business and economics. Singapore: McGraw-Hill Book Co.
55. Sankoff, D., Kruskall, J.B. (1983). Time warps, string edits, and macromolecules: The theory and practice of sequence comparison. Reading, MA: Addison-Wesley.
56. Savoy, J. (1993). Stemming of French words based on grammatical categories. Journal of the American Society for Information Science, 44(1), 1-9.
57. Sedgewick, R. (1988). Algorithms (2nd ed., Chapter 42, pp. 595-605). Reading, MA: Addison-Wesley.
58. Sembok, T.M.T, Yussoff, M., Ahmad, F. (1994). A Malay stemming algorithm for information retrieval. In Proceedings of the 4th international conference and exhibition on multi-lingual computing (pp. 5.1.2.1-5.1.2.10).
59. Sembok, T.M.T., Willett, P. (1995). Experiments with n-gram string-similarity measure on Malay texts. Technical Report, Universiti Kebangssan Malaysia.
60. Sidorov, A.A. (1979). Analysis of word similarity in spelling correction systems. Program and Computer Software, 5, 274-277.
61. Smeaton, A., Geary, M. (1988). A study of existing sound indexing algorithm and a proposed enhancement. Dublin City University, School of Computer Applications, Working Paper Ca-0788.
62. Stephen, G.A. (1994). String searching algorithm. Lecture Notes Series on Computing (Volume 3). Singapore: World Scientific Publishing.
63. Ukkonen, E. (1992). Approximate string-matching with q-grams and maximal matches. Theoretical Computer Science, 92, 191-211.
64. van Rijsbergen, C.J. (1979). Information retrieval (2nd ed.). London: Butterworths.
65. Velichko, V.M., Zagoruyko, N.G. (1970). Automatic recognition of 200 words. International Journal of Man-Machine Studies, 2, 223-234.
66. Vintsyuk, T.K. (1968). Speech discrimination by dynamic programming. Cybernetics, 4(1), 52-57.
67. Wagner, R.A., Fischer, M.J. (1994). The string-to-string correction problem. Journal of the Association for Computing Machinery, 21(1), 168-173.
68. Also in Stephen, G.A. (1994). String searching algorithm. Lecture Notes Series on Computing (Volume 3, pp. 53-57), Singapore: World Scientific Publishing.
69. Walker, S., Jones, R.M. (1987). Improving subject retrieval in online catalogues. 1. Stemming, automatic spelling correction and cross reference tables. London, British Library Research Paper 24.
70. Wong, C.K., Chandra, A.K. (1976). Bounds for the string editing problem. Journal of the ACM, 23(1), 13-16.
71. van Rijsbergen, C.J. 1979. Information Retrieval (2nd ed.). London: Butterworths.
72. Yannakoudakis, E.J., Fawthrop, D. (1983). The rules of spelling errors. Information Processing and Management, 19, 101-108.