Measuring the impact of character recognition errors on downstream text analysis

Proceedings of SPIE - The International Society for Optical Engineering

Volumn 6815, Issue , 2008, Pages

  Lopresti, Daniel ^a  

^a Lehigh University   (United States)

Author keywords

Error classification; Natural language processing; Optical character recognition; Text analysis

Indexed keywords

ERROR ANALYSIS; NATURAL LANGUAGE PROCESSING SYSTEMS; TEXT PROCESSING;

ERROR CLASSIFICATION; SENTENCE BOUNDARY DETECTION; TEXT ANALYSIS;

OPTICAL CHARACTER RECOGNITION;

EID: 41149164144     PISSN: 0277786X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1117/12.767131     Document Type: Conference Paper

References (21)

1. Y. Gotoh and S. Renals, "Sentence boundary detection in broadcast speech transcripts," in Proceedomgs of ISCA Tutorial and Research Workshop ASR-2000, (Paris, France), 2000.
2. D. D. Palmer and M. Ostendorf, "Improving information extraction by modeling errors in speech recognizer output," in Proceedings of the First International Conference on Human Language Technology Research, J. Allan, ed., 2001.
3. Workshop on Analytics for Noisy Unstructured Text Data, Hyderabad, India, January 2007. http://research.ihost.com/and2007/.
4. K. Taghva, J. Borsack, and A. Condit, "Effects of OCR errors on ranking and feedback using the vector space model," Information Processing and Management 32(3), pp. 317-327, 1996.
5. K. Taghva, J. Borsack, and A. Condit, "Evaluation of model-based retrieval effectiveness with OCR text," ACM Transactions on Information Systems 14, pp. 64-93, January 1996.
6. D. Miller, S. Boisen, R. Schwartz, R. Stone, and R. Weischedel, "Named entity extraction from noisy input: Speech and OCR," in Proceedings of the 6th Applied Natural Language Processing Conference, pp. 316-324, (Seattle, WA), 2000.
7. H. Jing, D. Lopresti, and C. Shih, "Summarizing noisy documents," in Proceedings of the Symposium on Document Image Understanding Technology, pp. 111-119, April 2003.
8. D. Lopresti, "Performance evaluation for text processing of noisy inputs," in Proceedings of the 20th Annual ACM Symposium on Applied Computing (Document Engineering Track), pp. 759-763, (Santa Fe, NM), March 2005.
9. "Tesseract open source OCR engine," November 2007. http://sourceforge.net/projects/tesseract-ocr.
10. J. C. Reynar and A. Ratnaparkhi, "A maximum entropy approach to identifying sentence boundaries," in Proceedings of the Fifth Conference on Applied Natural Language Processing, (Washington, DC), March-April 1997. ftp://ftp.cis.upenn.edu/pub/adwait/jmx/jmx.tar.gz.
11. R. MacIntyre, "Penn Treebank tokenizer (sed script source code)," 1995. http://www.cis.upenn.edu/treebank/tokenizer.sed.
12. A. Ratnaparkhi, "A maximum entropy part-of-speech tagger," in Proceedings of the Empirical Methods in Natural Language Processing Conference, May 1996. ftp://ftp.cis.upenn.edu/pub/adwait/jmx/jmx.tar.gz.
13. J. Esakov, D. P. Lopresti, and J. S. Sandberg, "Classification and distribution of optical character recognition errors," in Proceedings of Document Recognition I (IS&T/SPIE Electronic Imaging), 2181, pp. 204-216, (San Jose, CA), February 1994.
14. J. Esakov, D. P. Lopresti, J. S. Sandberg, and J. Zhou, "Issues in automatic OCR error classification," in Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval, pp. 401-412, April 1994.
15. R. A. Wagner and M. J. Fischer, "The string-to-string correction problem," Journal of the Association for Computing Machinery 21, pp. 168-173, 1974.
16. "Project Gutenberg," November 2007. http://www.gutenberg.net/.
17. D. D. Lewis, "Reuters-21578 test collection, distribution 1.0," November 2007. http://www.daviddlewis.com/resources/testcollections/ reuters21578/.
18. J. Foster, "Treebanks gone bad: Generating a treebank of ungrammatical English," in Proceedings of the Workshop on Analytics for Noisy Unstructured Text Data, (Hyderabad, India), January 2007. http://research.ihost.com/and2007/cd/Proceedings.files/p39.pdf.
19. L. R. Blando, J. Kanai, and T. A. Nartker, "Prediction of OCR accuracy using simple image features," in Proceedings of the Third International Conference on Document Analysis and Recognition, pp. 319-322, (Montréal, Canada), August 1995.
20. M. Cannon, J. Hochberg, and P. Kelly, "Quality assessment and restoration of typewritten document images," Tech. Rep. LA-UR 99-1233, Los Alamos National Laboratory, 1999.
21. V. Govindaraju and S. N. Srihari, "Assessment of image quality to predict readability of documents," in Proceedings of Document Recognition III (IS&T/SPIE Electronic Imaging), 2660, pp. 333-342, (San Jose, CA), January 1996.