Automated performance evaluation of document image analysis systems: issues and practice

International Journal of Imaging Systems and Technology

Volumn 7, Issue 4, 1996, Pages 363-369

  Kanai, Junichi ^a  

^a UNIVERSITY OF NEVADA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC TESTING; IMAGE ANALYSIS; OPTICAL CHARACTER RECOGNITION;

DOCUMENT IMAGE ANALYSIS SYSTEMS;

PATTERN RECOGNITION SYSTEMS;

EID: 0030378838     PISSN: 08999457     EISSN: None     Source Type: Journal    
DOI: 10.1002/(SICI)1098-1098(199624)7:4<363::AID-IMA11>3.0.CO;2-X     Document Type: Article

References (26)

1. R. M. Haralick, "Performance characterization protocol in computer vision," in Proceedings Workshop: Performance versus Methodology in Computer Vision, Seattle, WA, June 24-25, 1994, pp. 206-217.
2. G. Nagy, "Document image analysis: automated performance evaluation," in Document Image Analysis Systems. A. Dengel and L. Spitz, Eds., World Scientific, 1995, pp. 137-156.
3. J. Kanai, T. A. Nartker, S. V. Rice, and G. Nagy, "Performance metrics for document understanding systems," in Proceedings of the Second International Conference on Document Analysis and Recognition, Tsukuba Science City, Japan, Oct. 20-22, 1993, pp. 424-427.
4. S. V. Rice, "The OCR experimental environment," vers. 3, ISRI TR-93-06, University of Nevada, Las Vegas, 1993.
5. R. E. Probst, W. W. Klein, and G. Meyers, "Accuracy considerations in employing optical character recognition for automated data capture," TODAY (The Association for Work Processing Improvement) Sept.-Oct., 10-16 (1994).
6. A. D. Bagdanov and J. Kanai, "Evaluation of document image skew estimation techniques," in Document Recognition III, SPIE Proceeding Series, vol. 2660, 1996, pp. 343-353.
7. S. V. Rice, J. Kanai, and T. A. Nartker, An Evaluation of OCR Accuracy, ISRI TR-93-01, University of Nevada, Las Vegas, 1993.
8. F. Jenkins and J. Kanai, "The use of synthesized images to evaluate the performance of optical character recognition devices and algorithms," Document Recognition, SPIE Proceedings Series, vol. 2181, 1994, pp. 194-203.
9. O. D. Trier and A. K. Jain, "Goal-directed evaluation of binarization methods," IEEE Trans. Pattern Anal. Machine Intell. 17, 1191-1201 (1995).
10. L. Lam and C. Y. Suen, "Evaluation of thinning algorithms from an OCR viewpoint," in Proceedings of the Second International Conference on Document Analysis and Recognition, Tsukuba Science City, Japan, Oct. 20-22, 1993, pp. 287-290.
11. P. Stubberud, J. Kanai, and V. Kalluri, "Adaptive image restoration of text images that contain touching or broken characters," in Proceedings of the Third International Conference on Document Analysis and Recognition, Montreal, Canada, Aug. 14-16, 1995, pp. 778-781.
12. G. Nagy, "Validation of simulated OCR data sets," in Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval, University of Nevada, Las Vegas, 1994, pp. 127-136.
13. H. S. Baird, "Document image defect model," in Structured Document Image Analysis, H. S. Baird, H. Bunke, and K. Yamamoto, Eds., Springer-Verlag, 1992, pp. 546-556.
14. T. Kanungo, R. M. Haralick, and I. Phillips, "Global and local document degradation models," in Proceedings of the Second International Conference on Document Analysis and Recognition, Tsukuba Science City, Japan, Oct. 20-22, 1993, pp. 730-734.
15. H. S. Baird, "Calibration of document image defect models," in Proceedings of the Second Annual Symposium on Document Analysis and Information Retrieval, University of Nevada, Las Vegas, 1993, pp. 1-16.
16. Y. Li, D. Lopresti, and A. Tomkins, "Validation of document image defect models for optical character recognition," in Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval, University of Nevada, Las Vegas, 1994, pp. 137-150.
17. T. Kanungo, H S. Baird, and R. M. Haralick, "Validation and estimation of document degradation models," in Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval, University of Nevada, Las Vegas, 1995, pp. 217-229.
18. C. K. Chow, "On optimum recognition error tradeoff," IEEE Trans. IT-16, 41-46 (1970).
19. C. K. Chow, "Recognition error and reject trade-off," in Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval, University of Nevada, Las Vegas, 1994, pp. 1-8.
20. T. Matsui, T. Noumi, I. Yamashita, T. Wakahara, and M. Yoshimur, "State of the art or handwritten numeral recognition in Japan - the results of the first IPTP character recognition competition," in Proceedings of the Second International Conference on Document Analysis and Recognition, Tsukuba Science City, Japan, 1993, pp. 391-396.
21. R. A. Wagner and M. J. Fischer, "The string-to-string error correction problem," J. ACM. 21, 168-173 (1974).
22. H. F. Schantz, "Using OCR: Three crucial variables," INFORM. 6, 20-22 (1992).
23. H. S. Baird and J. Kanai, "Report of the evaluation-criteria working group, in Document Image Analysis Systems. A. Dengel and L. Spitz, Eds., World Scientific, 1995, pp. 455-457.
24. J. Kanai, S. V. Rice, T. A. Nartker, and G. Nagy, "Automated evaluation of OCR zoning," IEEE Trans. Pattern Anal. Machine Intell. 17, 86-90 (1995).
25. S. Randriamasy, L. Vincent, and B. Wittner, "An automatic benchmarking scheme for page segmentation," in Document Recognition, SPIE Proceeding Series, vol 2181, 1994, pp. 217-227.
26. S. Randriamasy and L. Vincent, "A region-based system for the automatic evaluation of page segmentation, algorithm," in Proceedings of the International Association for Pattern Recognition Workshop on Document Analysis Systems, Kaiserslautern, Germany, Oct., 1994.