Performance evaluation methodology for historical document image binarization

IEEE Transactions on Image Processing

Volumn 22, Issue 2, 2013, Pages 595-609

  Ntirogiannis, Konstantinos ^a,b   Gatos, Basilis ^b   Pratikakis, Ioannis ^c  

^a UNIVERSITY OF ATHENS   (Greece)

^b INSTITUTE OF INFORMATICS AND TELECOMMUNICATIONS   (Greece)

^c DEMOCRITUS UNIVERSITY OF THRACE   (Greece)

Author keywords

Document image binarization; ground truth; performance evaluation

Indexed keywords

BACKGROUND NOISE; BINARIZATIONS; DOCUMENT IMAGE ANALYSIS; DOCUMENT IMAGES; EVALUATION MEASURES; EVALUATION METHODOLOGIES; EVALUATION SCHEME; FALSE ALARMS; GROUND TRUTH; HISTORICAL DOCUMENTS; PERFORMANCE EVALUATION; PERFORMANCE EVALUATION METHODOLOGY; PERFORMANCE METRICS; POTENTIAL EVALUATION; RECALL AND PRECISION; RECOGNITION PROCESS; WEIGHTING SCHEME;

IMAGE PROCESSING; MATHEMATICAL MODELS;

PIXELS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; DOCUMENTATION; HANDWRITING; HUMAN; IMAGE PROCESSING; METHODOLOGY; PRINTING; REPRODUCIBILITY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; DOCUMENTATION; HANDWRITING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; PRINTING; REPRODUCIBILITY OF RESULTS;

MLCS; MLOWN;

EID: 84872256597     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2012.2219550     Document Type: Article

References (70)

1. A. Antonacopoulos and D. Karatzas, "Semantics-based content extraction in typewritten historical documents," in Proc. Int. Conf. Document Anal. Recognit., 2005, pp. 48-53. (Pubitemid 46442289)
2. B. Gatos, I. Pratikakis, and S. J. Perantonis, "Adaptive degraded document image binarization," Pattern Recognit., vol. 39, no. 3, pp. 317-327, Mar. 2006. (Pubitemid 41808613)
3. A. Antonacopoulos, D. Karatzas, H. Krawczyk, and B. Wiszniewski, "The lifecycle of a digital historical document: Structure and content," in Proc. ACM Symp. Document Eng., Milwaukee, WI, 2004, pp. 147-154. (Pubitemid 40788570)
4. A. Antonacopoulos and D. Karatzas, "Document image analysis for world war II personal records," in Proc. Int. Workshop Document Image Anal. Libraries, 2004, pp. 336-341.
5. B. Lamiroy, D. Lopresti, and T. Sun, "Document analysis algorithm contributions in end-to-end applications: Report on the ICDAR 2011 contest," in Proc. Int. Conf. Document Anal. Recognit., Beijing, China, Sep. 2011, pp. 1521-1525.
6. D. Trier and T. Taxt, "Evaluation of binarization methods for document images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 3, pp. 312-315, Mar. 1995.
7. I. K. Kim, D. W. Jung, and R. H. Park, "Document image binarization based on topographic analysis using a water flow model," Pattern Recognit., vol. 35, no. 1, pp. 265-277, 2002.
8. Y. Yang and H. Yan, "An adaptive logical method for binarization of degraded document images," Pattern Recognit., vol. 33, no. 5, pp. 787-807, 2000. (Pubitemid 30567261)
9. E. Kavallieratou and S. Stathis, "Adaptive binarization of historical document images," in Proc. Int. Conf. Pattern Recognit., vol. 3. Hong Kong, Aug. 2006, pp. 742-745. (Pubitemid 46553228)
10. J. G. Kuk, N. I. Cho, and K. M. Lee, "Map-MRF approach for binarization of degraded document image," in Proc. IEEE Int. Conf. Image Process., San Diego, CA, Oct. 2008, pp. 2612-2615.
11. J. G. Kuk and N. I. Cho, "Feature based binarization of document images degraded by uneven light condition," in Proc. Int. Conf. Document Anal. Recognit., Jul. 2009, pp. 748-752.
12. Q. Huang, W. Cao, and W. Cai, "Thresholding technique with adaptive window selection for uneven lighting image," Pattern Recognit. Lett., vol. 26, no. 6, pp. 801-808, 2005. (Pubitemid 40430499)
13. J. He, Q. D. M. Do, A. C. Downton, and J. H. Kim, "A comparison of binarization methods for historical archive documents," in Proc. Int. Conf. Document Anal. Recognit., vol. 1. Seoul, South Korea, Aug. 2005, pp. 538-542. (Pubitemid 46442383)
14. H. Cao and V. Govindaraju, "Preprocessing of low-quality handwritten documents using Markov random fields," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 7, pp. 1184-1194, Jul. 2009.
15. M. Ramirez, E. Tapia, M. Block, and R. L. Rojas, "Quantile linear algorithm for robust binarization of digitalized letters," in Proc. Int. Conf. Document Anal. Recognit., vol. 2. Curitiba, Brazil, Sep. 2007, pp. 1158-1162.
16. A. Dawoud and M. S. Kamel, "Iterative multimodel subimage bina-rization for handwritten character segmentation," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1223-1230, Sep. 2004.
17. H. Cecotti and A. Belaid, "Dynamic filters selection for textual document image binarization," in Proc. Int. Conf. Pattern Recognit., Tampa, FL, Dec. 2008, pp. 1-4.
18. L. Likforman-Sulem, J. Darbon, and E. H. B. Smith, "Pre-processing of degraded printed documents by non-local means and total variation," in Proc. Int. Conf. Document Anal. Recognit., Barcelona, Spain, Jul. 2009, pp. 758-762.
19. Z. Zhou, L. Li, and C. L. Tan, "Edge based binarization for video text images," in Proc. Int. Conf. Pattern Recognit., Istanbul, Turkey, Aug. 2010, pp. 133-136.
20. Y. T. Pai, Y. F. Chang, and S. J. Ruan, "Adaptive thresholding algorithm: Efficient computation technique based on intelligent block detection for degraded document images," Pattern Recognit., vol. 43, no. 9, pp. 3177-3187, 2010.
21. R. F. Moghaddam and M. Cheriet, "A multi-scale framework for adaptive binarization of degraded document images," Pattern Recognit., vol. 43, no. 6, pp. 2186-2198, 2010.
22. ABBYY FineReader OCR. (2011) [Online]. Available: http://finereader. abbyy.com/
23. P. Stathis, E. Kavallieratou, and N. Papamarkos, "An evaluation survey of binarization algorithms on historical document images," in Proc. Int. Conf. Pattern Recognit., Dec. 2008, pp. 1-4.
24. R. Paredes, E. Kavallieratou, and R. D. Lins, "ICFHR 2010 contest: Quantitative evaluation of binarization algorithms," in Proc. Int. Conf. Frontiers Handwrit. Recognit., Kolkata, India, Nov. 2010, pp. 733-736.
25. R. Hedjam, R. F. Moghaddam, and M. Cheriet, "A spatially adaptive statistical method for the binarization of historical manuscripts and degraded document images," Pattern Recognit., vol. 44, no. 9, pp. 2184-2196, 2011.
26. E. H. B. Smith, L. Likforman-Sulem, and J. Darbon, "Effect of preprocessing on binarization," Proc. SPIE, vol. 7534, p. 75340H, Jan. 2010.
27. E. H. B. Smith, "An analysis of binarization ground truthing," in Proc. Int. Workshop Document Anal. Syst., Boston, MA, Jun. 2010, pp. 27-33.
28. B. Bataineh, S. N. H. S. Abdullah, and K. Omar, "An adaptive local binarization method for document images based on a novel thresholding method and dynamic windows," Pattern Recognit. Lett., vol. 32, no. 14, pp. 1805-1813, 2011.
29. M. Sezgin and B. Sankur, "Survey over image thresholding techniques and quantitative performance evaluation," J. Electron. Imag., vol. 13, no. 1, pp. 146-168, 2004.
30. L. Y. Fan, C. L. Tan, and L. X. Fan, "Edge-preserving prefiltering for document image binarization," in Proc. Int. Conf. Image Process., vol. 1. Thessaloniki, Greece, Oct. 2001, pp. 1070-1073. (Pubitemid 33057609)
31. F. Kleber, M. Diem, and R. Sablatnig, "Scale space binarization using edge information weighted by a foreground estimation," in Proc. Int. Conf. Document Anal. Recognit., Beijing, China, Sep. 2011, pp. 1180-1184.
32. C. Bastos, C. Mello, J. Andrade, D. Falcao, M. Lima, W. Santos, and A. Oliveira, "Thresholding images of historical documents with back-to-front interference based on color quantization by genetic algorithms," in Proc. IEEE Int. Conf. Tools Artif. Intell., vol. 1. Patras, Greece, Oct. 2007, pp. 488-491.
33. T. Obafemi-Ajayi, G. Agam, and O. Frieder, "Ensemble LUT classification for degraded document enhancement," Proc. SPIE, vol. 6815, pp. 681509-1-681509-9, Jan. 2008.
34. R. D. Lins and J. M. M. da Silva, "Assessing algorithms to remove back-to-front interference in documents," in Proc. Int. Telecommun. Symp., Fortaleza, Brazil, 2006, pp. 868-873.
35. K. Ntirogiannis, B. Gatos, and I. Pratikakis, "An objective evaluation methodology for document image binarization techniques," in Proc. Int. Workshop Document Anal. Syst., Nara, Japan, Sep. 2008, pp. 217-224.
36. A. Clavelli, D. Karatzas, and J. Llados, "A framework for the assessment of text extraction algorithms on complex colour images," in Proc. Int. Workshop Document Anal. Syst., 2010, pp. 19-26.
37. I. Pratikakis, B. Gatos, and K. Ntirogiannis, "H-DIBCO 2010-handwritten document image binarization competition," in Proc. Int. Conf. Frontiers Handwrit. Recognit., Kolkata, India, Nov. 2010, pp. 727-732.
38. B. Gatos, K. Ntirogiannis, and I. Pratikakis, "DIBCO 2009: Document image binarization contest," Int. J. Document Anal. Recognit., vol. 14, no. 1, pp. 35-44, 2011.
39. I. Pratikakis, B. Gatos, and K. Ntirogiannis, "ICDAR 2011 document image binarization contest-DIBCO 2011," in Proc. Int. Conf. Document Anal. Recognit., Beijing, China, Sep. 2011, pp. 1506-1510.
40. I. B. Messaoud, H. E. Abed, V. Maergner, and H. Amiri, "A design of a preprocessing framework for large database of historical documents," in Proc. Workshop Historical Document Imag. Process., Beijing, China, Sep. 2011, pp. 177-183.
41. S. Lu, B. Su, and C. L. Tan, "Document image binarization using background estimation and stroke edges," Int. J. Document Anal. Recognit., vol. 13, no. 4, pp. 303-314, 2010.
42. H. S. Baird, "Document image defect models," in Proc. IAPR Workshop Syntactic Struct. Pattern Recognit., Murray Hill, NJ, Jun. 1990, pp. 38-46.
43. T. Kanungo, R. M. Haralick, and I. Phillips, "Global and local document degradation models," in Proc. Int. Conf. Document Anal. Recognit., Tsukuba, Japan, Oct. 2003, pp. 730-734.
44. E. H. B. Smith, "Characterization of image degradation caused by scanning," Pattern Recognit. Lett., vol. 19, no. 13, pp. 1191-1197, 1998.
45. P. Sarkar, H. S. Baird, and X. Zhang, "Training on severely degraded text-line images," in Proc. Int. Conf. Document Anal. Recognit., vol. 1. Edinburgh, Scotland, Aug. 2003, pp. 38-43.
46. T. K. Ho and H. S. Baird, "Large-scale simulation studies in image pattern recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 10, pp. 1067-1079, Oct. 1997. (Pubitemid 127762298)
47. E. Badekas and N. Papamarkos, "Automatic evaluation of document binarization results," in Proc. Iberoamer. Congr. Pattern Recognit., Havana, Cuba, Nov. 2005, pp. 1005-1014. (Pubitemid 43943707)
48. H. Lu, A. C. Kot, and Y. Q. Shi, "Distance-reciprocal distortion measure for binary document images," IEEE Signal Process. Lett., vol. 11, no. 2, pp. 228-231, Feb. 2004.
49. D. R. Martin, C. C. Fowlkes, and J. Malik, "Learning to detect natural image boundaries using local brightness, color, and texture cues," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 5, pp. 530-549, May 2004.
50. R. Kasturi, D. Goldgof, P. Soundararajan, V. Manohar, J. Garofolo, R. Bowers, M. Boonstra, V. Korzhova, and J. Zhang, "Framework for performance evaluation of face, text, and vehicle detection and tracking in video: Data, metrics, and protocol," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 319-336, Feb. 2009.
51. K. Ntirogiannis, B. Gatos, and I. Pratikakis, "A modified adaptive logical level binarization technique for historical document images," in Proc. Int. Conf. Document Anal. Recognit., Barcelona, Spain, 2009, pp. 1171-1175.
52. D. K. Reed and E. H. B. Smith, "Correlating degradation models and image quality metrics," Proc. SPIE, vol. 6815, pp. 681508-1-681508-8, Jan. 2008.
53. A. Souza, M. Cheriet, S. Naoi, and C. Y. Suen, "Automatic filter selection using image quality assessment," in Proc. Int. Conf. Document Anal. Recognit., 2003, pp. 508-512.
54. Library of Congress. (2011) [Online]. Available: http://memory. loc.gov/ammem
55. IMPACT Project. (2011) [Online]. Available: http://www.impact-project.eu
56. J. Canny, "A computational approach to edge detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 8, no. 6, pp. 679-698, Nov. 1986.
57. J. Sauvola and M. Pietikainen, "Adaptive document image binarization," Pattern Recognit., vol. 33, no. 2, pp. 225-236, 2000. (Pubitemid 32262190)
58. E. Saund, J. Lind, and P. Sarkar, "Pixlabeler: User interface for pixel-level labeling of elements in document images," in Proc. Int. Conf. Document Anal. Recognit., 2009, pp. 646-650.
59. G. Bal, G. Agam, and O. Frieder, "Interactive degraded document enhancement and ground truth generation," Proc. SPIE, vol. 6815, pp. 68150Z-1-68150Z-9, Mar. 2008.
60. H. J. Lee and B. Chen, "Recognition of handwritten Chinese characters via short line segments," Pattern Recognit., vol. 25, no. 5, pp. 543-552, May 1992.
61. A. Dawoud and M. Kamel, "New approach for the skeletonization of handwritten characters in grey-level images," in Proc. Int. Conf. Document Anal. Recognit., vol. 2. Edinburgh, Scotland, Aug. 2003, pp. 1233-1237.
62. J. van Beusekom, F. Shafait, and T. M. Breuel, "Automated OCR ground truth generation," in Proc. Int. Workshop Document Anal. Syst., Nara, Japan, Sep. 2008, pp. 111-117.
63. J. Zhang, D. Goldgof, and R. Kasturi, "A new edge-based text verification approach for video," in Proc. Int. Conf. Pattern Recognit., Tampa, FL, Dec. 2008, pp. 1-4.
64. J. Bernsen, "Dynamic thresholding of grey-level images," in Proc. Int. Conf. Pattern Recognit., 1986, pp. 1251-1255.
65. W. Niblack, An Introduction to Digital Image Processing. Englewood Cliffs, NJ: Prentice-Hall, 1986, pp. 115-116.
66. N. Otsu, "A threshold selection method from gray-level histograms," IEEE Trans. Syst., Man, Cybern., vol. 9, no. 1, pp. 62-66, Jan. 1979. (Pubitemid 9413341)
67. S. V. Rice, "Measuring the accuracy of page-reading systems," Ph.D. dissertation, Dept. Comput. Sci., Univ. Nevada, Las Vegas, 1996.
68. M. Kendall, "A new measure of rank correlation," Biometrika, vol. 30, nos. 1-2, pp. 81-89, 1938.
69. J. Bai, Y. Q. Yang, and R. L. Tian, "Complicated image's binarization based on method of maximum variance," in Proc. Int. Conf. Mach. Learn. Cybern., Dalian, China, Aug. 2006, pp. 3782-3785. (Pubitemid 46430991)
70. M. Pan, F. Zhang, and H. Ling, "An image binarization method based on HVS," in Proc. IEEE Int. Conf. Multimedia Expo, Beijing, China, Dec. 2007, pp. 1283-1286.