A general framework for the evaluation of symbol recognition methods

International Journal on Document Analysis and Recognition

Volumn 9, Issue 1, 2007, Pages 59-74

  Valveny, Ernest ^a   Dosch, Philippe ^b   Winstanley, Adam ^c   Zhou, Yu ^c   Yang, Su ^d   Yan, Luo ^e   Wenyin, Liu ^e   Elliman, Dave ^f   Delalandre, Mathieu ^h   Trupin, Eric ^g   Adam, Sébastien ^g   Ogier, Jean Marc ^h  

^a Barcelona Perception Computing Lab   (Spain)

^b LORIA   (France)

^c NATIONAL UNIVERSITY OF IRELAND   (Ireland)

^d FUDAN UNIVERSITY   (China)

^e CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^f UNIVERSITY OF NOTTINGHAM   (United Kingdom)

^g UNIVERSITÉ DE ROUEN   (France)

^h UNIVERSITÉ DE LA ROCHELLE   (France)

Author keywords

Performance evaluation; Symbol recognition

Indexed keywords

COMPUTER GRAPHICS; GRAPHIC METHODS;

EVALUATION METRICS; PERFORMANCE EVALUATION;

OBJECT RECOGNITION;

EID: 33847319755     PISSN: 14332833     EISSN: 14332825     Source Type: Journal    
DOI: 10.1007/s10032-006-0033-x     Document Type: Article

References (29)

1. Aksoy, S., Ye, M., Schauf, M., Song, M., Wang, Y., Haralick, R., Parker, J., Pivovarov, J., Royko, D., Sun, C., Algorithm performance contest. In: Proceedings of 15th International Conference on Pattern Recognition, vol. 4, pp. 870-876, Barcelona, Spain (2000)
2. Antonacopoulos, A., Gatos, B., Karatzas, D.: ICDAR 2003 page segmentation competition. In: Proceedings of 7th International Conference on Document Analysis and Recognition, Edinburgh (Scotland, UK), pp. 688-689 (2003)
3. Baird, H.S.: The state of the art of document image degradation modeling. In: Proceedings of 4th IAPR International Workshop on Document Analysis Systems, Rio de Janeiro (Brazil) (2000)
4. Chhabra A., Phillips I.T. (1998) The 2nd international graphics recognition contest-raster to vector conversion: A report. In: Tombre K., Chhabra A.K. (eds) Graphics Recognition - Algorithms and Systems. Lecture Notes in Computer Science, vol. 1389. Springer, Berlin Heidelberg New York, pp. 390-410
5. Chhabra A.K. (1998) Graphic symbol recognition: An overview. In: Tombre K., Chhabra A.K. (eds) Graphics Recognition - Algorithms and Systems. Lecture Notes in Computer Science, vol. 1389. Springer, Berlin Heidelberg New York, pp. 68-79
6. Clark A.F., Courtney P. (2000) Databases for performance characterization. In: Stiehl H.H., Viergever M.A., Vincken K.L. (eds) Performance Characterization in Computer Vision. Kluwer, Dordrecht
7. Cootes T.F., Taylor C.J., Cooper D.H., Graham J. (1995) Active shape models: Their training and application. Comput. Vis. Image Underst. 61(1): 38-59
8. Cordella L.P., Vento M. (2000) Symbol recognition in documents: A collection of techniques? Int. J. Doc. Anal. Recognit. 3(2): 73-88
9. Courtney, P., Thacker, N.A.: Performance characterization in computer vision: The role of statistics in testing and design. In: Blanc-Talon, J., Popescu, D.C. (eds.) Imaging and Vision Systems: Theory, Assessment and Applications. NOVA Science, Hungtington, NY (2003)
10. Delalandre M., Trupin E., Ogier J., Labiche J. (2005) Contextual system of symbol structural recognition based on an object-process methodology. Electron. Lett. Comput. Vis. Image Anal. 5(2): 16-29
11. Ghosh D., Shivaprasad A.P. (1999) An analytic approach for generation of artificial hand-printed character database from given generative models. Pattern Recognit. 32, 907-920
12. Guyon I., Haralick R.M., Hull J.J., Phipliops I.T. (1997) Data sets for OCR and document image understanding research. In: Bunke H., Wang P.S.P. (eds) Handbook of Character Recognition and Document Image Analysis. World Scientific, Singapore, pp. 779-800
13. Haralick R. (1992) Performance characterization in image analysis: thinning, a case in point. Pattern Recognit. Lett. 13, 5-12
14. Hilaire, X.: A matching scheme to enhance performance evaluation of raster-to-vector conversion algorithms. In: Proceedings of 7th International Conference on Document Analysis and Recognition, vol. 1, pp. 629-633. Edinburgh, Scotland (2003)
15. Kanungo, T., Haralick, R.M., Baird, H.S., Stuetzle, W., Madigan, D.: Document degradation models: Parameter estimation and model validation. In: Proceedings of IAPR Workshop on Machine Vision Applications, Kawasaki (Japan), pp. 552-557 (1994)
16. Kanungo T., Haralick R.M., Baird H.S., Stuezle W., Madigan D. (2000) A statistical, nonparametric methodology for document degradation model validation. IEEE Trans. Pattern Anal. Mach. Intell. 22(11): 1209-1223
17. Lladós J., Valveny E., Sánchez G., Martí, E. (2002) Symbol recognition: Current advances and perspectives. In: Kwon Y.-B. (eds) Graphics Recognition - Algorithms and Applications. Lecture Notes in Computer Science, vol. 2390. Springer, Berlin Heidelberg New York, pp. 104-127
18. Lopresti D., Nagy G. (2002) Issues in ground-truthing graphic documents. In: Blostein D., Kwon Y.-B. (eds) Graphics Recognition - Algorithms and Applications. Lecture Notes in Computer Science, vol. 2390. Springer, Berlin Heidelberg New York, pp. 46-66
19. Lucas S.M., Panaretos A., Sosa L., Tang A., Wong S., Young R., Ashida K., Nagai H., Okamoto M., Yamamoto H., Miyao H., Zhu J., Ou W., Wolf C., Jolion J.M., Todoran L., Worring M., Lin X. (2005) ICDAR 2003 robust reading competitions: Entries, results, and future directions. Int. J. Doc. Anal. Recognit. 7(2-3): 105-122
20. Mariano, V.Y., Min, J., Park, J.-H., Kasturi, R., Mihalcik, D., Li, H., Doermann, D., Drayer, T.: Performance evaluation of object detection algorithms. In: Proceedings of the 16th International Conference on Pattern Recognition, Quebec (Canada), vol. 3, pp. 965-969 (2002)
21. Philips P.J., Moon H., Rizvi S.A., Rauss P.J. (2000) The evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10): 1090-1104
22. Phillips I.T., Chhabra A.K. (1999) Empirical performance evaluation of graphics recognition systems. IEEE Trans. Pattern Anal. Mach. Intell. 21(9): 849-870
23. Tombre K., Chhabra A.K. (eds) (1998) Graphics Recognition - Algorithms and Systems. Lecture Notes in Computer Science, vol. 1389. Springer, Berlin Heidelberg New York
24. Valveny E., Dosch Ph. (2004) Performance evaluation of symbol recognition. In: Marinai S., Dengel A. (eds) Document Analysis Systems VI - Proceedings of 6th IAPR International Workshop on Document Analysis Systems, Florence (Italy). Lecture Notes in Computer Science, vol. 3163. Springer, Berlin Heidelberg New York, pp. 354-365
25. Valveny, E., Dosch, PhA: Symbol recognition contest: a synthesis. In: Selected Papers from 5th International Workshop on Graphics Recognition, GREC'03. Lecture Notes in Computer Science, vol. 3088, pp. 368-385. Springer, Berlin Heidelberg New York (2004)
26. Wenyin L., Dori D. (1997) A protocol for performance evaluation of line detection algorithms. Mach. Vis. Appl. 9, 240-250
27. Wenyin L., Zhai J., Dori D. (2002) Extended summary of the arc segmentation contest. In: Blostein D., Kwon Y.B. (eds) Graphics Recognition: Algorithms and Applications, Selected Papers from 4th International Workshop on Graphics Recognition, GREC'01. Lecture Notes in Computer Science, vol. 2390. Springer, Berlin Heidelberg New York, pp. 343-349
28. Wilson, C.L., Geist, J., Garris, M.D., Chellappa, R.: Design, integration and evaluation of form-based handprint and OCR systems. Technical report, National Institute of Standards and Technology, Technical Report NISTIR 5932 (1996)
29. Zhang Y.J. (1996) A survey on evaluation methods for image segmentation. Pattern Recognit. 29(8): 1335-1346