An open architecture for end-to-end document analysis benchmarking

Proceedings of the International Conference on Document Analysis and Recognition, ICDAR

Volumn , Issue , 2011, Pages 42-47

  Lamiroy, Bart ^a   Lopresti, Daniel ^b  

^a LORIA   (France)

^b Lehigh University   (United States)

Author keywords

benchmark; document analysis; performance evaluation; web services

Indexed keywords

ANALYSIS PROCESS; BENCHMARK; COARSE-GRAINED; DOCUMENT ANALYSIS; END-TO-END TASK; FULLY OPERATIONAL; OPEN ARCHITECTURE; PERFORMANCE EVALUATION; PLUG-INS; PROGRAMMING LANGUAGE; RUNNING SYSTEMS; STATE OF THE ART;

ALGORITHMS; BENCHMARKING; COMPUTER ARCHITECTURE; DATA STRUCTURES; FLEXIBLE STRUCTURES;

WEB SERVICES;

EID: 80054776619     PISSN: 15205363     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDAR.2011.18     Document Type: Conference Paper

References (22)

1. T. Kanungo, H. S. Baird, and R. M. Haralick, "Special issue on "performance evaluation: Theory, practice, and impact","IJDAR, vol. 4, no. 3, p. 139, 2002.
2. G. Nagy, "Document systems analysis: Testing, testing, testing,"in DAS 2010, Proceedings of the Ninth IAPR International Workshop on Document Analysis Systems, D. Doerman, V. Govindaraju, D. Lopresti, and P. Natarajan, Eds., 2010, p. 1.
3. E. Valveny, P. Dosch, A. C. Winstanley, Y. Zhou, S. Yang, L. Yan, W. Liu, D. Elliman, M. Delalandre, É. Trupin, S. Adam, and J.-M. Ogier, "A general framework for the evaluation of symbol recognition methods," IJDAR, vol. 9, no. 1, pp. 59-74, 2007. (Pubitemid 46321677)
4. I. T. Phillips, S. Chen, and R. M. Haralick, "CD-ROM document database standard," in Document image analysis. Los Alamitos, CA, USA: IEEE Computer Society Press, 1995, pp. 198-203.
5. G. Thoma, "Automating the production of bibliographic records for MEDLINE," National Library of Medicine, Bethesda, MD USA, Tech. Rep., September 2001.
6. G. Agam, S. Argamon, O. Frieder, D. Grossman, and D. Lewis, The Complex Document Image Processing (CDIP) test collection, Illinois Institute of Technology, 2006.
7. T. M. Breuel, "The ocropus open source ocr system," in DRR, ser. SPIE Proceedings, B. A. Yanikoglu and K. Berkner, Eds., vol. 6815. SPIE, 2008, p. 68150.
8. J. Rendek, G. Masini, P. Dosch, and K. Tombre, "The search for genericity in graphics recognition applications: Design issues of the qgar software system," in Document Analysis Systems, ser. Lecture Notes in Computer Science, S. Marinai and A. Dengel, Eds., vol. 3163. Springer, 2004, pp. 366-377.
9. Stefan Jaeger, Guangyu Zhu, David Doermann, Kevin Chen, and Summit Sampat, "DOCLIB: a Software Library for Document Processing," in International Conference on Document Recognition and Retrieval XIII. San Jose, CA, 2006, pp. 1-9.
10. David Doermann, Elena Zotkina, and Huiping Li, "GEDI -A Groundtruthing Environment for Document Images," Ninth IAPR International Workshop on Document Analysis Systems (DAS 2010), 2010.
11. D. Lopresti and B. Lamiroy, "Document Analysis Research in the Year 2021," in Twenty-fourth International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems (IEA/AIE 2011), ser. LNCS, Syracuse University. Syracuse, NY, United States: Springer, Jul. 2011.
12. B. Lamiroy and D. Lopresti, "A platform for storing, visualizing, and interpreting collections of noisy documents,"in Fourth Workshop on Analytics for Noisy Unstructured Text Data - AND'10, ser. ACM International Conference Proceeding Series, IAPR. Toronto Canada: ACM, October 2010.
13. B. Lamiroy, D. Lopresti, H. Korth, and H. Jeff, "How carefully designed open resource sharing can help and expand document analysis research," in Document Recognition and Retrieval XVIII, ser. SPIE Proceedings, G. Agam and C. Viard-Gaudin, Eds., vol. 7874. San Francisco, CA USA: SPIE, January 2011.
14. K. R. Popper, The Logic of Scientific Discovery, reprint ed. Routledge, October 1992,
15. original edition, 1934 "Logik der Forschung".
16. B. M. E. Moret and H. D. Shapiro, "Algorithms and experiments: The new (and old) methodology," Journal of Universal Computer Science, vol. 7, no. 5, pp. 434-446, 2001.
17. M. Schwab, M. Karrenbach, and J. Claerbout, "Making scientific computations reproducible," Computing in Science and Engg., vol. 2, pp. 61-67, November 2000.
18. A. F. Smeaton, P. Over, and W. Kraaij, "Evaluation campaigns and TRECVid," in MIR '06: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval. New York, NY, USA: ACM Press, 2006, pp. 321-330.
19. D. Booth and C. K. Liu, "Web services description language (WSDL) version 2.0 part 0: Primer," W3C, Candidate Recommendation, Mar. 2006.
20. T. Oinn, M. Greenwood, M. Addis, N. Alpdemir, J. Ferris, K. Glover, C. Goble, A. Goderis, D. Hull, D. Marvin, P. Li, P. Lord, M. Pocock, M. Senger, R. Stevens, A. Wipat, and C. Wroe, "Taverna: lessons in creating a workflow environment for the life sciences," Concurrency and Computation: Practice and Experience, vol. 18, no. 10, pp. 1067-1100, August 2006. (Pubitemid 44258498)
21. J. Watson, "Virtualbox: bits and bytes masquerading as machines,"Linux Journal, vol. 2008, February 2008.
22. B. Lamiroy, D. Lopresti, and T. Sun, "Document Analysis Algorithm Contributions in End-to-End Applications," in 11th International Conference on Document Analysis and Recognition - ICDAR 2011. Beijing, China: International Association for Pattern Recognition, Sep. 2011.