A Comparison of Text-Based Methods for Detecting Duplication in Scanned Document Databases

Information Retrieval

Volumn 4, Issue 2, 2001, Pages 153-173

  Lopresti, Daniel P ^a  

^a LUCENT TECHNOLOGIES   (United States)

Author keywords

Approximate string matching; Document analysis; Duplicate detection; Information retrieval; Optical character recognition

Indexed keywords

EID: 0043238076     PISSN: 13864564     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1011471129047     Document Type: Review

References (22)

1. Doermann D, Li H and Kia O (1997) The detection of duplicates in document image databases. In: Proceedings of the Fourth International Conference on Document Analysis and Recognition, Ulm, Germany, (1994) pp. 314-318.
2. Esakov J, Lopresti DP, Sandberg JS and Zhou J (1994) Issues in automatic OCR error classification. In: Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval, Las Vegas, NV, pp. 401-412.
3. Gilmore GG (1997) Former Army operations officers assist DoD's search. Army Link News, http://www. dtic.mil/armylink/news/Dec1997/a1997I209moredata.html.
4. Hull JJ (1998) Document image similarity and equivalence detection. International Journal on Document Analysis and Recognition, 1(1):37-42.
5. Hull JJ, Cullen J and Peairs M (1997) Document image matching and retrieval techniques. In: Proceedings of the Symposium on Document Image Understanding Technology, Annapolis, MD, April-May, pp. 31-35.
6. Kantor PB and Voorhees E (1996) Report on the TREC-5 confusion track. In NIST Special Publication 500-238: The Fifth Text Retrieval Conference (TREC-5), pp. 65-74. http://trec.nist.gov/pubs/ trec5/t5_proceedings.html.
7. Kantor PB and Voorhees EM (2000) The TREC-5 confusion track: Comparing retrieval methods for scanned text. Information Retrieval, 2(2/3):165-176.
8. Lee D-S and Hull JJ (1999). Duplicate detection for symbolically compressed documents. In: Proceedings of the Fifth International Conference on Document Analysis and Recognition, Bangalore, India, pp. 305-308.
9. Lopresti DP (1999a) Models and algorithms for duplicate document detection. In: Proceedings of the Fifth International Conference on Document Analysis and Recognition, Bangalore, India, pp. 297-300
10. Lopresti DP (1999b) String techniques for duplicate document detection. In: Proceedings of the Symposium on Document Image Understanding Technology, Annapolis, MD, pp. 101-112.
11. Lopresti DP (2000a). A comparison of text-based methods for detecting duplication in document image databases. In: Proceedings of Document Recognition and Retrieval VII (IS&T/SPIE Electronic Imaging), Vol. 3967, San Jose, CA, pp. 210-221.
12. Lopresti DP (2000b) String techniques for detecting duplicates in document databases. International Journal on Document Analysis and Recognition, 2(4):186-199.
13. Prokoski F (1999) Database partitioning and duplicate document detection based on optical correlation. In: Proceedings of the Symposium on Document Image Understanding Technology, Annapolis, MD, pp. 86-97.
14. Rogers R, Chalana V, Marchisio G, Nguyen T and Bruce A (1999) Duplicate document detection in DocBrowse. In: Proceedings of the Symposium on Document Image Understanding Technology, Annapolis, MD, pp. 119-127.
15. Salton G and Allan J (1997) Text retrieval using the vector processing model. In: Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval, Las Vegas. NV, pp. 9-22.
16. Salton G, Wong A and Yang C (1975) A vector space model for information retrieval. Communications of the Association for Computing Machinery, 18(11):613-620.
17. Shivakumar N and Garcia-Molina H (1995) SCAM: A copy detection mechanism for digital documents. In: Proceedings of the Second International Conference on Theory and Practice of Digital Libraries, Austin, TX http://www.csdl.tamu.edu/DL95/papers/shivakumar.ps.
18. Shivakumar N and Garcia-Molina H (1998) Finding near-replicas of documents on the Web. In: Proceedings of the Workshop on Web Databases.
19. Smith TF and Waterman MS (1981) Identification of common molecular sequences. Journal of Molecular Biology, 147:195-197.
20. Spitz AL (1997) Duplicate document detection. In: Proceedings of Document Recognition IV (IS&T/SPIE Electronic Imaging), San Jose, CA, pp. 88-94.
21. Taghva K, Borsack J, Condit A and Inaparthy P (1995) Effects of OCR errors on short documents. In: Annual Report of UNLV Information Science Research Institute, Las Vegas, NV, pp. 99-105.
22. Wagner RA and Fischer MJ (1974) The string-to-string correction problem. Journal of the Association for Computing Machinery, 21:168-173.