Text reuse detection using a composition of text similarity measures

24th International Conference on Computational Linguistics - Proceedings of COLING 2012: Technical Papers

Volumn , Issue , 2012, Pages 167-184

  Bär, Daniel ^a   Zesch, Torsten ^a,b   Gurevych, Iryna ^a,b  

^a DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^b German Institute for Educational Research and Educational Information   (Germany)

Author keywords

Paraphrase; Plagiarism; Text reuse; Text similarity

Indexed keywords

DIVERSE FEATURES; PARAPHRASE; PLAGIARISM; PLAGIARISM DETECTION; SOURCE TEXT; STANDARD EVALUATIONS; TEXT REUSE; TEXT SIMILARITY;

COMPUTATIONAL LINGUISTICS;

INTELLECTUAL PROPERTY;

EID: 84876813480     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (58)

1. Abdel-Hamid, O., Behzadi, B., Christoph, S., and Henzinger, M. (2009). Detecting the Origin of Text Segments Efficiently. In Proceedings of the 18th International Conference on World Wide Web, pages 61-70, Madrid, Spain.
2. Allison, L. and Dix, T. I. (1986). A bit-string longest-common- subsequence algorithm. Information Processing Letters, 23:305-310.
3. Artstein, R. and Poesio, M. (2008). Inter-Coder Agreement for Computational Linguistics. Computational Linguistics, 34(4):555-596.
4. Bär, D., Zesch, T., and Gurevych, I. (2011). A Reflective View on Text Similarity. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, pages 515-520, Hissar, Bulgaria.
5. Barrón-Cedeño, A., Rosso, P., Agirre, E., and Labaka, G. (2010). Plagiarism Detection across Distant Language Pairs. In Proceedings of the 23rd International Conference on Computational Linguistics, pages 37-45, Beijing, China.
6. Boonthum, C. (2004). iSTART: Paraphrase Recognition. In Proceedings of the 42nd Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 31-36, Barcelona, Spain.
7. Broder, A. Z. (1997). On the resemblance and containment of documents. Proceedings of Compression and Complexity of Sequences, pages 21-29.
8. Broder, A. Z., Glassman, S. C., Manasse, M. S., and Zweig, G. (1997). Syntactic clustering of the Web. In Proceedings of the 6th International World Wide Web Conference, pages 1157-1166, Santa Clara, CA, USA.
9. Burrows, S., Potthast, M., and Stein, B. (2012). Paraphrase Acquisition via Crowdsourcing and Machine Learning. Transactions on Intelligent Systems and Technology, V(January):1-22.
10. Callison-Burch, C. and Dredze, M. (2010). Creating Speech and Language Data With Amazon's Mechanical Turk. In Proceedings of the NAACL HLT Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, pages 1-12, Los Angeles, CA, USA.
11. Carletta, J. (1996). Assessing Agreement on Classification Tasks: The Kappa Statistic. Computational Linguistics, 22(2):249-254.
12. Charikar, M. S. (2002). Similarity Estimation Techniques from Rounding Algorithms. In Proceedings of the 34th Annual Symposium on Theory of Computing, pages 380-388, Montreal, Canada.
13. Chong, M., Specia, L., and Mitkov, R. (2010). Using Natural Language Processing for Automatic Detection of Plagiarism. In Proceedings of the 4th International Plagiarism Conference, Newcastle upon Tyne, UK.
14. Clough, P., Gaizauskas, R., Piao, S. S., and Wilks, Y. (2002). METER: MEasuring TExt Reuse. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 152-159, Philadelphia, PA, USA.
15. Clough, P. and Stevenson, M. (2011). Developing a Corpus of Plagiarised Short Answers. Language Resources and Evaluation: Special Issue on Plagiarism and Authorship Analysis, 45(1):5- 24.
16. Dinu, L. P. and Popescu, M. (2009). Ordinal measures in authorship identification. In Proceedings of the 3rd PAN Workshop. Uncovering Plagiarism, Authorship and Social Software Misuse, pages 62-66, San Sebastian, Spain.
17. Fellbaum, C. (1998). Word Net: An Electronic Lexical Database. MIT Press.
18. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378-382.
19. Gabrilovich, E. and Markovitch, S. (2007). Computing Semantic Relatedness using Wikipediabased Explicit Semantic Analysis. In Proceedings of the 20th International Joint Conference on Artificial Intelligence, pages 1606-1611, Hyderabad, India.
20. Gaizauskas, R., Foster, J., Wilks, Y., Arundel, J., Clough, P., and Piao, S. (2001). The METER Corpus: A corpus for analysing journalistic text reuse. In Proceedings of the Corpus Linguistics 2001 Conference, pages 214-223.
21. Gärdenfors, P. (2000). Conceptual Spaces: The Geometry of Thought. MIT Press.
22. Goodman, N. (1972). Seven strictures on similarity. In Goodman, N., editor, Problems and projects, pages 437-446. Bobbs-Merrill.
23. Gusfield, D. (1997). Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. Cambridge University Press.
24. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. (2009). The WEKA Data Mining Software: An Update. SIGKDD Explorations, 11(1):10-18.
25. Hatzivassiloglou, V., Klavans, J. L., and Eskin, E. (1999). Detecting text similarity over short passages: Exploring linguistic feature combinations via machine learning. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 203-212, College Park, MD, USA.
26. Henzinger, M. (2006). Finding Near-Duplicate Web Pages: A Large-Scale Evaluation of Algorithms. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 284-291, Seattle, WA, USA.
27. Hoad, T. C. and Zobel, J. (2003). Methods for identifying versioned and plagiarized documents. Journal of the American Society of Information Science and Technology, 54(3):203-215.
28. Jaro, M. A. (1989). Advances in record linkage methodology as applied to the 1985 census of Tampa Florida. Journal of the American Statistical Association, 84(406):414-420.
29. Jiang, J. J. and Conrath, D. W. (1997). Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of the 10th International Conference on Research in Computational Linguistics.
30. Keselj, V., Peng, F., Cercone, N., and Thomas, C. (2003). N-gram-based author profiles for authorship attribution. In Proceedings of the Conference of the Pacific Association for Computational Linguistics, pages 255-264.
31. Koehn, P. (2005). Europarl: A Parallel Corpus for Statistical Machine Translation. In Proceedings of the 10th Machine Translation Summit, pages 79-86, Phuket Island, Thailand.
32. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., and Herbst, E. (2007). Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pages 177-180, Prague, Czech Republic.
33. Landauer, T. K., Foltz, P. W., and Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2):259-284.
34. Landis, J. R. and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1):159-174.
35. Lee, J. (2007). A Computational Model of Text Reuse in Ancient Literary Texts. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 472-479.
36. Leuf, B. and Cunningham, W. (2001). The Wiki Way: Collaboration and Sharing on the Internet. Addison-Wesley Professional.
37. Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8):707-710.
38. Lin, D. (1998). An information-theoretic definition of similarity. In Proceedings of International Conference on Machine Learning, pages 296-304.
39. Lyon, C., Barrett, R., and Malcolm, J. (2004). A theoretical basis to the automated detection of copying between texts, and its practical implementation in the Ferret plagiarism and collusion detector. In In Plagiarism: Prevention, Practice and Policies Conference.
40. Lyon, C., Malcolm, J., and Dickerson, B. (2001). Detecting short passages of similar text in large document collections. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pages 118-125.
41. Manku, G. S., Jain, A., and Sarma, A. D. (2007). Detecting Near-Duplicates for Web Crawling. In Proceedings of the 16th International World Wide Web Conference, pages 141-149, Banff, AB, Canada.
42. McCarthy, P. M. and Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior research methods, 42(2):381- 392.
43. Mihalcea, R., Corley, C., and Strapparava, C. (2006). Corpus-based and Knowledge-based Measures of Text Semantic Similarity. In Proceedings of the 21st National Conference on Artificial Intelligence, pages 775-780, Boston, MA, USA.
44. Monge, A. and Elkan, C. (1997). An efficient domain-independent algorithm for detecting approximately duplicate database records. In Proceedings of the SIGMOD Workshop on Data Mining and Knowledge Discovery, pages 23-29, Tucson, AZ, USA.
45. Mosteller, F. and Wallace, D. L. (1964). Inference and disputed authorship: The Federalist. Addison-Wesley.
46. Potthast, M., Barrón-Cedeño, A., Eiselt, A., Stein, B., and Rosso, P. (2010). Overview of the 2nd International Competition on Plagiarism Detection. In Notebook Papers of CLEF 10 Labs and Workshops, Padua, Italy.
47. Resnik, P. (1995). Using Information Content to Evaluate Semantic Similarity in a Taxonomy. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448-453, Montreal, Canada.
48. Salton, G. and McGill, M. J. (1983). Introduction to Modern Information Retrieval. McGraw-Hill.
49. Sánchez-Vega, F., Villaseñor-Pineda, L., Montes-y-Gómez, M., and Rosso, P. (2010). Towards Document Plagiarism Detection Based on the Relevance and Fragmentation of the Reused Text. In Proceedings of the 9th Mexican International Conference on Artificial Intelligence, pages 24-31, Pachuca, Mexico.
50. Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19(3):321-325.
51. Shannon, C. E. (1951). Prediction and Entropy of Printed English. Bell System Technical Journal, 30:50-64.
52. Spärck Jones, K. (1972). A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28(1):11-21.
53. Stamatatos, E. (2011). Plagiarism detection using stopword n-grams. Journal of the American Society for Information Science and Technology, 62(12):2512-2527.
54. Templin, M. C. (1957). Certain language skills in children. University of Minnesota Press.
55. Tversky, A. (1977). Features of Similarity. In Psychological Review, volume 84, pages 327-352.
56. Winkler, W. E. (1990). String Comparator Metrics and Enhanced Decision Rules in the Fellegi- Sunter Model of Record Linkage. In Proceedings of the Section on Survey Research Methods, pages 354-359.
57. Wise, M. J. (1996). YAP3: Improved detection of similarities in computer program and other texts. In Proceedings of the 27th SIGCSE technical symposium on Computer science education, pages 130-134, Philadelphia, PA, USA.
58. Yule, G. U. (1939). On sentence-length as a statistical characteristic of style in prose: With application to two cases of disputed authorship. Biometrika, 30(3/4):363-390.