Blocking blog spam with language model disagreement

Proceedings of the 1st International Workshop on Adversarial Information Retrieval on the Web, AIRWeb 2005 - Held in Conjunction with the 14th International World Wide Web Conference

Volumn , Issue , 2005, Pages 1-6

  Mishne, Gilad ^a   Carmel, David ^b   Lempel, Ronny ^b  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b IBM HAIFA RESEARCH LAB   (Israel)

Author keywords

Blogs; Comment spam; Language models

Indexed keywords

COMMENT SPAM; LANGUAGE MODEL; RULE SET; SPAM FILTERING;

COMPUTATIONAL LINGUISTICS; INFORMATION RETRIEVAL; WORLD WIDE WEB;

BLOGS;

EID: 36448955121     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. J. Allan, C. Wade, and A. Bolivar. Retrieval and novelty detection at the sentence level. In SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 314-321. ACM Press, 2003.
2. E. Amitay, D. Carmel, A. Darlow, R. Lempel, and A. Soffer. The connectivity sonar: detecting site functionality by structural patterns. In HYPERTEXT '03: Proceedings of the fourteenth ACM conference on Hypertext and hypermedia, pages 38-47. ACM Press, 2003.
3. I. Androutsopoulos, J. Koutsias, K. V. Chandrinos, and C. D. Spyropoulos. An experimental comparison of naive bayesian and keyword-based anti-spam filtering with personal e-mail messages. In SIGIR '00: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 160-167. ACM Press, 2000.
4. Nofollow tag cheers bloggers, but fails blogs, URL: http://www.platinax. co.uk/news/archives/2005/01/new-nofollow-ta.html.
5. R. Baeza-Yates and E. Davis. Web page ranking using link attributes. In WWW Alt. '04: Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, pages 328-329. ACM Press, 2004.
6. B. Davison. Recognizing nepotistic links on the web. In AAAI-2000 workshop on Artificial Intelligence for Web Search, pages 23-28. AAAI Press, 2000.
7. The Berkeley/Stanford Web Term Document Frequency and Rank project, URL: http://elib.cs.berkeley.edu/docfreq/.
8. D. Fetterly, M. Manasse, and M. Najork. Spam, damn spam, and statistics: using statistical analysis to locate spam web pages. In WebDB '04: Proceedings of the 7th International Workshop on the Web and Databases, pages 1-6. ACM Press, 2004.
9. M. R. Henzinger, R. Motwani, and C. Silverstein. Challenges in web search engines. SIGIR Forum, 36(2):11-22, 2002.
10. J. M. G. Hidalgo. Evaluating cost-sensitive unsolicited bulk email categorization. In SAC '02: Proceedings of the 2002 ACM symposium on Applied computing, pages 615-620. ACM Press, 2002.
11. V. Lavrenko and W. B. Croft. Relevance based language models. In SIGIR '01: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages 120-127. ACM Press, 2001.
12. Movable Type Blacklist Filter, with content filtering, URL: http://www.jayallen.org/projects/mt-blacklist/.
13. Joint statement from Yahoo, Google, and others regarding the "nofollow" tag, URLs: http://www.google.com/googleblog/2005/01/ preventing-comment-spam.html, http://www.ysearchblog.com/archives/000069.html.
14. No nofollow: fight spam, not blogs, URL: http://www.nonofollow.net.
15. J. M. Ponte and W. B. Croft. A language modeling approach to information retrieval. In SIGIR '98: Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pages 275-281. ACM Press, 1998.
16. M. Sahami, S. Dumais, D. Heckerman, and E. Horvitz. A bayesian approach to filtering junk E-mail. In Learning for Text Categorization: Papers from the 1998 Workshop, Madison, Wisconsin, 1998. AAAI Technical Report WS-98-05.
17. L. von Ahn, M. Blum, and J. Langford. Telling humans and computers apart automatically. Commun. ACM, 47(2):56-60, 2004.