Identifying web spam with the wisdom of the crowds

ACM Transactions on the Web

Volumn 6, Issue 1, 2012, Pages

  Liu, Yiqun ^a   Chen, Fei ^a   Kong, Weize ^a   Yu, Huijia ^a   Zhang, Min ^a   Ma, Shaoping ^a   Ru, Liyun ^a  

^a TSINGHUA NATIONAL LABORATORY FOR INFORMATION SCIENCE AND TECHNOLOGY   (China)

Author keywords

Experimentation; Human Factors; Measurement

Indexed keywords

DETECTION FRAMEWORK; EXPERIMENTATION; LEARNING SCHEMES; WEB ACCESS; WEB ACCESS LOGS;

EXPERIMENTS; HUMAN ENGINEERING; MEASUREMENTS; SEARCH ENGINES; WORLD WIDE WEB;

BEHAVIORAL RESEARCH;

EID: 84859400554     PISSN: 15591131     EISSN: 1559114X     Source Type: Journal    
DOI: 10.1145/2109205.2109207     Document Type: Article

References (42)

1. ABERNETHY, J., CHAPELLE, O., AND CASTILLO, C. 2008. WITCH: A new approach to Web spam detection. Yahoo! Res. rep. no. YR-2008-001.
2. AGICHTEIN, E., BRILL, E., AND DUMAISM, S. 2006. Improving Web search ranking by incorporating user behavior information. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'06). ACM, New York, NY, 19-26.
3. AMITAY, E., CARMEL, D., DARLOW, A., LEMPEL, R., AND SOFFER, A. 2003. The connectivity sonar: Detecting site functionality by structural patterns. In Proceedings of the 14th ACM Conference on Hypertext and Hypermedia. ACM, New York, NY, 38-47.
4. BACARELLA, V.,GIANNOTTI, F., NANNI, M., AND PEDRESCHI, D. 2004. Discovery of adsWeb hosts through traffic data analysis. In Proceedings of the 9th ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery. ACM, New York, NY, 76-81.
5. BECCHETTI, L., CASTILLO, C., DONATO, D., LEONARDI, S., AND BAEZA-YATES, R. 2006. Using rank propagation and probabilistic counting for link-based spam detection. In Proceedings of the Workshop on Web Mining and Web Usage Analysis.
6. BILENKO, M. AND WHITE, R. W. 2008. Mining the search trails of surfing crowds: Identifying relevant websites from user activity. In Proceeding of the 17th International World Wide Web Conference. ACM, New York, NY, 51-60.
7. BRIN, S. AND PAGE, L. 1998. The anatomy of a large-scale hypertextual Web search engine. In Proceedings of the 7th International World Wide Web Conference. 107-117.
8. BUEHRER, G., STOKES, J. W., AND CHELLAPILLA, K. 2008. A large-scale study of automated web search traffic. In Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web (AIRWeb'08). ACM, New York, NY, 1-8.
9. CAI, D., YU, S.,WEN, J., AND MA, W. 2004. Block-based web search. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'04). ACM, New York, NY, 456-463.
10. CASTILLO, C. AND DAVISON, B. 2011. Adversarial Web search. Found. Trends Inform. Retrieval 4, 5, 377-486.
11. CASTILLO, C., CORSI, C., DONATO, D., FERRAGINA, P., AND GIONIS, A. 2008. Query-log mining for detecting spam. In Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web (AIRWeb'08). ACM, New York, NY, 17-20.
12. CHELLAPILLA, K. AND CHICKERING, D. M. 2006. Improving cloaking detection using search query popularity and monetizability. In Proceedings of the 2nd International Workshop on Adversarial Information Retrieval on the Web. 17-24.
13. CNNIC (CHINA INTERNET NETWORK INFORMATION CENTER). 2009. Search engine user behavior research report.
14. CORMACK, G. V., SMUCKER, M. D., AND CLARKE, C. L. A. 2011. Efficient and effective spam filtering and re-ranking for large Web datasets. Inform. Retrieval. 1-25.
15. CRASWELL, N., HAWKING, D., AND ROBERTSON, S. 2001. Effective site finding using link anchor information. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'01). ACM, New York, NY, 250-257. (Pubitemid 32951078)
16. DAVISON, B. 2000. Recognizing nepotistic links on the Web. In Proceedings of the AAAI-2000 Workshop on Artificial Intelligence for Web Search. Tech. rep. WS-00-01. 23-28.
17. DENIS, F. 1998. PAC learning from positive statistical queries. In Proceedings of the 9th International Conference on Algorithmic Learning Theory. Lecture Notes in Computer Science, vol. 1501, 112-126. (Pubitemid 128149820)
18. FETTERLY, D., MANASSE, M., AND NAJORK, M. 2004. Spam, damn spam, and statistics: Using statistical analysis to locate spam Webpages. In Proceedings of the 7th International Workshop on the Web and Databases. 1-6.
19. FUXMAN, A., TSAPARAS, P., ACHAN, K., AND AGRAWAL, R. 2008. Using the wisdom of the crowds for keyword generation. In Proceeding of the 17th International World Wide Web Conference. ACM, New York, NY, 61-70.
20. GENG, G., WANG, C., LI, Q., XU, L., AND JIN, X. 2007. Boosting the performance of web spam detection with ensemble under-sampling classification. In Proceedings of the 4th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD'07). 583-587. (Pubitemid 351711103)
21. GYONGYI, Z. AND GARCIA-MOLINA, H. 2005. Web spam taxonomy. In Proceedings of the 1st International Workshop on Adversarial Information Retrieval on the Web. 1-9.
22. GYÖNGYI, Z., GARCIA-MOLINA, H., AND PEDERSEN, J. 2004. Combating Web spam with trustrank. In Proceedings of the 13th International Conference on Very Large Data Bases. 576-587.
23. HENZINGER, M. R., MOTWANI, R., AND SILVERSTEIN, C. 2003. Challenges in Web search engines. In Proceedings of the 18th International Joint Conference on Artificial Intelligence. 1573-1579.
24. JANSEN, J. B. 2007. Click fraud. Comput. 40, 7, 85-86.
25. KLEINBERG, J. M. 1999. Authoritative sources in a hyperlinked environment. J. ACM 46, 5, 604-632.
26. KRISHNAN, V. AND RAJ, R. 2006. Web spam detection with anti-trust-rank. In Proceedings of the 2nd International Workshop on Adversarial Information Retrieval on the Web (AIRWeb).
27. LIU, Y., GAO, B., LIU, T., ZHANG, Y., MA, Z., HE, S., AND LI, H. 2008. BrowseRank: Letting Web users vote for page importance. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'08). ACM, New York, NY, 451-458.
28. LIU, Y., CEN, R., ZHANG, M., MA, S., AND RU, L. 2008a. Identifying Web spam with user behavior analysis. In Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web (AIRWeb'08). ACM, New York.
29. LIU, Y., ZHANG, M.,MA, S., AND RU, L. 2008b. User behavior oriented Web spam detection. In Proceeding of the 17th International World Wide Web Conference (WWW'08). ACM, New York, NY, 1039-1040.
30. LIU, Y., ZHANG, M.,MA, S., AND RU, L. 2009. User browsing graph: Structure, evolution, and application. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining (WSDM'09).
31. MANEVITZ, L. M. AND YOUSEF, M. 2002. One-class SVMs for document classification. Mach. Learn. 2, 139-154.
32. MITCHELL, T. 1997. Chapter 6: Bayesian Learning, Machine Learning, McGraw-Hill Education, New York, NY.
33. NIGAM, K., MCCALLUM, A. K., THRUN, S., AND MITCHELL, T. 2000. Text classification from labeled and unlabeled documents using EM. Mach. Learn. 39, 2-3, 103-134. (Pubitemid 30594822)
34. NTOULAS, A., NAJORK, M., MANASSE, M., AND FETTERLY, D. 2006. Detecting spam Web pages through content analysis. In Proceedings of the 15th International World Wide Web Conference (WWW'06). ACM Press, New York, NY, 83-92. (Pubitemid 46946600)
35. PISKORSKI, J., SYDOW, M., AND WEISS, D. 2008. Exploring linguistic features for Web spam Detection: A preliminary study. In Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web (AIRWeb). ACM, New York, NY, 25-28.
36. SILVERSTEIN, C., MARAIS, H., HENZINGER, M., AND MORICZ, M. 1999. Analysis of a very large Web search engine query log. SIGIR Forum 33, 1, 6-12.
37. SONG, R., LIU, H.,WEN, J., AND MA, W. 2004. Learning block importance models for webpages. In Proceedings of the 13th international World Wide Web Conference (WWW'04). ACM, New York, NY, 203-211. (Pubitemid 40752755)
38. SVORE, K.,WU, Q., BURGES, C. AND RAMAN, A. 2007. Improving Web spam classification using rank-time features. In Proceedings of the 3rd International Workshop on Adversarial Information Retrieval on the Web (AIRWeb'07).
39. VOORHEES, E. M. 2001. The philosophy of information retrieval evaluation. In Revised Papers from the 2nd Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems (CLEF'01). 355-370.
40. WANG, Y., MA, M., NIU, Y., AND CHEN, H. 2007. Spam double-funnel: Connecting Web spammers with advertisers. In Proceedings of the 16th International World Wide Web Conference (WWW'07). ACM, New York, NY, 291-300. (Pubitemid 47582259)
41. WU, B. AND DAVISON, B. 2005. Cloaking and redirection: A preliminary study. In Proceedings of the 1st International Workshop on Adversarial Information Retrieval on the Web.
42. YU, H., HAN, J., AND CHANG, K. C. 2004. PEBL: Web page classification without negative examples. IEEE Trans. Knowl. Data Engin. 16, 1, 70-81.