A personalized search engine based on Web-snippet hierarchical clustering

Software - Practice and Experience

Volumn 38, Issue 2, 2008, Pages 189-225

  Ferragina, P ^a   Gulli, A ^b  

^a UNIVERSITY OF PISA   (Italy)

^b Askcom   (United States)

Author keywords

Personalized Web ranking; Search engines; Web snippet clustering

Indexed keywords

CLUSTERING ALGORITHMS; DATA MINING; INFORMATION RETRIEVAL; WEB SERVICES;

PERSONALIZED WEB RANKING; WEB SNIPPET CLUSTERING;

SEARCH ENGINES;

EID: 39449131909     PISSN: 00380644     EISSN: 1097024X     Source Type: Journal    
DOI: 10.1002/spe.829     Document Type: Article

References (80)

1. Ntoulas A, Cho J, Olston C. What's new on the Web? The evolution of the Web from a search engine perspective. Proceedings of the 13th International World Wide Web Conference, New York, 2004. ACM Press: New York, 2004; 1-12.
2. Gulli A, Signorini A. The indexable Web is more than 11.5 billion pages. Proceedings of the 14th International World Wide Web Conference, Chiba, Japan, 2005; 902-903.
3. Fallows D, Rainie L, Mudd G. The popularity and importance of search engines. Technical Report, The Pew Internet & American Life Project, 2004.
4. Broder AZ. A taxonomy of Web search. SIGIR Forum 2002; 36(2):3-10.
5. Searchenginewatch. http://searchenginewatch.com/showPage.html?page= 2156241 [22 April 2007].
6. Searchenginelowdown. http://www.searchenginelowdown.com/2004/10/Web-20- exclusive-demonstration-of.html [22 April 2007].
7. Microsoft clustering. http://www.betanews.com/article/Microsoft- Tests_Search_Clustering/1106319504 [22 April 2007].
8. Mostafa J. Seeking better Web searches. Scientific American, February 2005.
9. Grabmeier J, Rudolph A. Techniques of cluster algorithms in data mining. Journal of Data Mining and Knowledge Discovery 2002; 6(4):303-360.
10. Frakes WB, Baeza-Yates R. Clustering Algorithms, ch. 16. Prentice-Hall: Englewood Cliffs, NJ, 1992.
11. Fasulo D. An analysis of recent work on clustering algorithms. Technical Report 01-03-02, Department of Computer Science and Engineering, University of Washington, 1999.
12. Pratt W, Hearst MA, Fagan LM. A knowledge-based approach to organizing retrieved documents. Proceedings of the 16th National Conference on Artificial Intelligence (AAAI'99), Orlando, FL, 1999. AAAI Press/The MIT Press: Cambridge, MA, 1999; 80-85.
13. Attardi G, Gulli A, Sebastiani F. Theseus: Categorization by context. Proceedings of the 8th International World Wide Web Conference, Toronto, CA, 1999. ACM Press: New York, 1999; 136-137.
14. Chen H, Dumais ST. Bringing order to the Web: Automatically categorizing search results. Proceedings of the ACM SIGGHI International Conference on Human Factors in Computing Systems (CHF'00), The Hague, The Netherlands, 2000. ACM Press: New York, 2000; 145-152.
15. Hearst MA, Pedersen JO. Reexamining the cluster hypothesis: Scatter/gather on retrieval results. Proceedings of the 19th ACM SIGIR International Conference on Research and Development in Information Retrieval, Zurich, 1996. ACM Press: New York, 1996.
16. Maarek YS, Fagin R, Ben-Shaul IZ, Pelleg D. Ephemeral document clustering for Web applications. Technical Report RJ 10186, IBM Research, 2000.
17. Salton G, McGill M. Introduction to Modern Information Retrieval. McGraw Hill: New York, 1983.
18. Google Labs, http://labs.google.com/personalized/ [22 April 2007].
19. My Yahoo! http://mysearch.yahoo.com/ [22 April 2007].
20. My Jeeves, http://myjeeves.ask.com/ [22 April 2007].
21. Eurekster. http://www.eurekster.com/ [22 April 2007].
22. Chirria PA, Olmedilla D, Nejdi W. PROS: A personalized ranking platform for Web search. Proceedings of the 3rd International Conference on Adaptive Hypermedia and Adaptive Web-based Systems, Eindhoven, Netherlands, 2004. Springer: Berlin, 2004; 34-43.
23. Jeh G, Widom J. Scaling personalized Web search. Proceedings of the 12th International World Wide Web Conference, Budapest, Hungary, 2003. ACM Press: New York, 2003; 271-279.
24. SnakeT's queries dataset. http://roquefort.di.unipi.it/ ~gulliflistAllowed/testSnakeT/ [22 April 2007].
25. CNN.com. Better search results than Google? Next-generation sites help narrow internet searches. Associated Press, January 2004. http://edition.cnn. com/2004/TECH/internet/01/05/seeing.searchl.ap/index.html [22 April 2007].
26. Zamir O. Clustering Web Documents: A phrase based method for grouping seach engine results. PhD Thesis, University of Washington, 1999.
27. Time Warner press release. http://media.timewarner.com/media/newmedia/ cb_press_view.cfm?release_num=55254348 [22 April 2007].
28. Searchenginewatch. http://searchenginewatch.com/showPage.html?page= 2226841 [22 April 2007].
29. Giannotti F, Nanni M, Pedreschi D. Webcat: Automatic categorization of Web search results. Proceedings of the 11th Italian Symposium on Advanced Database Systems, Cetraro, Italy, 2003. Rubettini Editor: Soveria Mannelli, Italy, 2003. 507-518.
30. Jiang Z, Joshi A, Krishnapuram R, Yi L. Managing business with Electronic Commerce 02. Retriever: Improving Web Search Engine Results Using Clustering, ch. 4, Gangopadhyay A (ed.). Idea Group Publishing: Hersley, PA, 2002.
31. Wang Y, Kitsuregawa M. On combining link and contents information for Web page clustering. Proceedings of Database and Expert Systems Applications, Aix en Provence, France, 2002. Springer: Berlin, 2002; 902-913.
32. Zamir O, Etzioni O. Grouper: A dynamic clustering interface to Web search results. Proceedings of the 8th International World Wide Web Conference, Toronto, Canada, 1999. ACM Press: New York, 1999; 1361-1374.
33. Osinski S, Weiss D. Conceptual clustering using lingo algorithm: Evaluation on open directory project data. Proceedings of New Trends Intelligent Information Processing and Web Mining, Zakopane, Poland, 2004. Springer: Berlin, 2004; 369-377.
34. Zeng H, He Q, Chen Z, Ma W. Learning to cluster Web search results. Proceedings of SIGIR04, Sheffield, U.K., 2004. ACM Press: New York, 2004; 210-217.
35. Weiss D, Stefanowski J. Web search results clustering in Polish: Experimental evaluation of carrot. Proceedings of Intelligent Information Processing and Web Mining, Zakopane, Poland, 2003. Springer: Berlin, 2003; 209-218.
36. Fung B, Wang K, Ester M. Large hierarchical document clustering using frequent itemsets. Proceedings of SIAM International Conference on Data Mining, San Francisco, CA, 2003. SIAM: Philadelphia, PA, 2003; 209-218.
37. Agrawal R, Imielinski T, Swami AN. Mining association rales between sets of items in large databases. Proceedings of ACM SIGMOD International Conference on Management of Data, Washington, DC, 1993. ACM Press: New York, 1993; 207-216.
38. Carpineto C, Romano G. Concept Data Analysis: Theory and Applications. Wiley: New York, 2004.
39. Zhang D, Dong Y. Semantic, hierarchical, on-line clustering of Web search results. Proceedings of Advanced Web Technologies and Applications, 6th Asia-Pacific Web Conference, Hangzhou, China, 2004. Springer: Berlin, 2004; 69-78.
40. Wu Y, Chen X. Extracting features from Web search returned hits for hierarchical classification. Proceedings of the International Conference on Information and Knowledge Engineering, Las Vegas, NV, 2003. CSREA Press: New York, 2003; 103-108.
41. Di Giacomo E, Didimo W, Grilli L, Liotta G. A topology-driven approach to the design of Web meta-search clustering engines. Proceedings of the 31st Annual Conference on Current Trends in Theory and Practice of Informatics, Liptovský Ján, Slovakia, 2005. Springer: Berlin, 2005; 106-116.
42. Lawrie DJ, Croft WB. Generating hierachical summaries for Web searches. Proceedings of SIGIR03, Toronto, Canada, 2003. ACM Press: New York, 2003; 457-458.
43. Lawrie DJ. Language models for hierarchical summarization. PhD Thesis, Amherst, 2003.
44. Ferragina P, Gulli A. Personalized profile based search interface with ranked and clustered display. Proceedings of the International Conference on Intelligent Agents, Web Technologies and Internet Commerce, Las Vegas, NV, 2001.
45. Kummamuru K, Lotlikar R, Roy S, Singal K, Krishnapuram R. A hierarchical monothetic document clustering algorithm for summarization and browsing search results. Proceedings of the 13th International World Wide Web Conference, New York, 2004. ACM Press: New York, 2004; 658-665.
46. Kumar S, Oztekin BU, Ertoz L, Singhal S, Han E, Kumar V. Topic-sensitive pagerank. Proceedings of the 11th International World Wide Web Conference, Honululu, 2002. IEEE Computer Society Press: Washington, DC, 2002; 517-526.
47. Google search history, http://www.google.com/searchhistory/ [22 April 2007].
48. A9 search engine, http://www.a9.com/ [22 April 2007].
49. Teevan J, Alvarado C, Ackerman MS, Karger DR. The perfect search engine is not enough: A study of orienteering behavior in directed search. Proceedings of Computer-Human Interaction Conference, Vienna, Austria, 2004. Springer: Berlin, 2004; 415-422.
50. Teevan J, Dumais ST, Horvitz E. Beyond the commons: Investigating the value of personalizing Web search. Workshop on New Technologies for Personalized Information Access, Edinburgh, Scotland, U.K., 2005. Springer: Berlin, 2005; 84-92.
51. Brin S, Page L. The anatomy of a large-scale hypertextual Web search engine. Proceedings of the 7th International World Wide Web Conference, Brisbane, Australia, 1998. ACM Press: New York, 1998; 107-117.
52. Chirita PA, Nejdl W, Paiu R, Kohlschutter C. Using odp metadata to personalize search. Proceedings of the 28th ACM International SIGIR Conference on Research and Development in Information Retrieval, Salvador, Brazil, 2005. ACM Press: New York, 2005.
53. Sarlos T, Benczur AA, Csalogany K, Fogaras D, Racz B. To randomize or not to randomize: Space optimal summaries for hyperlink analysis. Proceedings of the 15th International World Wide Web Conference, Edinburgh, Scotland, U.K., 2006. ACM Press: New York, 2006.
54. Kritikopoulos A, Sideri M. The compass filter: Search engine result personalization using Web communities. Proceedings of Intelligent Techniques for Web Personalization, Acapulco, Mexico, 2003. Springer: Berlin, 2003.
55. Sun J, Zeng H, Liu H, Lu Y, Chen Z. Cubesvd: A novel approach to personalized Web search. Proceedings of the 14th International World Wide Web Conference, Chiba, Japan, 2005. ACM Press: New York, 2005; 382-390.
56. Qiu F, Cho J. Automatic identification of user interest for personalized search. Proceedings of the 15th International World Wide Web Conference, Edinburgh, Scotland, U.K., 2006. ACM Press: New York, 2006.
57. Shen X, Zhai CX. Exploiting query history for document ranking in interactive information retrieval. Proceedings of the 26th Annual International ACM SIGIR (Poster), Toronto, Canada, 2003. ACM Press: New York, 2003; 377-378.
58. Morita M, Shinoda Y. Information filtering based on user behavior analysis and best match text retrieval. Proceedings of the 17th Annual International SIGIR, Dublin, Ireland, 1994. ACM Press: New York, 1994; 272-281.
59. Sugiyama K, Hatano K, Yoshikawa M. Adaptive Web search based on user profile constructed without any effort from user. Proceedings of the 13th International Conference on World Wide Web, New York, 2004. ACM Press: New York, 2004; 675-684.
60. Freyne J, Smyth B. Communities, collaboration and cooperation in personalized Web search. Proceedings of the 3rd Workshop on Intelligent Techniques for Web Personalization, Edinburgh, Scotland, 2005. IJCAI Secretary Treasurer: Rochester Hills, MI, 2005.
61. Bharat K. Searchpad: Explicit capture of search context to support Web search. Proceedings of the 10th International World Wide Web Conference, Amsterdam, The Netherlands, 2000. ACM Press: New York, 2000; 493-501.
62. Halkidi M, Batistakis Y, Vazirgiannis M. On clustering validation techniques. Journal of the Intelligent Information Systems 2001; 17(2-3):107-145.
63. Meila M. Comparing clusterings. Technical Report 418, University of Washington, 2002.
64. Haveliwala T, Gionis A, Klein D, Indyk P. Evaluating strategies for similarity search on the Web. Proceedings of the 11th International World Wide Web Conference, Honululu, 2002. ACM Press: New York, 2002; 432-442.
65. Nutch. http://www.nutch.org/ [22 April 2007].
66. Najork M, Wiener JL. Breadth-first search crawling yields high-quality pages. Proceedings of the 10th International World Wide Web Conference, Hong Kong, 2001. ACM Press: New York, 2001.
67. Witten IA, Moffat A, Bell TC. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann: San Francisco, CA, 1999.
68. Henzinger M, Chang B-W, Milch B, Brin S. Query-free news search. Proceedings of the 12th International World Wide Web Conference, Budapest, Hungary, 2003. ACM Press: New York, 2003; 1-10.
69. DMOZ. http://www.dmoz.com/ [22 April 2007].
70. Gulli A, Signorini A. Building an open source meta search engine. Proceedings of the 14th International World Wide Web Conference, Chiba, Japan, 2005. ACM Press: New York, 2005; 1004-1005.
71. Ferragina P, Gulli A. A personalized search engine based on Web snippet hierarchical clustering. Proceedings of the 14th International World Wide Web Conference, Chiba, Japan, 2005. ACM Press: New York, 2005; 801-810.
72. RankComparison Engine, http://rankcomparison.di.unipi.it/ [22 April 2007].
73. Snowball stemmer. http://snowball.tartarus.org/ [22 April 2007],
74. CPAN. http://www.cpan.org/ [22 April 2007].
75. Vivisimo. http://vivisimo.com/docs/personalization.pdf [22 April 2007].
76. Fagin R, Kumar R, Mahdian M, Sivakumar D, Vee E. Comparing and aggregating rankings with ties. Proceeedings of Symposium on Principles of Database Systems, Paris, France, 2004. ACM Press: New York, 2004; 47-58.
77. SnakeT's evaluation results, http://roquefort.di.unipi.it/~gulli/ listAllowed/testing/ [22 April 2007].
78. Kleinberg J. Authoritative sources in a hyper-linked environment. Proceedings of the 9th ACM-SIAM Symposium on Discrete Algorithms, San Francisco, CA, 1998. SIAM: Philadelphia, PA, 1998; 668-677.
79. Bharat K, Broder AZ. A technique for measuring the relative size and overlap of public Web search engines. Proceedings of the 7th International World Wide Web Conference, Brisbane, Australia, 1998. ACM Press: New York, 1998; 379-388.
80. Cho J, Roy S. Impact of Web search engines on page popularity. Proceedings of the 13th International World Wide Web Conference, New York, 2004. ACM Press: New York, 2004; 20-29.