Searching the world wide web

Science

Volumn 280, Issue 5360, 1998, Pages 98-100

  Lawrence, Steve ^a   Giles, C Lee ^a,b  

^a NEC LABORATORIES AMERICA   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPUTER NETWORK; INFORMATION SYSTEM; INTERNET; PRIORITY JOURNAL; TECHNOLOGY;

EID: 0032478628     PISSN: 00368075     EISSN: None     Source Type: Journal    
DOI: 10.1126/science.280.5360.98     Document Type: Article

References (22)

1. W. Howe, "When did the Internet start?: A brief capsule history," www.delphi.com/navnet/faq/history. html (9 May 1996); "Network Wizards Internet Domain Survey," www.nw.com/zone/WWW/top.html (July 1997); A. M. Rutkowski, "Internet trends," www. genmagic.com/Internet/Trends/(February 1997).
2. W. Howe, "When did the Internet start?: A brief capsule history," www.delphi.com/navnet/faq/history. html (9 May 1996); "Network Wizards Internet Domain Survey," www.nw.com/zone/WWW/top.html (July 1997); A. M. Rutkowski, "Internet trends," www. genmagic.com/Internet/Trends/(February 1997).
3. W. Howe, "When did the Internet start?: A brief capsule history," www.delphi.com/navnet/faq/history. html (9 May 1996); "Network Wizards Internet Domain Survey," www.nw.com/zone/WWW/top.html (July 1997); A. M. Rutkowski, "Internet trends," www. genmagic.com/Internet/Trends/(February 1997).
4. J. M. Barrie and D. E. Presti, Science 274, 371 (1996).
5. G. Taubes, ibid. 271, 764 (1996).
6. P. T. Fox and J. L. Lancaster, ibid. 266, 994 (1994).
7. B. R. Schatz, ibid. 275, 327 (1997).
8. S. G. Steinberg, Wired 4 (no. 5), 108 (1996).
9. M. Gray, "Measuring the growth of the Web: June 1993 to June 1995," www.mit.edu/people/mkgray/ growth/ (1996).
10. G. Salton and M. J. McGill, Introduction to Modern Information Retrieval (McGraw-Hill, New York, 1983).
11. E. Selberg and O. Etzioni, in Proceedings of the Fourth International World Wide Web Conference, Boston, MA, 11 to 14 December 1995, p. 195.
12. M. Eisenberg and C. Barry, Proceedings of the 49th Annual Meeting of the American Society for Information Science (Learned Information, Medford, NJ, 1986), p. 80.
13. The URLs were normalized by (i) removing any "index.html" suffix or trailing "/", (ii) removing a port 80 designation (the default), (iii) removing the first segment of the domain name for URLs with a directory depth greater than 1 (to account for machine aliases), and (iv) unescaping any "escaped" characters (for example, %7E in a URL is equivalent to the tilde character).
14. The search engines typically impose an upper limit on the number of documents that can be retrieved [current limits are 200 for AltaVista (simple search), 500 for Infoseek, 1000 for HotBot, Excite, and Lycos, and >10,000 for Northern Light], and we checked to ensure that these limits were not exceeded.
15. M. Cunningham, "Brewster's millions," www.irish-times.com/irish-times/paper/1997/0127/cmp1.html (27 January 1997).
16. C. Guglielmo, Upside (November 1997), p. 48.
17. D. Brake, New Sci. 154, 12 (1997).
18. PR Newswire, "Wired Digital's HotBot search site unveils largest Web index; 110 million page database extends and enhances Web power searching" (10 December 1997).
19. T. Mitchell, personal communication.
20. O. Etzioni and D. Weld, Commun. ACM 37 (no. 7), 72 (1994).
21. J. Shakes, M. Langheinrich, O. Etzioni, in Proceedings of the Sixth International World Wide Web Conference, Santa Clara, CA, 7 to 11 April 1997.
22. We would like to thank A. Grove, B. Horne, B. Krovetz, J. Oliensis, S. Omohundro, H. Stone, L. Williams, P. Yianilos, and the anonymous reviewers for useful comments and suggestions. All registered and unregistered trademarks are the property of their respective owners.