On the universality of rank distributions of website popularity

Computer Networks

Volumn 50, Issue 11, 2006, Pages 1769-1780

  Krashakov, Serge A ^a   Teslyuk, Anton B ^a   Shchur, Lev N ^a  

^a LANDAU INSTITUTE FOR THEORETICAL PHYSICS   (Russian Federation)

Author keywords

Internet; Rank distribution; Web traffic; Zipf law

Indexed keywords

APPROXIMATION THEORY; COMPUTER NETWORKS; DATA ACQUISITION; DISTRIBUTED DATABASE SYSTEMS; INTERNET; PARAMETER ESTIMATION; QUERY LANGUAGES; SERVERS; STATISTICAL METHODS; TELECOMMUNICATION TRAFFIC;

CACHE SERVERS; RANK DISTRIBUTIONS; WEB TRAFFIC; ZIPF LAW;

WEBSITES;

EID: 33744510185     PISSN: 13891286     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.comnet.2005.07.009     Document Type: Article

References (37)

1. S. Glassman, A caching relay for the World Wide Web, in: Proc. 1st Int. Conference on the World-Wide Web, CERN, Geneva, Switzerland, May 1994. Comput. Networks ISDN Syst. 27(2) (1994) 165-173.
2. L. Breslau, P. Cao, L. Fan, G. Phillips, S. Shenker, Web caching and Zipf-like distributions: evidence and possible implications, in: Proc. IEEE INFOCOM '99: 18th Annual Joint Conference of the IEEE Computer and Communications Societies, vol. 1, 1999, pp. 126-134.
3. Kelly T., and Mogul J. Aliasing on the World Wide Web: prevalence and performance implications. Proc. 11th Int. WWW Conf., Honolulu, May 2002 (2002), ACM Press pp. 281-292
4. Doyle R.P., Chase J.S., Gadde S., and Vahdat A.M. The trickle-down effect: web caching and server request distribution. Comput. Commun. 25 (2002) 345-356
5. V. Pareto, Cours d'economie politique, Rouge, Lausanne et Paris, 1897.
6. Zipf G.K. Human Behavior and the Principle of Least-Effort (1949), Addison-Wesley, Cambridge, MA
7. Li W. Random texts exhibit Zipf's-law-like word frequency distribution. IEEE Trans. Inform. Theory 38 6 (1992) 1842-1845
8. Mantegna R.N., Buldyrev S.V., Goldberger A.L., Havlin S., Peng C.K., Simons M., and Stanley H.E. Linguistic features of noncoding DNA sequences. Phys. Rev. Lett. 73 (1994) 3169-3172
9. Martindale C., and Konopka A.K. Oligonucleotide frequencies in DNA follow a Yule distribution. Comput. Chem. 20 (1996) 35-38
10. Mandelbrot B.B. The Fractal Geometry of Nature (1977), Freeman, New York
11. Shklovskii B.I., and Efros A.L. Electronic Properties of Doped Semiconductors, Percolation Theory (1984), Springer-Verlag, Berlin pp. 95-136
12. Shchur L.N. Incipient spanning clusters in square and cubic percolation. In: Landau D.P., Lewis S.P., and Schüttler H.B. (Eds). Springer Proceedings in Physics. Computer Simulation Studies in Condensed Matter Physics XII (2000), Springer-Verlag, Berlin
13. Han D.-D., Liu J.-G., Ma Y.-G., Cai X.-Z., and Shen W.-Q. Scale-free download network for publications. Chin. Phys. Lett. 21 9 (2004) 1855-1857 Available from:
14. Zanette D.H., and Manrubia S.C. Role of intermittence in urban development: a model of large-scale city formation. Phys. Rev. Lett. 79 3 (1997) 523-526
15. Marsili M., and Zhang Y.-C. Interacting individuals leading to Zipf's law. Phys. Rev. Lett. 80 12 (1998) 2741-2744
16. Redner S. How popular is your paper? An empirical study of the citation distribution. Eur. Phys. J. B 4 (1998) 131-134
17. Malamud B.D., Morein G., and Turcotte D.L. Forest fires: an example of self-organized critical behavior. Science 281 (1998) 1840-1842
18. M.E.J. Newman, Power laws, Pareto distributions and Zipf's law, 2004. Available from: .
19. Mitzenmacher M. A brief history of generative models for power law and lognormal distributions. Internet Math. 1 2 (2004) 226-251
20. Laherrère J., and Sornette D. Stretched exponential distributions in nature and economy: "fat tails" with characteristic scales. Eur. Phys. J. B 2 (1998) 525-539
21. Bestavros A. WWW traffic reduction and load balancing through server-based caching. IEEE Concurrency 5 1 (1997) 56-67
22. Hatashima T., Motoda T., and Yamamoto S. An "interest" index for WWW servers and CyberRanking. IEICE Trans. Inform. Syst. E83-D (2000) 729-734
23. V.N. Padmanabhan, L. Qiu, The content and access dynamics of a busy web site: findings and implications, in: Proc. ACM SIGCOMM'00, Stockholm, Sweden, 2000, pp. 111-123.
24. Oke A., and Bunt R. Hierarchical workload characterization for a busy web server. Computer Performance Evaluation. In: Field T., Harrison P.G., Bradley J., and Harder U. (Eds). Proc. TOOLS'2002: 12th Int. Conf. on Modeling Techniques and Tools, London, UK, 14-17 April 2002. Lecture Notes in Computer Science vol. 2324 (2002), Springer-Verlag, Berlin 309-328
25. Aida M., Takahashi N., and Abe T. A proposal of dual Zipfian model for describing HTTP access trends and its application to address cache design. IEICE Trans. Commun. E81-B 7 (1998) 1475-1485
26. C.R. Cunha, A. Bestavros, M.E. Crovella, Characteristics of WWW client-based Traces, Technical report BU-CS-95-010, Boston University, July, 1995.
27. P. Barford, A. Bestavros, A. Bradley, M. Crovella, Changes in web client access patterns: characteristics and caching implications, World Wide Web J., Spec. Issue on Characterization and Performance Evaluation, 2, 15-28 (1999).
28. S. Jin, A. Bestavros, Sources and characteristics of web temporal locality, in: Proc. MASCOTS'2000: The 8th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, San Francisco, CA, 29 August-1 September 2000, pp. 28-35.
29. A. Mahanti, C. Williamson. Web proxy workload characterization, Tech. Report, Department of Computer Science, University of Saskatchewan, February 1999.
30. M.E. Crovella, M.S. Taqqu, Estimating the heavy tail index from scaling properties, in: Methodology and Computing in Applied Probability, vol. 1, 1999, pp. 55-79.
31. Mahanti A., Williamson C., and Eager D. Traffic analysis of a web proxy caching hierarchy. IEEE Network Mag. 14 3 (2000) 16-23
32. C. Roadknight, I. Marshall, D. Vearer, File popularity characterization, in: Proc. WISP'99: 2nd Workshop on Internet Server Performance, Atlanta, Georgia, May 1999.
33. V. Almeida, A. Bestavros, M.E. Crovella, A. de Oliveira, Characterizing reference locality in the WWW, in: Proc. PDIS'96, December 1996, pp. 92-103.
34. Nishikawa N., Hosokawa T., Mori Y., Yoshida K., and Tsuji H. Memory-based architecture for distributed WWW caching proxy. Comput. Networks ISDN Syst. 30 (1998) 205-214
35. S. Krashakov, L. Shchur. WWW caching in Russia-current state and future development, in: Proc. 3rd Int. Web Caching Workshop, Manchester, 15-17 June, 1998.
36. The IRCache project. Available from: .
37. Krashakov S.A., and Shchur L.N. Active measurements (experiments) of the Internet traffic using cache-mesh. Int. J. Mod. Phys. C 12 (2001) 549-562