Cloud resource usage - Extreme distributions invalidating traditional capacity planning models

ScienceCloud'11 - Proceedings of the 2nd International Workshop on Scientific Cloud Computing

Volumn , Issue , 2011, Pages 7-14

  Loboz, Charles Z ^a  

^a MICROSOFT   (United States)

Author keywords

capacity planning; power law; probability distributions; resource usage; volatility

Indexed keywords

CAPACITY BUFFER; CAPACITY MODELS; CAPACITY PLANNING; CENTRAL LIMIT THEOREM; CONFIDENCE INTERVAL; EXTREME DISTRIBUTION; HIGH VOLATILITY; INTERNAL NETWORK; LOG-NORMAL; NON-NORMAL DISTRIBUTION; POWER LAW; POWER LAW DISTRIBUTION; RESOURCE USAGE; SAMPLE DISTRIBUTIONS; UNDERLYING DISTRIBUTION; VOLATILITY; WORKLOAD MODELING;

COMPUTER SYSTEMS; FORECASTING; MATHEMATICAL TECHNIQUES; NORMAL DISTRIBUTION;

CLOUD COMPUTING;

EID: 79961040259     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1996109.1996112     Document Type: Conference Paper

References (25)

1. Clauset, A., Shalizi, C. R. and Newman, M. E. J. (2009). "Power-law distributions in empirical data". SIAM Review 51: 661-703.
2. Clauset, A., M. Young, and K. S. Gleditsch, 2007, Journal of Conflict Resolution 51, 58.
3. Goldstein M. L., Morri S. A. s, and Yen G. G.. Problems with fitting to the power-law distribution. The European Physical Journal B - Condensed Matter and Complex Systems, 41(2):255-258, September 2004.
4. Gunther, N. Guerilla Capacity Planning, iUniverse (October 31, 2000), ISBN-10: 3642065570
5. James, A. and Plank, M. J. On fitting power laws to ecological data arxiv:0712.0613
6. Leland, W., Taqqu M., Willinger W., Wilson D., On the Self-Similar Nature of Ethernet Traffic, IEEE/ACM TON, 1994
7. Lilifoers, H.W., Journal of the American Statistical Association 64, (1969) 387-389.
8. Mantegna, Rosario N. , Stanley, H. Eugene , An Introduction to Econophysics: Correlations and Complexity in Finance, Cambridge University Press (Cambridge, 1999)
9. Marvasti, M. A., How 'Normal' is your IT data, Proceedings of the Computer Measurement Group's 2009 International Conference, www.cmg.org
10. Newman, M. E. J., 2006. Power laws, Pareto distributions and Zipf's law. Cont. Phys., 46, 323-351.
11. Shalizi, C Power Law Distributions, 1/f Noise, Long-Memory Time Series http://cscs.umich.edu/~crshalizi/notabene/power-laws.html
12. Van der Loo, Mark P.J Distribution based outlier detection in univariate data, discussion paper 10003, Statistic Netherlands
13. Agrawal, N, Bolosky, W. J., Douceur, J. R., and.Lorch, J. R,. 2007. A five-year study of file-system metadata. Trans. Storage 3,3,Article 9 (October 2007). DOI= 10.1145/1288783.1288788 http://doi.acm.org/10.1145/1288783.1288788
14. Li, H., Workload dynamics on clusters and grids, The Journal of Supercomputing, Volume 47 Issue 1, January 2009
15. Li, H., Muskulus, M., Wolters, L., Modeling job arrivals in a dataintensive grid, in: E. Frachtenberg, U. Schwiegelshohn (Eds.), Int'l. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Revised Selected Papers, in: Lecture Notes in Computer Science, vol. 4376, Springer, 2007, pp. 210-231
16. Litzkow, M.J.; Livny, M.; Mutka, M.W.; Condor-a hunter of idle workstations, 8th International Conference on Distributed Computing Systems, 1988, p 104 - 111
17. Iosup A., Li, H., Jan, M., Anoep, S, Dumitrescu,C., Wolters ,L., Dick, H., Epema, J.: The Grid Workloads Archive. Future Generation Comp. Syst. 24(7): 672-686 (2008)
18. Li, H., Heusdens, R., Muskulus, M.v, Wolters, L., Analysis and synthesis of pseudo-periodic job arrivals in grids: A matching pursuit approach IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid) IEEE Computer Society, 2007, pp. 183-196.
19. Li, H., Muskulus, M., Wolters, L., Modeling job arrivals in a dataintensive grid, in: Frachtenberg, E., Schwiegelshohn, U. (Eds.), Int'l. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Revised Selected Papers, in: Lecture Notes in Computer Science, vol. 4376, Springer, 2007, pp. 210-231.
20. Li, H., Wolters, L., Towards a better understanding of workload dynamics on data-intensive clusters and grids, in: Int'l. Parallel &Distributed Processing Symposium (IPDPS), IEEE Computer Society, 2007, pp. 1-10
21. Li, H., Workload Characterization, Modeling, and Prediction in Grid Computing, PhD thesis, https://openaccess.leidenuniv.nl/bitstream/1887/12574/1/ Thesis.pdf
22. Park, C., Hernandez-Campos, F., Marron, J. S., and Donelson Smith, F.. 2005. Long-range dependence in a changing internet traffic mix. Comput. Netw. 48, 3 (June 2005), 401-422
23. Allspaw, J. , The Art of Capacity Planning: Scaling Web Resources, O'Reilly Media; 1 edition (September 15, 2008), ISBN-10: 0596518579
24. Albert, R.v, Barabási, A.-L. , (2002) Statistical mechanics of complex networks. Reviews of Modern Physics 74, 47-97
25. Rasch, D., Guiard, V. The robustness of parametric statistical methods, Psychology Science, Vol. 46, 2004(2), p.175-208