Host load prediction in a Google compute cloud with a Bayesian model

International Conference for High Performance Computing, Networking, Storage and Analysis, SC

Volumn , Issue , 2012, Pages

  Di, Sheng ^a   Kondo, Derrick ^a   Cirne, Walfredo ^b  

^a INRIA   (France)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURATE PREDICTION; AUTO-REGRESSION; LOAD PREDICTIONS; MEAN SQUARED ERROR; MOVING AVERAGES; PREDICTION METHODS; SERVICE LEVEL AGREEMENTS; STATE-OF-THE-ART METHODS;

BAYESIAN NETWORKS; DIGITAL STORAGE;

FORECASTING;

EID: 84877707061     PISSN: 21674329     EISSN: 21674337     Source Type: Conference Proceeding    
DOI: 10.1109/SC.2012.68     Document Type: Conference Paper

References (30)

1. L. Carrington, A. Snavely, and N. Wolter, "A performance prediction framework for scientific applications," Future Generature Computing Systems, vol. 22, pp. 336-346, February 2006. (Pubitemid 41689815)
2. C. Dabrowski and F. Hunt, "Using markov chain analysis to study dynamic behaviour in large-scale grid systems," in Seventh Australasian Symposium on Grid Computing and e-Research (AusGrid 2009), ser. CRPIT, vol. 99. Wellington, New Zealand: ACS, 2009, pp. 29-40.
3. S. Akioka and Y. Muraoka, "Extended forecast of cpu and network load on computational grid," in Proceedings of the 2004 IEEE International Symposium on Cluster Computing and the Grid (CCGRID'04), Washington, DC, USA: IEEE Computer Society, 2004, pp. 765-772.
4. P. A. Dinda and D. R. O'Hallaron, "Host load prediction using linear models," Cluster Computing, vol. 3, pp. 265-280, October 2000.
5. Y. Zhang, W. Sun, and Y. Inoguchi, "Cpu load predictions on the computational grid," in IEEE International Symposium on Cluster Computing and the Grid (CCGrid'2006), Los Alamitos, CA, USA, 2006, pp. 321-326.
6. R. Wolski, "Dynamically forecasting network performance using the network weather service," Cluster Computing, vol. 1, pp. 119-132, January 1998.
7. J. M. Tirado, D. Higuero, F. Isaila, and J. Carretero, "Multimodel prediction for enhancing content locality in elastic server infrastructures, " in Proceedings of 18th High Performance Computing (HiPC'2011), 2011.
8. Y. Wu, K. Hwang, Y. Yuan, and W. Zheng, "Adaptive workload prediction of grid performance in confidence windows," IEEE Transactions on Parallel and Distributed Systems, vol. 21, no. 7, pp. 925-938, july 2010.
9. Google cluster-usage traces: online at http://code.google.com/p/ googleclusterdata.
10. J. Wilkes, "More Google cluster data," Google resarch blog, Nov 2011, online at http://googleresearch.blogspot.com/2011/11/more-google-cluster- data.html.
11. A. K. Mishra, J. L. Hellerstein, W. Cirne, and C. R. Das, "Towards characterizing cloud backend workloads: insights from google compute clusters," SIGMETRICS Perform. Eval. Rev., vol. 37, no. 4, pp. 34-41, Mar. 2010.
12. The grid workloads archive: online at http://gwa.ewi.tudelft.nl/pmwiki/.
13. B. Sharma, V. Chudnovsky, J. L. Hellerstein, R. Rifaat, and C. R. Das, "Modeling and synthesizing task placement constraints in google compute clusters," in Proceedings of the 2nd ACM Symposium on Cloud Computing (SOCC'11), New York, USA: ACM, 2011, pp. 3:1-3:14.
14. Q. Zhang, J. L. Hellerstein, and R. Boutaba, "Characterizing task usage shapes in google's compute clusters," in Large Scale Distributed Systems and Middleware Workshop (LADIS'11), 2011.
15. J. O. Berger, Statistical Decision Theory and Bayesian Analysis. New York: Springer-Verlag, 1985.
16. Webb, Boughton, and W. Zhihai, "Not so naive bayes: Aggregating one-dependence estimators," Machine Learning, vol. 58, no. 1, pp. 5-24, Jan. 2005.
17. Auvergrid: online at http://www.auvergrid.fr/.
18. P. S. R. Diniz, Adaptive Filtering: Algorithms and Practical Implementation, softcover reprint of hardcover 3rd ed. 2008 ed. Springer, Oct. 2010.
19. M. B. Wilk and R. Gnanadesikan, "Probability plotting methods for the analysis of data." Biometrika, vol. 55, no. 1, pp. 1-17, Mar. 1968.
20. M. A. Farahat and M. Talaat, "A new approach for short-term load forecasting using curve fitting prediction optimized by genetic algorithms," in Proceedings of the 14th International Middle East Power Systems Conference (MEPCON'10), 2010, pp. 106-110.
21. R. E. Kalman, "A new approach to linear filtering and prediction problems," Transactions of the ASME - Journal of Basic Engineering, no. 82 (Series D), pp. 35-45, 1960.
22. P. Domingos and M. Pazzani, "On the optimality of the simple bayesian classifier under zero-one loss," Machine Learning, vol. 29, no. 2-3, pp. 103-130, Nov. 1997. (Pubitemid 127510035)
23. M. H. Dunham, Data Mining: Introductory and Advanced Topics, 1st ed. Prentice Hall, Sep. 2002.
24. R. K. Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation and Modelling. John Wiley & Sons, April 1991.
25. C. Spearman, "The proof and measurement of association between two things. by c. spearman, 1904." The American journal of psychology, vol. 100, no. 3-4, pp. 441-471, 1987.
26. P. Domingos and M. J. Pazzani, "Beyond independence: Conditions for the optimality of the simple bayesian classifier," in Proceedings of the 13th International Conference on Machine Learning (ICML'96), 1996, pp. 105-112.
27. Y. Wu, Y. Yuan, G. Yang, and W. Zheng, "Load prediction using hybrid model for computational grid," in The 8th IEEE/ACM International Conference on Grid Computing (Grid'07), 2007, pp. 235-242.
28. S. J. Orfanidis, Introduction to signal processing. Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 1995.
29. B. J. Barnes, B. Rountree, D. K. Lowenthal, J. Reeves, B. de Supinski, and M. Schulz, "A regression-based approach to scalability prediction," in Proceedings of the 22nd annual international conference on Supercomputing (ICS'08). New York, NY, USA: ACM, 2008, pp. 368-377.
30. A. Khan, X. Yan,, S. Tao, and N. Anerousis, "Workload characterization and prediction in the cloud: A multiple time series approach," in 3rd IEEE/IFIP International Workshop on Cloud Management (Cloudman'12), 2012.