Adaptive workload prediction of grid performance in confidence windows

IEEE Transactions on Parallel and Distributed Systems

Volumn 21, Issue 7, 2010, Pages 925-938

  Wu, Yongwei ^a   Hwang, Kai ^b   Yuan, Yulai ^a   Zheng, Weiming ^a  

^a TSINGHUA UNIVERSITY   (China)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

And parallel applications; Autoregression method; Grid computing; Kalman filter; Performance prediction; Savitzky Golay filter; Workload characterization

Indexed keywords

AUTOREGRESSION; AUTOREGRESSION METHOD; PARALLEL APPLICATION; SAVITZKY-GOLAY FILTER; WORKLOAD CHARACTERIZATION;

FORECASTING; KALMAN FILTERS; REGRESSION ANALYSIS; TELECOMMUNICATION SERVICES;

GRID COMPUTING;

EID: 77953090451     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2009.137     Document Type: Article

References (47)

1. H. Akaike, "Fitting Autoregressive Models for Prediction," Annals of the Inst. of Statistical Math., vol.21, no.1, pp. 243-247, Dec. 1969.
2. S. Akioka and Y. Muraoka, "Extended Forecast of CPU and Network Load on Computational Grid," Proc. IEEE Int'l Symp. Cluster Computing and Grid (CCGrid '04), pp. 765-772, 2004.
3. AuverGrid Workload Report, http://gwa.ewi.tudelft.nl/pmwiki/ reports/gwa-t-4/trace-analysis-report.html, 2009.
4. Grid Computing: Making the Global Infrastructure a Reality, F. Berman, G.C. Fox, and T. Hey, eds. Wiley, 2003.
5. L. Carrington, A. Snavely, and N. Wolter, "A Performance Prediction Framework for Scientific Applications," Future Generation Computer Systems, vol.22, pp. 336-346, 2006.
6. W. Cirne and F. Berman, "A Comprehensive Model of the Supercomputer Workload," Proc. IEEE Fourth Ann. Workshop Workloads Characterization, 2001.
7. A.A. Chien, X. Sun, and Z. Xu, "Viewpoints on Grid Standards," J. Computer Science and Technology, vol.20, no.1, 2005.
8. ChinaGrid Website, http://www.chinagrid.edu.cn, 2006.
9. M.J. Clement and M.J. Quinn, "Analytical Performance Prediction on Multicomputers," J. Supercomputing, pp. 886-894, 1993.
10. P.A. Dinda and D.R. O'Hallaron, "Host Load Prediction Using Linear Models," Cluster Computing, vol.3, pp. 265-280, 2000.
11. Sourcebook of Parallel Computing, J. Dongarra, I. Fister, G. Fox, W. Gropp, K. Kennedy, L. Torczon, and A. White, eds. Kaufman Publishers, 2002.
12. D.G. Feitelson, Workload Modeling for Computer Systems Performance Evaluation, Draft Version 0.7, Hebrew Univ. of Jerusalem, 2006.
13. I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infrastructure. Kaufmann Publishers, 2004.
14. G. Fox, D. Gannon, and M. Thomas, "Overview of Grid Computing environments," Grid Computing, F. Berman, G. Fox, and T. Hey, eds., Chapter 20, Wiley, 2003.
15. L. Gong, S.H. Sun, and E.F. Watson, "Performance Modeling and Prediction of Non-Dedicated Network Computing," IEEE Trans. Computers, vol.51, no.9, pp. 1041-1055, Sept. 2002.
16. G.C. Goodwin and K.S. Sin, Adaptive Filtering Prediction and Control. Prentice-Hall, 1984.
17. Grid Workloads Archive, http://gwa.ewi.tudelft.nl/, 2009.
18. Grid'5000, ALADDIN-G5K: Ensuring the Development of Grid '5000, https://www.grid5000.fr/, 2009.
19. K. Hwang and Z. Xu, Scalable Parallel Computing. McGraw-Hill, 1998.
20. S. Jang, X. Wu, and V. Taylor, "Using Performance Prediction to Allocate Grid Resources," GriPhyN Technical Report 2004-25, pp. 1-11, 2004.
21. M.A. Iverson, F. Ozguner, and L. Potter, "Statistical Prediction of Task Execution Time through Analytical Benchmarking for Scheduling in a Heterogeneous Environment," IEEE Trans. Computers, vol.48, no.12, pp. 1374-1379, Dec. 1999.
22. M. Kalantari and M. Akbari, "Fault-Aware Grid Scheduling Using Performance Prediction by Workload Modeling," J. Supercomputing, vol.46, no.1, pp. 15-39, Oct. 2008.
23. R.E. Kalman, "A New Approach to Linear Filtering and Prediction Problems," Trans. ASME-J. Basic Eng., vol.82, pp. 35-45, Mar. 1960.
24. N.H. Kapadia, J.A. Fortes, and C.E. Brodley, "Predictive Application-Performance Modeling in a Computational Grid Environment," Proc. IEEE Int'l Symp. High-Performance Distributed Computing (HPDC), 1999.
25. H. Li, "Workload Characterization, Modeling, and Prediction in Grid Computing," PhD thesis, ASCI Graduate School, Univ. of Leiden, Jan. 2008.
26. H. Li and R. Buyya, "Model-Driven Simulation of Grid Scheduling Strategies," Proc. Third IEEE Int'l Conf. e-Science and Grid Computing (eScience), 2007.
27. J. Liang, K. Nahrstedt, and Y. Zhou, "Adaptive Multi-Resource Prediction in Distributed Resource Sharing Environment," Proc. IEEE Int'l Symp. Cluster Computing and Grid (CCGrid '04), pp. 1-8, 2004.
28. C. Liu, L. Yang, I. Foster, and D. Angulo, "Design and Evaluation of a Resource Selection Framework for Grid Applications," Proc. 11th IEEE Int'l Symp. High-Performance Distributed Computing (HPDC '02), 2002.
29. D. Lu, H. Sheng, and P. Dinda, "Size-Based Scheduling Policies with Inaccurate Scheduling Information," Proc. 12th IEEE Int'l Symp. Modeling, Analysis, and Simulation of Computer and Telecomm. Systems (MASCOTS '04), pp. 31-38, 2004.
30. U. Lublin and D.G. Feitelson, "The Workload on Parallel Supercomputers: Modeling the Characteristics of Rigid Jobs," J. Parallel and Distributed Computing (JPDC), vol.63, no.11, pp. 1105-1122, Nov. 2003.
31. M. Maheswaran, H.J. Siegel, D. Haensgen, and R.F. Freud, "Dynamic Mapping of a Class of Independent Tasks onto Heterogeneous Computing Systems," J. Parallel and Distributed Computing (JPDC), vol.59, no.2, pp. 107-131, Feb. 1999.
32. S.L. Marple Jr., Digital Spectral Analysis with Applications. Prentice- Hall, 1987.
33. S.J. Orfanidis, Introduction to Signal Processing. Prentice-Hall, 1996.
34. J. Schopf and F. Berman, "Performance Prediction in Production Environments," Proc. 12th Int'l Parallel Processing Symp., pp. 647- 653, Apr. 1998.
35. W. Smith, I. Foster, and V. Taylor, "Predicting Application Run Times with Historical Information," J. Parallel and Distributed Computing (JPDC), vol.64, no.9, pp. 1007-1016, 2004.
36. M. Snir and D.A. Bader, "A Framework for Measuring Supercomputer Productivity," Int'l J. High-Performance Computer Applications, vol.18, no.4, pp. 417-432, 2004.
37. S. Song, K. Hwang, and Y. Kwok, "Risk-Tolerant Heuristics and Genetic Algorithms for Security-Assured Grid Job Scheduling," IEEE Trans. Computers, vol.55, no.6, pp. 703-719, June 2006.
38. V. Taylor, X. Wu, J. Geisler, and R. Stevens, "Using Kernel Computings to Predict Parallel Application Performance," Proc. Int'l Symp. High-Performance Distributed Computing (HPDC), 2002.
39. G. Welch and G. Bishop, "An Introduction to the Kalman Filter," Univ. of North Carolina at Chapel Hill, 1995.
40. R. Wolski, "Dynamically Forecasting Network Performance Using the Network Weather Service," J. Cluster Computing, vol.1, pp. 119-132, Jan. 1998.
41. R. Wolski, N. Spring, and J. Hayes, "Predicting the CPU Availability of Time-Shared Unix Systems," Proc. Eighth IEEE High Performance Distributed Computing Conf. (HPDC), 1999.
42. Y. Yuan, Y. Wu, G. Yang, and W. Zheng, "Adaptive Hydrid Model for Long-Term Load Prediction in Computational Grid," Proc. IEEE Eighth Int'l Conf. Cluster Computing and Grid (CCGrid '08), pp. 340-347, May 2008.
43. Z. Xu and K. Hwang, "Early Prediction of MPP Performance: SP2, T3D, and Paragon Experiences," J. Parallel Computing, vol.22, no.7, pp. 917-942, Oct. 1996.
44. L. Yang, I. Foster, and J.M. Schopf, "Homeostatic and Tendency- Based CPU Load Predictions," Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS '03), pp. 42-50, 2003.
45. L. Yang, X. Ma, and F. Muller, "Cross-Platform Performance Prediction of Parallel Applications Using Partial Execution," Proc. Supercomputing Conf., 2005.
46. Y. Zhang, W. Sun, and Y. Inoguchi, "CPU Load Predictions on the Computational Grid," Proc. IEEE Sixth Int'l Conf. Cluster Computing and Grid (CCGrid '06), May 2006.
47. J. Zhang and R.J. Figueiredo, "Adaptive Predictor Integration for System Performance Prediction," Proc. IEEE Int'l Parallel and Distributed Processing Symp. (IPDPS '07), Mar. 2007.