Predicting the execution time of workflow activities based on their input features

Proceedings - 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC 2012

Volumn , Issue , 2012, Pages 64-72

  Miu, Tudor ^a   Missier, Paolo ^a  

^a NEWCASTLE UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

AVERAGE EXECUTION TIME; BUILDING BLOCKES; C4.5 DECISION TREES; E-SCIENCE WORKFLOWS; LEARNING METHODS; RELEVANT FEATURES; REPETITIVE TRAINING; WORKFLOW EXECUTION;

DECISION TREES; ESTIMATION; REGRESSION ANALYSIS;

SCHEDULING ALGORITHMS;

EID: 84876562891     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SC.Companion.2012.21     Document Type: Conference Paper

References (29)

1. I. Rao and E.-N. Huh, "A probabilistic and adaptive scheduling algorithm using system-generated predictions for inter-grid resource sharing," J. Supercomput., vol. 45, no. 2, Aug. 2008.
2. C. Lin and S. Lu, "Scheduling scientific workflows elastically for cloud computing," in Cloud Computing (CLOUD), 2011 IEEE International Conference on, july 2011.
3. D. Tsafrir, Y. Etsion, and D. G. Feitelson, "Modeling user runtime estimates," in In 11th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 2005, 2005.
4. D. Tsafrir, Y. Etsion, and D. Feitelson, "Backfilling using systemgenerated predictions rather than user runtime estimates," Parallel and Distributed Systems, IEEE Transactions on, vol. 18, no. 6, june 2007.
5. V. Viana, D. de Oliveira, and M. Mattoso, "Towards a cost model for scheduling scientific workflows activities in cloud environments," in Services (SERVICES), 2011 IEEE World Congress on, july 2011.
6. E. Deelman, D. Gannon, M. Shields, and I. Taylor, "Workflows and e-Science: An overview of workflow system features and capabilities," Future Generation Computer Systems, vol. 25, no. 5, pp. 528-540, 2009. [Online]. Available: http://www.sciencedirect.com/science/article/ B6V06-4SYCPKX-2/2/ bb631979e3dd7071ddede90bbff65a91
7. J. Cartmell, S. Enoch, D. Krstajic, and D. Leahy, "Automated QSPR through Competitive Workflow," Journal of Computer-Aided Molecular Design, vol. 19, no. 11, pp. 821-833, 2005. [Online]. Available: http://dx.doi.org/10. 1007/s10822-005-9029-8
8. J. Cala, P. Watson, and S. Woodman, "Cloud Computing for Fast Prediction of Chemical Activity," in Procs. 2nd International Workshop on Cloud Computing and Scientific Applications (CCSA), Ottawa, Canada, 2012.
9. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Neurocomputing: foundations of research," J. A. Anderson and E. Rosenfeld, Eds., 1988, ch. Learning representations by back-propagating errors.
10. J. R. Quinlan, C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
11. I. H. Witten, E. Frank, and M. A. Hall, Data Mining: Practical Machine Learning Tools and Techniques, 3rd ed. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2011.
12. A. Frank and A. Asuncion, "UCI machine learning repository," 2010. [Online]. Available: http://archive.ics.uci.edu/ml
13. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The weka data mining software: an update," SIGKDD Explor. Newsl., vol. 11, no. 1, Nov. 2009.
14. R Development Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2008, ISBN 3-900051-07-0. [Online]. Available: http://www.R-project.org
15. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, Aug. 1996.
16. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition, ser. Springer Series in Statistics. Springer, Sep. 2009.
17. M. Dobber, R. van der Mei, and G. Koole, "Effective prediction of job processing times in a large-scale grid environment," in High Performance Distributed Computing, 2006 15th IEEE International Symposium on, 2006.
18. F. Nadeem and T. Fahringer, "Predicting the execution time of grid workflow applications through local learning," in Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, ser. SC '09, 2009.
19. Using templates to predict execution time of scientific workflow applications in the grid," in Cluster Computing and the Grid, 2009. CCGRID '09. 9th IEEE/ACM International Symposium on, may 2009.
20. R. Duan, F. Nadeem, J. Wang, Y. Zhang, R. Prodan, and T. Fahringer, "A Hybrid Intelligent Method for Performance Modeling and Prediction of Workflow Activities in Grids," 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 339-347.
21. X. Liu, Z. Ni, D. Yuan, Y. Jiang, Z. Wu, J. Chen, and Y. Yang, "A novel statistical time-series pattern based interval forecasting strategy for activity durations in workflow systems," J. Syst. Softw., vol. 84, no. 3, Mar. 2011.
22. A. Matsunaga and J. A. B. Fortes, "On the use of machine learning to predict the time and resources consumed by applications," in Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, ser. CCGRID '10, 2010.
23. "Adaps a three-phase adaptive prediction system for the run-time of jobs based on user behaviour," Journal of Computer and System Sciences, vol. 77, no. 2.
24. M. Tao, S. Dong, and L. Zhang, "A multi-strategy collaborative prediction model for the runtime of online tasks in computing cluster/grid," Cluster Computing, vol. 14, no. 2, Jun. 2011.
25. P. Missier, "Incremental workflow improvement through analysis of its data provenance," in Procs. TAPP'11 (Theory and Practice of Provenance), Heraklyion, Crete, Greece, June 2011.
26. M. A. Iverson, F. zgner, and L. C. Potter, "Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment," IEEE Transactions on Computers, 1999.
27. M. Kuperberg, K. Krogmann, and R. Reussner, "Performance prediction for black-box components using reengineered parametric behaviour models," in Proceedings of the 11th International Symposium on Component-Based Software Engineering, ser. CBSE '08, 2008.
28. S. Krishnaswamy, A. Zaslavsky, and S. W. Loke, in 9th International Conf. on Information Processing and Management of Uncertainty in Knowledge-Based Systems.
29. N. Kapadia, J. Fortes, and C. Brodley, "Predictive applicationperformance modeling in a computational grid environment," in High Performance Distributed Computing, 1999. Proceedings. The Eighth International Symposium on, 1999.