A learning architecture for scheduling workflow applications in the cloud

Proceedings - 9th IEEE European Conference on Web Services, ECOWS 2011

Volumn , Issue , 2011, Pages 83-90

  Barrett, Enda ^a   Howley, Enda ^a   Duggan, Jim ^a  

^a NATIONAL UNIVERSITY OF IRELAND   (Ireland)

Author keywords

Bayesian Model Learning; Genetic Algorithm; Markov Decision Process; Workflow Scheduling

Indexed keywords

BAYESIAN MODEL; COMPUTATIONAL COSTS; COMPUTATIONAL PROCESSING; COMPUTATIONAL RESOURCES; ENVIRONMENTAL STATE; EXPERIMENTAL DATA; LARGE AMOUNTS OF DATA; LEARNING ARCHITECTURES; MARKET MODEL; MARKOV DECISION PROCESSES; NON-FUNCTIONAL; PAY-PER-USE; PLANNING PROBLEM; PRECEDENCE CONSTRAINTS; SCIENTIFIC WORKFLOWS; SERVICE REQUIREMENTS; STORAGE SITES; WORKFLOW APPLICATIONS; WORKFLOW EXECUTION; WORKFLOW SCHEDULING; WORKFLOW-BASED APPLICATIONS;

BAYESIAN NETWORKS; CLOUD COMPUTING; COSTS; GENETIC ALGORITHMS; MARKOV PROCESSES; QUALITY OF SERVICE;

WEB SERVICES;

EID: 81255191512     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ECOWS.2011.27     Document Type: Conference Paper

References (24)

1. Montage: An astronomical image engine, http://montage.ipac.caltech.edu.
2. Illumina:, http://www.illumina.com.
3. C. Vecchiola, S. P, and R. Buyya, "High-performance cloud computing: A view of scientific applications."
4. G. Juve and E. Deelman, "Scientific workflows and clouds," ACM Crossroads, vol. 16, no. 3, pp. 14-18, 2010.
5. E. Deelman, G. Singh, M. Livny, B. Berriman, and J. Good, "The cost of doing science on the cloud: the montage example," in Proceedings of the 2008 ACM/IEEE conference on Supercomputing, ser. SC '08. Piscataway, NJ, USA: IEEE Press, 2008, pp. 50:1-50:12. [Online]. Available: http://portal.acm.org/ citation.cfm?id=1413370.1413421
6. J. Yu, R. Buyya, and C. K. Tham, "Cost-based scheduling of scientific workflow application on utility grids," in Proceedings of the First International Conference on e-Science and Grid Computing. Washington, DC, USA: IEEE Computer Society, 2005, pp. 140-147. [Online]. Available: http://portal.acm.org/citation.cfm?id=1107836.1107867
7. J. Yu and R. Buyya, "Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms," in Scientific Programming. IOS Press, 2006, pp. 217-230. (Pubitemid 46474559)
8. J. Yu, M. Kirley, and R. Buyya, "Multi-objective planning for workflow execution on grids," in Proceedings of the 8th IEEE/ACM International Conference on Grid Computing, ser. GRID '07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 10-17. [Online]. Available: http://dx.doi.org/10.1109/GRID.2007.4354110
9. E. Deelman, G. Singh, M. hui Su, J. Blythe, A. Gil, C. Kesselman, G. Mehta, K. Vahi, G. B. Berriman, J. Good, A. Laity, J. C. Jacob, and D. S. Katz, "Pegasus: a framework for mapping complex scientific workflows onto distributed systems," Scientific Programming Journal, vol. 13, pp. 219-237, 2005.
10. B. Ludscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. A. Lee, J. Tao, and Y. Zhao, "Scientific workflow management and the kepler system," in Concurr. Comput. : Pract. Exper, 2005, p. 2006.
11. T. Oinn, M. Addis, J. Ferris, D. Marvin, T. Carver, M. R. Pocock, and A. Wipat, "Taverna: A tool for the composition and enactment of bioinformatics workflows," Bioinformatics, vol. 20, p. 2004, 2004.
12. S. P, L. Wu, S. M. Guru, and R. Buyya, "A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments."
13. Z. Wu, X. Liu, Z. Ni, D. Yuan, and Y. Yang, "A market-oriented hierarchical scheduling strategy in cloud workflow systems," The Journal of Supercomputing, pp. 1-38, 2011, 10.1007/s11227-011-0578-4. [Online]. Available: http://dx.doi.org/10.1007/s11227-011-0578-4
14. D. Vengerov, "A reinforcement learning approach to dynamic resource allocation," Eng. Appl. Artif. Intell., vol. 20, no. 3, pp. 383-390, 2007. (Pubitemid 46389478)
15. G. Tesauro, N. K. Jong, R. Das, and M. N. Bennani, "On the use of hybrid reinforcement learning for autonomic resource allocation,"Cluster Computing, vol. 10, no. 3, pp. 287-299, 2007. (Pubitemid 47278342)
16. D. Gmach, J. Rolia, L. Cherkasova, and A. Kemper, "Workload analysis and demand prediction of enterprise data center applications," in Workload Characterization, 2007. IISWC 2007. IEEE 10th International Symposium on, Sept. 2007, pp. 171-180.
17. J. Perez, C. Germain-Renaud, B. Kégl, and C. Loomis, "Grid differentiated services: A reinforcement learning approach," in CCGRID '08: Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid. Washington, DC, USA: IEEE Computer Society, 2008, pp. 287-294.
18. R. S. Sutton and A. G. Barto, "Reinforcement learning :an introduction,"1998.
19. G. R. Nudd, D. J. Kerbyson, E. Papaefstathiou, S. C. Perry, J. S. Harper, and D. V. Wilcox, "Pace - a toolset for the performance prediction of parallel and distributed systems," 1999.
20. W. Smith, I. Foster, and V. Taylor, "Predicting application run times using historical information," 1997.
21. J. D. Ullman, "Np-complete scheduling problems," J. Comput. Syst. Sci., vol. 10, pp. 384-393, June 1975. [Online]. Available: http://dx.doi.org/10.1016/S0022-0000(75)80008-0
22. J. Oh and C. Wu, "Genetic-algorithm-based real-time task scheduling with multiple goals," J. Syst. Softw., vol. 71, pp. 245-258, May 2004. [Online]. Available: http://portal.acm.org/citation.cfm?id=993017.993022
23. P. Doshi, "Dynamic workflow composition using markov decision processes," International Journal of Web Services Research, vol. 2, pp. 1-17, 2005. (Pubitemid 40080055)
24. R. N. Calheiros, R. Ranjan, C. A. F. D. Rose, and R. Buyya, "Cloudsim: A novel framework for modeling and simulation of cloud computing infrastructures and services," CoRR, vol. abs/0903.2525, 2009.