Temporal QoS Management in Scientific Cloud Workflow Systems

Temporal QoS Management in Scientific Cloud Workflow Systems

Volumn , Issue , 2012, Pages

  Liu, Xiao ^a   Chen, Jinjun ^b   Yang, Yun ^a  

^a SWINBURNE UNIVERSITY OF TECHNOLOGY   (Australia)

^b UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85013812109     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1016/C2011-0-06996-X     Document Type: Book

References (110)

1. Academy of Technology Science and Engineering, Australia. Cloud computing: opportunities and challenges for Australia, [accessed 1.09.01]. http://www.atse.org.au/component/remository/ATSE-Reports/Information-Technology/CLOUD-COMPUTING-Opportunities-and-Challenges-for-Australia-2010/.
2. van der Aalst W.M.P., Hee K.M.V. Workflow management: models, methods, and systems 2002, The MIT Press, Cambridge.
3. van der Aalst W.M.P., Hee K.M.V., Reijers H.A. Analysis of discrete-time Stochastic Petri Nets. Statistica Neerlandica 2000, 54(1):237-255.
4. S. Akioka, Y. Muraoka, Extended forecast of CPU and network load on computational grid. In: Proc. IEEE international symposium on cluster computing and the grid (CCGrid2004). Chicago, IL, USA; 2004. pp. 765-772.
5. M. Armbrust, A. Fox, R. Griffith, A.D. Joseph, R.H. Katz, A. Konwinski, et al. Above the clouds: a Berkeley view of cloud computing. Technical report, University of California at Berkeley, [accessed 1.09.11]. http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf.
6. Behnamian J., Zandieh M., Fatemi Ghomi S.M.T. Parallel-machine scheduling problems with sequence-dependent setup times using an ACO, SA and VNS hybrid algorithm. Expert Syst Appl 2009, 36(6):9637-9644.
7. Blum C., Roli A. Metaheuristics in combinatorial optimization: overview and conceptual comparison. ACM Comput Surv 2003, 35(3):268-308.
8. G. Boss, P. Malladi, D. Quan, L. Legregni, H. Hall, IBM cloud computing (White Paper). Technical report, [accessed 1.09.11]. http://www.download.boulder.ibm.com/ibmdl/pub/software/dw/wes/hipods/Cloud_computing_wp_final_8Oct.pdf.
9. Bucci G., Sassoli L., Vicario E. Correctness verification and performance analysis of real-time systems using stochastic preemptive time petri nets. IEEE Trans Softw Eng 2005, 31(11):913-927.
10. Buhr P.A., Mok W.Y.R. Advanced exception handling mechanisms. IEEE Trans Softw Eng 2000, 26(9):820-836.
11. Buyya R., Bubendorfer K. Market oriented grid and utility computing 2009, Wiley Press, New York, NY.
12. Buyya R., Yeo C.S., Venugopal S., Broberg J., Brandic I. Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comp Syst 2009, 25(6):599-616.
13. Chang R.-S., Chang J.-S., Lin P.-S. An ant algorithm for balanced job scheduling in grids. Future Gener Comp Syst 2009, 25(1):20-27.
14. Chatfield C. The analysis of time series: an introduction 2004, Chapman and Hall/CRC, New York, NY. 6th ed.
15. H. Che-Lun, Y. Don-Lin, C. Yeh-Ching, H. Ming-Chuan, A novel mining algorithm for periodic clustering sequential patterns. In: Proc. 19th international conference on industrial, engineering and other applications of applied intelligent systems (IEA/AIE2006). Annecy, France; 2006. pp. 1299-1308.
16. J. Chen, Towards effective and efficient temporal verification in grid workflow systems, PhD thesis. Swinburne University of Technology, [accessed 1.09.11]. http://researchbank.swinburne.edu.au/vital/access/manager/Repository/swin:7425.
17. Chen J., Yang Y. Localising temporal constraints in scientific workflows. J Comput Syst Sci 2010, 76(6):464-474.
18. Chen J., Yang Y. Multiple states based temporal consistency for dynamic verification of fixed-time constraints in grid workflow systems. Concurrency Comput Pract Exper 2007, 19(7):965-982. [Wiley].
19. Chen J., Yang Y. Activity completion duration based checkpoint selection for dynamic verification of temporal constraints in grid workflow systems. Int J High Perform Comput Appl 2008, 22(3):319-329.
20. Chen J., Yang Y. A taxonomy of grid workflow verification and validation. Concurrency Comput Pract Exper 2008, 20(4):347-360.
21. Chen J., Yang Y. Temporal dependency based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems. ACM Trans Softw Eng Meth 2011, 20(3). [article 9].
22. Chen J., Yang Y. Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems. ACM Trans Autonomous Adaptive Syst 2007, 2(2). [article 6].
23. J. Chen, Y. Yang, Temporal dependency based checkpoint selection for dynamic verification of fixed-time constraints in grid workflow systems, In: Proc. 30th international conference on software engineering (ICSE2008). Leipzig, Germany; 2008. pp. 141-150.
24. J. Chen, Y. Yang, T.Y. Chen, Dynamic verification of temporal constraints on-the-fly for workflow systems, In: Proc. 11th Asia-Pacific software engineering conference. Busan, Korea; 2004. pp. 30-37.
25. Chen T.Y., Merkel R. An upper bound on software testing effectiveness. ACM Trans Softw Eng Meth 2008, 17(3):1-27.
26. W-N. Chen, J. Zhang, Y. Yu, Workflow scheduling in grids: an ant colony optimization approach. In: Proc. 2007 IEEE congress on evolutionary computation (CEC2007). Singapore; 2007. pp. 3308-3315.
27. W-N. Chen, J. Zhang, An ant colony optimization approach to a grid scheduling problem with various QoS requirements, IEEE Trans Systems, Man and Cybernetics, Part C: Applications and Reviews, 39 (1) (2009) 29-43.
28. Choudhury P., Kumar R., Chakrabarti P.P. Hybrid scheduling of dynamic task graphs with selective duplication for multiprocessors under memory and time constraints. IEEE Trans Parallel Distr Syst 2008, 19(7):967-980.
29. E. Commission. The future of cloud computing, opportunities for European cloud computing beyond 2010, [accessed 1.09.11]. http://www.cordis.europa.eu/fp7/ict/%20ssai/docs/cloud-report-final.pdf.
30. K. Cooper, A. Dasgupta, K. Kennedy, C. Koelbel, A. Mandal, New grid scheduling and rescheduling methods in the GrADS project. In: Proc. 18th international parallel and distributed processing symposium (IPDPS2004). Santa Fe, NM, USA; 2004. pp. 199-206.
31. DAGMan, Condor Project, [accessed 1.09.11]. http://www.cs.wisc.edu/condor/.
32. Deelman E., Gannon D., Shields M., Taylor I. Workflows and e-science: an overview of workflow system features and capabilities. Future Gener Comp Syst 2008, 25(6):528-540.
33. Department of Finance and Deregulation, Australian Government: Cloud Computing Strategic Direction Paper, [accessed 1.09.11]. http://www.finance.gov.au/e-government/strategy-andgovernance/cloud-computing.html.
34. Dinda P.A., O'Hallaron D.R. Host load prediction using linear models. Cluster Comput 2000, 3(4):265-280.
35. Dobber M., van der Mei R., Koole G. A prediction method for job runtimes on shared processors: survey, statistical analysis and new avenues. Performance Evaluation 2007, 64(7-8):755-781.
36. M. Dobber, R. van der Mei, G. Koole, Statistical properties of task running times in a global-scale grid environment. In: Proc. 6th IEEE international symposium on cluster computing and the grid (CCGRID06). Singapore; 2006. vol. 1, pp. 150-153.
37. J. Eder, E. Panagos, M. Rabinovich, Time constraints in workflow systems. In: Proc. 11th international conference on advanced information systems engineering (CAiSE99). Heidelberg, Germany; 1999. pp. 286-300.
38. Erl T. SOA: principles of service design 2008, Prentice Hall, Boston, MA.
39. B. Fechner, U. Honig, J. Keller, W. Schiffmann, Fault-tolerant static scheduling for grids. In: Proc. 2008 IEEE international parallel and distributed processing symposium (IPDPS2008). Miami, FL, USA; 2008. pp. 1-6.
40. S. Ferretti, V. Ghini, F. Panzieri, M. Pellegrini, E. Turrini, QoS-aware clouds. In: Proc. IEEE 3rd international conference on cloud computing; 2010. pp. 321-328.
41. I. Foster, Z. Yong, I. Raicu, S. Lu, Cloud computing and grid computing 360-degree compared. In: Proc. 2008 grid computing environments workshop (GCE08). Austin, TX, USA; 2008. pp. 1-10.
42. The Apache Software Foundation, MapReduce, ; mapreduce/ [accessed 1.11.10]. http://www.hadoop.apache.org/.
43. Fu-Lai C., Tak-Chung F., Ng V., Luk R.W.P. An evolutionary approach to pattern-based time series segmentation. IEEE Trans Evolut Comput 2004, 8(5):471-489.
44. Glasner C., Volkert J. Adaps - a three-phase adaptive prediction system for the run-time of jobs based on user behaviour. J Comput Syst Sci 2010, [in press]. 10.1016/j.jcss.2010.01.005.
45. Hadoop, [accessed 1.09.11]. http://www.hadoop.apache.org/.
46. Hagen C., Alonso G. Exception handling in workflow management systems. IEEE Trans Softw Eng 2000, 26(10):943-958.
47. Han J.W., Kamber M. Data mining: concepts and techniques 2006, Elsevier, San Francisco, CA 94111. 2nd ed.
48. HPGC, Proc. 6th high-performance grid computing workshop in conjunction with 23rd parallel and distributed processing symposium (IPDPS09). Rome, Italy; 2009.
49. Huang Y.-P., Hsu C.-C., Wang S.-H. Pattern recognition in time series database: a case study on financial database. Expert Syst Appl 2007, 33(1):199-205.
50. Hwang S.Y., Wang H., Tang J., Srivastava J. A probabilistic approach to modelling and estimating the QoS of web-services-based workflows. Inform Sci 2007, 177(23):5484-5503.
51. IBM, Understanding quality of service for web services, [accessed 1.09.11]. http://www.ibm.com/developerworks/library/ws-quality.html.
52. Kao B., Garcia-Molina H. Deadline assignment in a distributed soft real-time system. IEEE Trans Parallel Distr Syst 1997, 8(12):1268-1274.
53. E. Keogh, S. Chu, D. Hart, M. Pazzani, An online algorithm for segmenting time series. In: Proc. 2001 IEEE international conference on data mining (ICDM2001), San Jose, CA, USA; 2001. pp. 289-296.
54. Law A.M., Kelton W.D. Simulation modelling and analysis 2007, McGraw-Hill, New York, NY. 4th ed.
55. Liangzhao Z., Benatallah B., Ngu A.H.H., Dumas M., Kalagnanam J., Chang H. QoS-aware middleware for web services composition. IEEE Trans Softw Eng. 2004, 30(5):311-327.
56. SAP Library, Workflow System Administration, ; saphelp_nw2004s/helpdata/en [accessed 1.09.11]. http://www.help.sap.com/.
57. L. Yang, I. Foster, J.M. Schopf, Homeostatic and tendency-based CPU load predictions, In: Proc. 2003 international parallel and distributed processing symposium (IPDPS2003). Nice, France; 2003.
58. Liu K., Jin H., Chen J., Liu X., Yuan D., Yang Y. A compromised-time-cost scheduling algorithm in SwinDeW-C for instance-intensive cost-constrained workflows on cloud computing platform. Inter J High Perform Comput Appl 2010, 24(4):445-456.
59. X. Liu, J. Chen, K. Liu, Y. Yang, Forecasting duration intervals of scientific workflow activities based on time-series patterns, In: Proc. 4th IEEE international conference on e-science (e-Science08). Indianapolis, IN, USA; 2008. pp. 23-30.
60. X. Liu, J. Chen, Z. Wu, Z. Ni, D. Yuan, Y. Yang, Handling recoverable temporal violations in scientific workflow systems: a workflow rescheduling based strategy, In: Proc. 10th IEEE/ACM international symposium on cluster, cloud and grid computing (CCGRID10). Melbourne, Australia; 2010. pp. 534-537.
61. X. Liu, J. Chen, Y. Yang, A probabilistic strategy for setting temporal constraints in scientific workflows. In: 6th international Conference on business process management (BPM08). Milan, Italy; 2008. pp. 180-195.
62. X. Liu, Z. Ni, J. Chen, Y. Yang, A probabilistic strategy for temporal constraint management in scientific workflow systems. Concurrency and computation: practice and experience [online], [accessed 1.09.11]. http://www.onlinelibrary.wiley.com/doi/10.1002/cpe.1739/pdf.
63. X. Liu, Z. Ni, Z. Wu, D. Yuan, J. Chen, Y. Yang, An effective framework of light-weight handling for three-level fine-grained recoverable temporal violations in scientific workflows. In: Proc. 16th IEEE international conference on parallel and distributed systems (ICPADS2010). Shanghai, China; 2010. pp. 43-50.
64. Liu X., Ni Z., Wu Z., Yuan D., Chen J., Yang Y. A novel general framework for automatic and cost-effective handling of recoverable temporal violations in scientific workflow systems. J Syst Softw 2011, 84(3):492-509. [Elsevier].
65. Liu X., Ni Z., Yuan D., Jiang Y., Wu Z., Chen J., et al. A novel statistical time-series pattern based interval forecasting strategy for activity durations in workflow systems. J Syst Softw. 2011, 84(3):354-376. [Elsevier].
66. X. Liu, Y. Yang, J. Chen, Q. Wang, M. Li, Achieving on-time delivery: a two-stage probabilistic scheduling strategy for software projects. In: Proc. 2009 international conference on software process: trustworthy software development processes. Vancouver, Canada; 2009.
67. Liu X., Yang Y., Jiang Y., Chen J. Preventing temporal violations in scientific workflows: where and how. IEEE Trans Softw Eng 2010, [online].
68. Liu X., Yuan D., Zhang G., Chen J., Yang Y. SwinDeW-C: a peer-to-peer based cloud workflow system. Handbook of cloud computing 2010, 309-332. Springer, New York, USA. B. Furht, A. Escalante (Eds.).
69. Marjanovic O., Orlowska M.E. On modelling and verification of temporal constraints in production workflows. Know Inform Syst 1999, 1(2):157-192.
70. Martinez A., Alfaro F.J., Sanchez J.L., Quiles F.J., Duato J. A new cost-effective technique for QoS support in clusters. IEEE Trans Parallel Distr Syst 2007, 18(12):1714-1726.
71. Moore M. An accurate parallel genetic algorithm to schedule tasks on a cluster. Parallel Comput 2004, 30:567-583.
72. Nadarajah S., Kotz S. Exact distribution of the max/min of two Gaussian random variables. IEEE Trans Very Large Scale Integration Syst 2008, 16(2):210-212.
73. F. Nadeem, T. Fahringer, Using templates to predict execution time of scientific workflow applications in the grid. In: Proc. 2009 IEEE/ACM international symposium on cluster computing and the grid (CCGRID09). Shanghai, China; 2009. pp. 316-323.
74. F. Nadeem, M.M. Yousaf, R. Prodan, T. Fahringer, Soft benchmarks-based application performance prediction using a minimum training set. In: Proc. 2nd IEEE international conference on e-science and grid computing (e-Science06). Amsterdam, Netherlands; 2006.
75. Oh J., Wu C. Genetic-algorithm-based real-time task scheduling with multiple goals. J Syst Softw 2004, 71:245-258.
76. PDSEC. Proc. 10th IEEE international workshop on parallel and distributed scientific and engineering computing, in conjunction with 23rd parallel and distributed processing symposium (IPDPS09). Rome, Italy; 2009.
77. Prodan R., Fahringer T. Overhead analysis of scientific workflows in grid environments. IEEE Trans Parallel Distr Syst 2008, 19(3):378-393.
78. Askalon Project, [accessed 1.09.11]. http://www.dps.uibk.ac.at/projects/askalon.
79. CROWN Project, CROWN portal, [accessed 1.09.11]. http://www.crown.org.cn/en/.
80. GridBus Project, [accessed 1.09.11]. http://www.gridbus.org.
81. JOpera Project, [accessed 1.09.11]. http://www.iks.ethz.ch/jopera.
82. Kepler Project, [accessed 1.09.11]. http://www.kepler-project.org/.
83. Triana Project, [accessed 1.09.11]. http://www.trianacode.org/.
84. Taverna Project, [accessed 1.09.11]. http://www.mygrid.org.uk/tools/taverna/.
85. UNICORE Project, [accessed 1.09.11]. http://www.unicore.eu/.
86. K. Ren, X. Liu, J. Chen, N. Xiao, J. Song, W. Zhang, A QSQL-based efficient planning algorithm for fully-automated service composition in dynamic service environments. In: Proc. 2008 IEEE international conference on services computing (SCC08). Honolulu, HI, USA; 2008. pp. 301-308.
87. N. Russell, W.M.P. van der Aalst, A.H.M. Ter Hofstede, Exception handling patterns in process-aware information systems. Technical report BPM-06-04, ; 2006. http://www.workflowpatterns.com/documentation/documents/BPM-06-04.pdf.
88. N. Russell, W.M.P. van der Aalst, A.H.M. ter Hofstede, Workflow exception patterns. In: Proc. 18th international conference on advanced information systems engineering (CAiSE06). Luxembourg; 2006. pp. 288-302.
89. SECES, Proc. 1st international workshop on software engineering for computational science and engineering, in conjunction with 30th international conference on software engineering (ICSE2008). Leipzig, Germany; 2008.
90. Smith W., Foster I., Taylor V. Predicting application run times with historical information. J Parallel Distr Comput 2004, 64(9):1007-1016.
91. Son J.H., Sun Kim J., Kim M.Ho Extracting the workflow critical path from the extended well-formed workflow schema. J Comput Syst Sci 2005, 70(1):86-106.
92. Stroud K.A. Engineering mathematics 2007, Palgrave Macmillan, Basingstoke. 6th ed.
93. Taylor I.J., Deelman E., Gannon D.B., Shields M. Workflows for e-science: scientific workflows for grids 2007, Springer, London.
94. S. Tirthapura, B. Xu, C. Busch, Sketching asynchronous streams over a sliding window. In: Proc. 25th annual ACM symposium on principles of distributed computing (PODC06). Denver, CO, USA; 2006. pp. 82-91.
95. Tracy D.B., Howard J.S., Noah B. Comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J Parallel Distr Comput 2001, 61(6):810-837.
96. VMware, [accessed 1.09.11]. http://www.vmware.com/.
97. Wang L., Jie W., Chen J. Grid computing: infrastructure, service, and applications 2009, CRC Press, Taylor &Francis Group, Boca Raton, FL 33487.
98. R. Wolski, Forecasting network performance to support dynamic scheduling using the network weather service. In: Proc. 6th IEEE international symposium on high performance distributed computing (HPDC97). Portland, OR, USA; 1997. pp. 316-325.
99. Wu Z., Liu X., Ni Z., Yuan D., Yang Y. A market-oriented hierarchical scheduling strategy in cloud workflow systems. J Supercomput 2011, [online] [accessed 1.09.11].
100. Y. Yang, K. Liu, J. Chen, J. Lignier, H. Jin, Peer-to-peer based grid workflow runtime environment of SwinDeW-G. In: Proc. 3rd international conference on e-science and grid computing (e-Science07). Bangalore, India; 2007. pp. 51-58.
101. Y. Yang, K. Liu, J. Chen, X. Liu, D. Yuan, H. Jin, An algorithm in SwinDeW-C for scheduling transaction-intensive cost-constrained cloud workflows. In: Proc. 4th IEEE international conference on e-science (e-Science08). Indianapolis, IN, USA; 2008. pp. 374-375.
102. W. Yongwei, Y. Yulai, Y. Guangwen, Z. Weimin, Load prediction using hybrid model for computational grid. In: Proc. 8th IEEE/ACM international conference on grid computing (Grid07). Austin, TX, USA; 2007. pp. 235-242.
103. Yu J., Buyya R. A taxonomy of workflow management systems for grid computing. J Grid Comput 2005, (3):171-200.
104. Yu J., Buyya R. Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms. Scientific Program 2006, 14(3):217-230.
105. Yu J., Buyya R. Workflow scheduling algorithms for grid computing. Metaheuristics for scheduling in distributed computing environments 2008, Springer, Berlin. F. Xhafa, A. Abraham (Eds.).
106. Z. Yu, W. Shi, An adaptive rescheduling strategy for grid workflow applications. In: Proc. 2007 IEEE international symposium on parallel and distributed processing (IPDPS2007). Long Beach, CA, USA; 2007; pp. 115-122.
107. L. Zhang, Y. Chen, B. Yang, Task scheduling based on PSO algorithm in computational grid. In: Proc. 6th international conference on intelligent systems design and applications (ISDA2006). Jinan, China; 2006. vol. 2, pp. 696-701.
108. Zhang Y., Sun W., Inoguchi Y. Predict task running time in grid environments based on cpu load predictions. Future Gener Comput Syst 2008, 24(6):489-497.
109. Y.Y. Zhang, W. Sun, Y. Inoguchi, Predicting running time of grid tasks based on CPU load predictions. In: Proc. 2006 IEEE international conference on grid computing. Barcelona, Spain; 2006. pp. 286-292.
110. Zhuge H., Cheung T., Pung H. A timed workflow process model. J Syst Softw 2001, 55(3):231-243.