The design of cloud workflow systems

SpringerBriefs in Computer Science

Volumn 0, Issue 9781461419327, 2012, Pages 1-97

  Liu, Xiao ^a   Yuan, Dong ^a   Zhang, Gaofeng ^a   Li, Wenhao ^a   Cao, Dahai ^a   He, Qiang ^a   Chen, Jinjun ^a,b   Yang, Yun ^a  

^a SWINBURNE UNIVERSITY OF TECHNOLOGY   (Australia)

^b UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTERS;

CLOUD WORKFLOW;

COMPUTER SCIENCE;

EID: 84881386144     PISSN: 21915768     EISSN: 21915776     Source Type: Book Series    
DOI: None     Document Type: Chapter

References (92)

1. Australian Academy of Technology Science and Engineering, Cloud Computing: Opportunities and Challenges for Australia. http://www.atse.org.au/component/remository/ATSE-Reports/Information-Technology/CLOUD-COMPUTING-Opportunities-and-Challenges-for-Australia-2010/. Accessed 1 Aug 2011
2. Aalst, W.M.P., van der Hee, K.M.V.: Workflow Management: Models, Methods and Systems. The MIT Press, Cambridge (2002)
3. Alexandru, I.: Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans. Parallel Distributed Syst. 22(6), 931–945 (2011)
4. Deutsch A, Papakonstantinou Y, Privacy in database publishing. In: 10th International Conference on Database Theory, pp. 230–245 (2005)
5. Alonso, G., Günthör, R., Kamath, M., Agrawal, D., El Abbadi, A., Mohan, C.: Exotica/ FMDC: a workflow management system for mobile and disconnected clients. Distributed Parallel Databases 4, 229–247 (1996)
6. Pereira, A.L., Muppavarapu, V., Chung S.M.,: Role-based access control for grid database services using the community authorization service. IEEE Trans. Dependable Secure Comput. 3, 156–166 (2006)
7. Ardagna, D., Pernici, B.: Adaptive service composition in flexible processes. IEEE Trans. Softw. Eng. 33, 369–384 (2007)
8. Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R.H., Konwinski, A., Lee, G., Patterson, D.A., Rabkin, A., Stoica, I., Zaharia, M.: Above the clouds: a Berkeley view of cloud computing. Technical report, UCB/EECS-2009-28, University of California, Berkeley (2009)
9. Assuncao, M.D.d., Costanzo, A.d., Buyya, R.: Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters. In: 18th ACM International Symposium on High Performance Distributed Computing, pp. 1–10 (2009)
10. Australian Government Department of Finance and Deregulation: Cloud Computing Strategic Direction Paper. http://www.finance.gov.au/e-government/strategy-and-governance/cloud-computing.html. Accessed 1 Aug 2011
11. Australian Government, Department of the Environment, Water, Heritage and the Arts: Australian Government ICT Sustainability Plan. http://www.environment.gov.au/sustainability/government/ictplan/publications/plan/index.html. Accessed 1 Aug 2011
12. Barr J., Varia J., Wood M.: Animoto - Scaling Through Viral Growth. http://aws.typepad. com/aws/2008/04/animoto—scali.html. Accessed 1 Aug 2011
13. Bhargav-spantzel, A., Squicciarini, A.C., Bertino, E.: Trust negotiation in identity management. IEEE Security Privacy 5, 55–63 (2007)
14. Borthakur, D.: The hadoop distributed file system: architecture and design. http://hadoop. apache.org/common/docs/r0.18.3/hdfs_design.html. Accessed 1 Aug 2011
15. Bose, R., Frew, J.: Lineage retrieval for scientific data processing: a survey. ACM Comput. Surv. 37, 1–28 (2005)
16. Boss, G., Malladi, P., Quan, D., Legregni, L., Hall, H.: IBM cloud computing (White Paper) (2007)
17. Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandic, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25, 599–616 (2009)
18. Cai, T., Gloor, P.A., Nog, S.: DartFlow: A Workflow Management System on the Web Using Transportable Agents. Dartmouth College, Hanover (1996)
19. Calheiros, R.N., Ranjan, R., Rose, C.A.F.D., Buyya, R.: CloudSim: A Novel Framework for Modeling and Simulation of Cloud Computing Infrastructures and Services. Grid Computing and Distributed Systems (GRIDS) Laboratory, Technical report. Department of Computer Science and Software Engineering, The University of Melbourne (2009)
20. Cardoso, J.: Stochastic workflow reduction algorithm. Technical report, LSDIS Lab, Department of Computer Science, University of Georgia (2002)
21. Chen, J., Yang, Y.: Multiple states based temporal consistency for dynamic verification of fixed-time constraints in grid workflow systems. Concurr. Comput.: Pract. Experience 19, 965–982 (2007)
22. Chen, J., Yang, Y.: Temporal dependency based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems. ACM Trans. Softw. Eng. Methodol. 20(3), article 9 (2011)
23. Chen, J., Yang, Y.: Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems. ACM Trans. Auton. Adapt. Syst. 2, article 6 (2007)
24. Chen, J., Yang, Y.: Temporal dependency based checkpoint selection for dynamic verification of fixed-time constraints in grid workflow systems. In: 30th International Conference on Software Engineering, pp. 141–150 (2008)
25. Lin, C., Varadharajan, V., Wang, Y., Pruthi, V.: Enhancing grid security with trust management. In: 2004 IEEE International Conference on Services Computing, pp. 303–310 (2004)
26. Coulouris, G., Dollimore, J., Kindberg, T.: Distributed Systems: Concepts and Design, 4th edn. Pearson Education Limited, Harlow (2005)
27. Deelman, E., Gannon, D., Shields, M., Taylor, I.: Workflows and e-science: an overview of workflow system features and capabilities. Future Gener. Comput. Syst. 25, 528–540 (2008)
28. Dogac, A., Gokkoca, E., Arpinar, S., Koksal, P., Cingil, I., Arpinar, B., Tatbul, N., Karagoz, P., Halici, U., Altinel, M.: Design and implementation of a distributed workflow management system: METUFlow. In: Doğaç, A., Kalinichenko, L., Özsu, M.T., Sheth, A. (eds.) Workflow Management Systems and Interoperability, pp. 61–91 (1998)
29. Erl, T.: SOA: Principles of Service Design. Prentice Hall, London (2008)
30. European Commission, The Future of Cloud Computing, Opportunities for European Cloud Computing Beyond 2010. http://cordis.europa.eu/fp7/ict/ssai/docs/cloud-report-final.pdf. Accessed 1 Aug 2011
31. Ferretti, S., Ghini, V., Panzieri, F., Pellegrini, M., Turrini, E.: QoS-Aware clouds. In: 3rd IEEE International Conference on Cloud Computing, pp. 321–328 (2010)
32. Foster, I., Yong, Z., Raicu, I., Lu, S.: Cloud computing and grid computing 360-degree compared. In: Grid Computing Environments Workshop, pp. 1–10 (2008)
33. Zhang G., Yang Y., Yuan D., Chen J.: A trust-based noise injection strategy for privacy protection in cloud computing. Software: practice and experience, Wiley, to appear. www.ict.swin.edu.au/personal/yyang/papers/SPE-privacy-2010.pdf. Accessed 1 Aug 2011
34. Gartner: Gartner says worldwide cloud services revenue will grow 21.3 percent in 2009. http://www.gartner.com/it/page.jsp?id=920712. Accessed 1 Aug 2011
35. Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google file system. SIGOPS Oper. Syst. Rev. 37, 29–43 (2003)
36. Gottfrid D.: Self-service, prorated supercomputing fun. http://open.blogs.nytimes.com/2007/11/01/self-service-prorated-super-computing-fun/. Accessed 1 Aug 2011
37. Grundy, J.C., Apperley, M.D., Hosking, J.G., Mugridge, W.B.: A decentralized architecture for software process modeling and enactment. IEEE Internet Comput. 2, 53–62 (1998)
38. Hoffa, C., Mehta, G., Freeman, T., Deelman, E., Keahey, K., Berriman, B., Good, J.: On the use of cloud computing for scientific workflows. In: 4th IEEE International Conference on e-Science, pp. 640–645 (2008)
39. IBM, Understanding quality of service for web services. http://www.ibm.com/developerworks/library/ws-quality.html. Accessed 1 Aug 2011
40. Juve, G., Deelman, E., Vahi, K., Mehta, G., Berriman, B., Berman, P.B., Maechling, P.: Scientific workflow applications on amazon EC2. In: Workshop on Cloud-based Services and Applications in conjunction with 5th IEEE International Conference on e-Science, pp. 59–66 (2009)
41. Kao, B., Garcia-Molina, H.: Deadline assignment in a distributed soft real-time system. IEEE Trans. Parallel Distributed Syst. 8, 1268–1274 (1997)
42. Keahey, K., Figueiredo, R., Fortes, J., Freeman, T., Tsugawa, M.: Science clouds: early experiences in cloud computing for scientific applications. In: First Workshop on Cloud Computing and its Applications, pp. 1–6 (2008)
43. Khodakaram-Salimifard, M.W.: Petri-net based modelling of workflow systems: an overview. Eur. J. Oper. Res. 134, 664–676 (2001)
44. Kondo, D., Javadi, B., Malecot, P., Cappello, F., Anderson, D.P.: Cost-benefit analysis of cloud computing versus desktop grids. In: IEEE International Symposium on Parallel and Distributed Processing, pp. 1-12 (2009)
45. Krutz, R.L., Vines, R.D.: Cloud Security: A Comprehensive Guide to Secure Cloud Computing. Wiley, New York (2010)
46. Liangzhao, Z., Benatallah, B., Ngu, A.H.H., Dumas, M., Kalagnanam, J., Chang, H.: QoS-aware middleware for web services composition. IEEE Trans. Softw. Eng. 30, 311–327 (2004)
47. Liu, K., Jin, H., Chen, J., Liu, X., Yuan, D., Yang, Y.: A compromised-time-cost scheduling algorithm in SwinDeW-C for instance-intensive cost-constrained workflows on cloud computing platform. Int. J. High Perform. Comput. Appl. 24(4), 445–456 (2010)
48. Liu, X., Chen, J., Wu, Z., Ni, Z., Yuan, D., Yang, Y.: Handling recoverable temporal violations in scientific workflow systems: a workflow rescheduling based strategy. In: 10th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 534–537 (2010)
49. Liu, X., Chen, J., Yang, Y.: A probabilistic strategy for setting temporal constraints in scientific workflows. In: 6th International Conference on Business Process Management, pp. 180–195 (2008)
50. Liu, X., Ni, Z., Chen, J., Yang, Y.: A probabilistic strategy for temporal constraint management in scientific workflow systems. Concurr. Comput: Pract. Experience 23(16), 1893–1919 (2011)
51. Liu, X., Ni, Z., Wu, Z., Yuan, D., Chen, J., Yang, Y.: A novel general framework for automatic and cost-effective handling of recoverable temporal violations in scientific workflow systems. J. Syst. Softw. 84(3), 492–509 (2011a)
52. Liu, X., Ni, Z., Yuan, D., Jiang, y., Wu, Z., Chen, J., Yang, Y.: A novel statistical time-series pattern based interval forecasting strategy for activity durations in workflow systems. J. Syst. Softw. 84(3), 354–376 (2011b)
53. Liu, X., Yang, Y., Jiang, Y., Chen, J.: Preventing temporal violations in scientific workflows: where and how. IEEE Trans. Softw. Eng. doi:ieeecomputersociety.org/10.1109/TSE.2010.99. Accessed 1 Aug 2011
54. Liu, X., Yuan, D., Zhang, G., Chen, J., Yang, Y.: SwinDeW-C: a peer-to-peer based cloud workflow system. In: Furht, B., Escalante, A. (eds.) Handbook of Cloud Computing. Springer, Heidelberg (2010b)
55. Ludascher, B., Altintas, I., Berkley, C., Higgins, D., Jaeger, E., Jones, M., Lee, E.A.: Scientific workflow management and the Kepler system. Concurr. Comput: Pract. Experience 18, 1039–1065 (2005)
56. Mark, D.: Ryan: cloud computing privacy concerns on our doorstep. Commun. ACM 54, 36– 38 (2011)
57. McCormick, W.T., Sehweitzer, P.J., White, T.W.: Problem decomposition and data reorganization by a clustering technique. Oper. Res. 20, 993–1009 (1972)
58. Greer, M.B. Jr.: Software as a Service Inflection Point. iUniverse (2009)
59. Moore, M.: An accurate parallel genetic algorithm to schedule tasks on a cluster. Parallel Comput. 30, 567–583 (2004)
60. Moretti, C., Bulosan, J., Thain, D., Flynn, P.J.: All-Pairs: an abstraction for data-intensive cloud computing. In: IEEE International Parallel and Distributed Processing Symposium, pp. 1–11 (2008)
61. Nurmi, D., Wolski, R., Grzegorczyk, C.: Eucalyptus: A technical report on an elastic utility computing archietcture linking your programs to useful systems. UCSB Computer Science Technical Report (2008)
62. Pandey, S., Karunamoorthy, D., Buyya, R.: Workflow engine for clouds. In: Buyya, R., Broberg, J., Goscinski, A.M. (eds.) Cloud Computing: Principles and Paradigms. Wiley, New York (2011)
63. Paul, S., Park, E., Chaar, J.: RainMan: a workflow system for the internet. In: USENIX Symposium on Internet Technologies and System, pp. 15–15 (1997)
64. Poniatowski, M.: Foundations of Green IT. Prentice Hall, London (2010)
65. Raghavan, B., Vishwanath, K.V., Ramabhadran, S., Yocum, K., Snoeren. A.C.: Cloud control with distributed rate limiting. In: Proceedings of 2007 ACM SIGCOMM, pp. 337–348 (2007)
66. Richard, E.B.: A Bayes explanation of an apparent failure rate paradox. IEEE Trans. Reliab. 34, 107–108 (1985)
67. Rosenberg, F., Curbera, F., Duftler, M.J., Khalaf, R.: Composing RESTful services and collaborative workflows—a lightweight approach. IEEE Internet Comput. 12, 24–31 (2008)
68. Russell, N., van der Aalst, W.M.P., Hofstede, A.H.M.t.: Exception handling patterns in process-aware information systems. Technical Report, BPMcenter.org (2006)
69. Russell, N., van der Aalst, W.M.P., Hofstede, A.H.M.t.: Workflow exception patterns. In: 18th International Conference on Advanced Information Systems Engineering, pp. 288–302 (2006)
70. SECES: In: First International Workshop on Software Engineering for Computational Science and Engineering, in conjuction with the 30th International Conference on Software Engineering (2008)
71. Serhani, M.A., Dssouli, R., Hafid, A., Sahraoui, H.: A QoS broker based architecture for efficient Web services selection. In: 2005 IEEE International Conference on Web Services, pp. 113–120 (2005)
72. Simmhan, Y.L., Plale, B., Gannon, D.: A survey of data provenance in e-science. SIGMOD Record 34, pp. 31–36 (2005)
73. Sosinsky, B.: Cloud Computing Bible. Wiley, New York (2010)
74. Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M.: Workflows for e-Science: Scientific Workflows for Grids. Springer, Heidelberg (2007)
75. Vecchiola, C., Chu, X., Buyya, R.: Aneka: A software platform for.NET-based cloud computing. In: 2008 High Performance Computing Workshop, pp. 267–295 (2008)
76. Wang C., Wang Q., Ren K., Lou W.: Ensuring data storage security in cloud computing. In: 17th IEEE International Workshop on Quality of Service, pp. 1–9 (2009)
77. Wang, L., Jie, W., Chen, J. (eds.): Grid Computing: Infrastructure, Service, and Applications. CRC Press, Talyor & Francis Group, Boca Raton (2009)
78. Wang, L.H., Kunze, M., Tao, J.: Performance evaluation of virtual machine-based grid workflow system. Concurr. Comput.: Pract. Experience 20(15), 1759–1771 (2008)
79. Weiss, A.: Computing in the cloud. ACM Networker 11, 18–25 (2007)
80. White, T.: Haddop the Definite Guide. O’Reilly Media, Sebastopol (2009)
81. Winsborough, W.H., Li, N.: Safety in automated trust negotiation. ACM Trans. Inf. Syst. Secur. 9, 352–390 (2006)
82. Workflow Management Coalition: The Workflow Reference Model, WFMC-TC-1003 (1995)
83. Wu, Z., Liu, X., Ni, Z., Yuan, D., Yang, Y.: A market-oriented hierarchical scheduling strategy in cloud workflow systems. J Supercomput. http://dx.doi.org/10.1007/s11227-011-0578-4. Accessed 1 Aug 2011
84. Yan, J., Yang, Y., Raikundalia, G.K.: SwinDeW—a peer-to-peer based decentralized workflow management system. IEEE Trans. Systems, Man and Cybernetics, Part A 36, 922– 935 (2006)
85. Yang, Y., Liu, K., Chen, J., Lignier, J., Jin, H.: Peer-to-peer based grid workflow runtime environment of SwinDeW-G. In: 3rd International Conference on e-Science and Grid Computing, pp. 51–58 (2007)
86. Young, J.W.: A first order approximation to the optimal checkpoint interval. Commun. ACM 17, 530–531 (1974)
87. Yu, J., Buyya, R.: A taxonomy of Workflow Management Systems for Grid Computing. J. Grid Comput. 3(3–4), 171–200 (2005)
88. Yuan, D., Yang, Y., Liu, X., Chen, J.: A cost-effective strategy for intermediate data storage in scientific cloud workflow systems. In: 24th IEEE International Parallel and Distributed Processing Symposium, pp. 1–12 (2010)
89. Yuan, D., Yang, Y., Liu, X., Chen, J.: A local-optimisation based strategy for cost-effective datasets storage of scientific applications in the cloud. In: IEEE International Conference on Cloud Computing, pp. 179–186 (2011)
90. Yuan, D., Yang, Y., Liu, X., Chen, J.:On-demand minimum cost benchmarking for intermediate datasets storage in scientific cloud workflow systems. J. Parallel Distrib. Comput. 72, 316–332 (2011)
91. Yuan, D., Yang, Y., Liu, X., Zhang, G., Chen, J.: A data dependency based strategy for intermediate data storage in scientific cloud workflow systems. Concurr. Comput.: Pract. Experience. http://onlinelibrary.wiley.com/doi/10.1002/cpe.1636/pdf. Accessed 1 Aug 2011
92. Zaharia, M., Konwinski, A., Joseph, A.D., Katz, R., Stoica, I.: Improving MapReduce performance in heterogeneous environments. In: 8th USENIX Symposium on Operating Systems Design and Implementation, pp. 29–42 (2008)