BTS: Resource capacity estimate for time-targeted science workflows

Journal of Parallel and Distributed Computing

Volumn 71, Issue 6, 2011, Pages 848-862

  Byun, Eun Kyu ^a   Kee, Yang Suk ^b   Kim, Jin Soo ^c   Deelman, Ewa ^d   Maeng, Seungryoul ^a  

^a Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^b ORACLE CORPORATION   (United States)

^c Sungkyunkwan University   (South Korea)

^d UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Cloud computing; Distributed system; Resource allocation; Scientific application; Workflow management

Indexed keywords

CRITICAL CHALLENGES; DISTRIBUTED SYSTEMS; FINISH TIME; LOWER BOUNDS; PARALLEL TASK; RESOURCE CAPACITY; RESOURCE DESCRIPTION LANGUAGES; RESOURCE PROVISIONING; SCIENTIFIC APPLICATIONS; TIME SCHEDULING; WORK-FLOWS; WORKFLOW MANAGEMENT; WORKFLOW TECHNOLOGY;

APPROXIMATION ALGORITHMS; COMPUTER SYSTEMS; ESTIMATION; RESOURCE ALLOCATION; TURNAROUND TIME; WORK SIMPLIFICATION;

CLOUD COMPUTING;

EID: 79955639124     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2011.01.008     Document Type: Article

References (50)

1. T.L. Adam, K.M. Chandy, and J.R. Dickson A comparison of list schedules for parallel processing systems Communications of the ACM 17 12 1974 685 690
2. A. Anjonshoaa, F. Brisard, M. Drescher, D. Fellows, A.L. Ans, S. McGough, and D. Pulsipher Job submission description language (JSDL) specification, version 1.0, GFD-R.056 http://forge.gridforum.org/projects/jsdl-wg
3. S. Bharathi, A. Chervenak, E. Deelman, G. Mehta, M.H. Su, K. Vahi, Characterization of scientific workflows, in: Proceedings of the 3rd Workshop on Workflows in Support of Large-Scale Science, 2008.
4. J. Blythe, S. Jain, E. Deelman, T. Gil, K. Vahi, A. Mandal, K. Kennedy, Task scheduling strategies for workflow-based applications in Grids, in: Proceedings of the 5th IEEE International Symposium on Cluster Computing on Grid, 2005.
5. D.A. Brown, P.R. Brady, A. Dietz, J. Cao, B. Johnson, and J. McNabb A case study on the use of workflow technologies for scientific analysis: gravitational wave data analysis Workflows for e-Science 2006 Springer Chapter
6. R. Buyya, M. Murshed, D. Abramson, and S. Venugopal Scheduling parameter sweep applications on global Grids: a deadline and budget constrained cost-time optimization algorithm SoftwarePractice and Experience 35 2005 491 512 (Pubitemid 40479560)
7. K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith, and S. Tuecke A resource management architecture for metacomputing systems IPPS/SPDP'98 Workshop on Job Scheduling Strategies for Parallel Processing Lecture Notes in Computer Science vol. 1459 1998 62 820
8. Condor Team, The directed acyclic graph manager, 2002. http://www.cs.wisc.edu/condor/dagman.
9. E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G.B. Berriman, J. Good, A. Laity, J.C. Jacob, and D. Katz Pegasus: a framework for mapping complex scientific workflows onto distributed systems Scientific Programming Journal 13 3 2005 219 237 (Pubitemid 43022937)
10. F. Dong, S.G. Akl, Scheduling algorithms for Grid computing: state of the art and open problems, Tech. Rep., School of Computing, Queen's University, Kingston, Ontario, January 2006.
11. Amazon elastic compute cloud, Amazon EC2. http://aws.amazon.com/ec2.
12. T. Fahringer, A. Jugravu, S. Pllana, R. Prodan, C. Seragiotto Jr., and H.-L. Truong ASKALON: a tool set for cluster and grid computing Concurrency and Computation: Practice and Experience 17 24 2005
13. Usc epigenomic center. http://epigenome.usc.edu.
14. A. Geras A comparison of clustering heuristics for scheduling directed acyclic graphs on multiprocessors Journal of Parallel and Distributed Computing 16 4 1992 276 291
15. R. Huang, H. Casanova, A.A. Chien, Automatic resource specification generation for resource selection, in: Proceedings of the 20th ACM/IEEE International Conference on High Performance Computing and Communication, 2007.
16. P.M. Institute A Guide to the Project Management Body of Knowledge 3rd ed. 2003 Project Management Institute
17. D. Irwin, J. Chase, L. Grit, A. Yumerefendi, D. Becker, K. Yocumet, Sharing networked resources with brokered leases, in: Proceedings of the USENIX Technical Conference, 2006.
18. J.C. Jacob, D.S. Katz, T. Prince, The montage architecture for grid-enabled science processing of large, distributed datasets, in: Proceedings of the 4th Earth Science Technology Conference, ESTC2004, 2004.
19. Y.-S. Kee, E.-K. Byun, E. Deelman, K. Vahi, J.-S. Kim, Pegasus on the virtual Grid: a case study of workflow planning over captive resources, in: Proceedings of the 3rd Workshop on Workflows in Support of Large-Scale Science, 2008.
20. Y.-S. Kee, C. Kesselman, Grid resource abstraction, virtualization, and provisioning for time-targeted applications, in: Proceedings of the 8th ACM/IEEE International Symposium on Cluster Computing and the Grid, 2008.
21. Y.-S. Kee, D. Logothetis, R. Huang, H. Casanova, A.A. Chien, Efficient resource description and high quality selection for virtual Grids, in: Proceedings of the 5th IEEE International Symposium on Cluster Computing and the Grid, 2005.
22. Y.-S. Kee, K. Yocum, A.A. Chien, H. Casanova, Improving grid resource allocation via integrated selection and binding, in: Proceedings of the 19th ACM/IEEE International Conference on High Performance Computing and Communication, 2006.
23. J. Klaus, and Z. Hu An approximation algorithm for scheduling malleable tasks under general precedence constraints ACM Transactions on Algorithms 2 3 2006 416 434
24. Y.K. Kwok, and I. Ahmad Benchmarking and comparison of the task graph scheduling algorithms Journal of Parallel and Distributed Computing 59 3 1999 381 422 (Pubitemid 129592990)
25. J. Liou, M.A. Palis, CASS: an efficient task management system for distributed memory architectures, in: Proceedings of the 11th International Symposium on Parallel Processing, 1997.
26. J. Livny, H. Teonadi, M. Livny, and M.K. Waldor High-throughput, kingdom-wide prediction and annotation of bacterial non-coding RNAs PLoS One 3 9 2008
27. S.J. Ludtke, P.R. Baldwin, and W. Chiu EMAN: semiautomated software for high-resolution single-particle reconstructions Journal of Structural Biology 128 1999 82 97 (Pubitemid 30013436)
28. H. Magistrale, S. Day, R.W. Clayton, and R. Graves The SCEC southern California reference three-dimensional seismic velocity model version 2 Bulletin of the Seismological Society of America 90 2000 S65 S76
29. D.A. Menasce, E. Casalicchio, A framework for resource allocation in Grid computing, in: Proceedings of the 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications System, 2004.
30. T. N'Takp, F. Suter, Critical path and area based scheduling of parallel task graphs on heterogeneous platforms, in: Proceedings of the 12th International Conference on Parallel and Distributed Systems, 2001.
31. D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, D. Zagorodnov, The eucalyptus open-source cloud-computing system, in: Proceedings of the 1st Workshop on Cloud Computing and its Applications, 2008.
32. S. Pandey, W. Voorsluys, M. Rahman, R. Buyya, J. Dobson, and K. Chiu A Grid workflow environment for brain imageing analysis on distributed systems Concurrency and Computation: Practice and Experience 21 16 2009 2118 2139
33. B. Plale CASA and LEAD: adaptive cyberinfrastructure for real-time multiscale weather forecasting IEEE Computer 39 2006 56 64
34. A. Radulescu, A. van Gemund, A low-cost approach towards mixed task and data parallel scheduling, in: Proceedings of the 2001 International Conference on Parallel Processing, 2001.
35. M. Rahman, S. Venugopal, R. Buyya, A dynamic critical path algorithm for scheduling scientific workflow applications on global Grids, in: Proceedings of the 3rd IEEE International Conference on e-Science and Grid Computing, 2007.
36. R. Raman, M. Livny, M. Solomon, Matchmaking: distributed resource management for high throughput computing, in: Proceedings of the 16th IEEE International Symposium on High Performance Distributed Computing, 1998.
37. S. Ramaswamy, S. Sapatnekar, and P. Banerjee A framework for exploiting task and data parallelism on distributed memory multicomputers IEEE Transactions on Parallel and Distributed Systems 8 11 1997 1098 1116 (Pubitemid 127763322)
38. S. Ranaweera, D.P. Agrawal, A task duplication based scheduling algorithm for heterogeneous systems, in: Proceedings of 14th International Parallel and Distributed Processing Symposium, 2000.
39. Amazon simple storage service, Amazon s3. http://aws.amazon.com/s3/.
40. R. Sakellariou, H. Zhao, A hybrid heuristic for DAG scheduling on heterogeneous systems, in: Proceedings of the 18th International Conference on Parallel and Distributed Processing Symposium, 2004.
41. G.C. Shih, and E.A. Lee A compile-tim scheduling heuristics for interconnection-constrained heterogeneous processor architecture IEEE Transactions on Parallel and Distributed Systems 4 2 1993 75 87
42. G. Singh, C. Kesselman, E. Deelman, Application-level resource provisioning on the grid, in: Proceedings of the 3rd IEEE International Conference on e-Science and Grid Computing, 2006.
43. A. Sudarsanam, M. Srinivasan, S. Panchanathan, Resource estimation and task scheduling for multithreaded reconfigurable architectures, in: Proceedings of the 10th International Conference on Parallel and Distributed Systems, 2004.
44. I. Taylor, M. Shields, I. Wang, and A. Harrison Visual grid workflow in triana Journal of Grid Computing 3 34 2005 153 169 (Pubitemid 43280300)
45. H. Topcuoglu, S. Hariri, and M. Wu Performance-effective and low-complexity task scheduling for heterogeneous computing IEEE Transactions on Parallel and Distributed Systems 13 3 2002 260 274 (Pubitemid 34448780)
46. M. Wieczorek, S. Podlipnig, R. Prodan, T. Fahringer, Bi-criteria scheduling of scientific workflows for the Grid, in: Proceedings of the 8th ACM/IEEE International Symposium on Cluster Computing and the Grid, 2008.
47. T. Yang, and A. Gerasoulis DSC: scheduling parallel tasks on an unbounded number of processors IEEE Transactions on Parallel and Distributed Systems 5 9 1994 951 967
48. J. Yu, and R. Buyya Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms Scientific Programming Journal 14 34 2006 217 230 (Pubitemid 46474559)
49. J. Yu, R. Buyya, and K. Ramanohanarao Metaheuristics for Scheduling in Distributed Computing Environments 2008 Springer Berlin, Germany Chapter, Workflow scheduling algorithms for Grid computing
50. J. Yu, R. Buyya, C.K. Tham, Cost-based scheduling of scientific workflow applications on utility Grids, in: Proceedings of the 1st IEEE International Conference on e-Science and Grid Computing, 2005.