DDS: A deadlock detection-based scheduling algorithm for workflow computations in HPC systems with storage constraints

Parallel Computing

Volumn 39, Issue 8, 2013, Pages 291-305

  Wang, Yang ^a   Lu, Paul ^b  

^a UNIVERSITY OF NEW BRUNSWICK   (Canada)

^b UNIVERSITY OF ALBERTA   (Canada)

Author keywords

Computational workflow; Concurrency; Deadlock detection; Storage resource constraints; Workflow scheduling

Indexed keywords

COMPUTATIONAL WORKFLOW; CONCURRENCY; DEADLOCK DETECTION; STORAGE RESOURCES; WORKFLOW SCHEDULING;

ALGORITHMS; SCHEDULING;

DIGITAL STORAGE;

EID: 84878839973     PISSN: 01678191     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.parco.2013.04.006     Document Type: Article

References (34)

1. T. Werner Target gene identification from expression array data by promoter analysis Biomolecular Engineering 17 2001 87 94 (Pubitemid 32140951)
2. D. Szafron, P. Lu, R. Greiner, D. Wishart, B. Poulin, R. Eisner, Z. Lu, J. Anvik, C. Macdonell, A. Fyshe, and D. Meeuwis Proteome analyst: custom predictions with explanations in a web-based tool for high-throughput proteome annotations Nucleic Acids Research 32 2004 W365 W371
3. GROMACS, , access date: Oct. 2, 2012.
4. M. Schmidt, K. Baldridge, J. Boatz, S. Elbert, M. Gordon, J. Jensen, S. Koseki, N. Matsunaga, and J. Montgomery The general atomic and molecular electronic structure system Journal of Computational Chemistry 14 1993 1347 1363
5. B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E.A. Lee, J. Tao, and Y. Zhao Scientific workflow management and the kepler system: research articles Concurrency and Computation: Practice and Experience 18 10 2006 1039 1065 (Pubitemid 44258497)
6. K. Srimanotham, V. Muangsin, Scheduling workflow-based parameter-sweep applications with best-intermediate-result-first heuristic, in: IEEE International Conference on Cluster Computing, Los Alamitos, CA, USA, 2006, pp. 1-6.
7. S. Smanchat, M. Indrawan, S. Ling, C. Enticott, D. Abramson, Scheduling multiple parameter sweep workflow instances on the grid, in: Proceedings of the Fifth IEEE International Conference on e-Science, E-SCIENCE '09, 2009, pp. 300-306.
8. D. Abramson, B. Bethwaite, C. Enticott, S. Garic, and T. Peachey Parameter exploration in science and engineering using many-task computing IEEE Transactions on Parallel and Distributed Systems 22 2011 960 973
9. S. Djorgovski, R. Gal, S. Odewahn, R. de Carvalho, R. Brunner, G. Longo, and R. Scaramella The digital palomar sky survey (dposs) Wide Field Surveys in Cosmology 1 1998 10 20
10. B. Barish, R. Weiss, Ligo and the detection of gravitational waves, Physics Today 52, October 1999, p. 44.
11. C. Vecchiola, S. Pandey, R. Buyya, High-performance cloud computing: a view of scientific applications, in: Proceedings of the 10th International Symposium on Pervasive Systems, Algorithms and Networks, Kaohsiung, Taiwan, 2009.
12. L. Ramakrishnan, P.T. Zbiegel, S. Campbell, R. Bradshaw, R.S. Canon, S. Coghlan, I. Sakrejda, N. Desai, T. Declerck, and A. Liu Magellan: experiences from a science cloud Proceedings of the Second International Workshop on Scientific Cloud Computing, ScienceCloud '11 2011 ACM New York, NY, USA 49 58
13. L. Wang, J. Tao, M. Kunze, A. Castellanos, D. Kramer, W. Karl, Scientific cloud computing: early definition and experience, in: Proceedings of the 10th IEEE International Conference on High Performance Computing and Communications, Dalian, China, 2008.
14. J. Gray, D. Liu, M. Nieto-Santisteban, A.S. Szalay, D. DeWitt, G. Heber, Scientific data management in the coming decade, Tech. Rep. MSR-TR-2005-10, Microsoft Corporation, 2005.
15. S. Pandey, R. Buyya, Scheduling of scientific workflows on data grids, in: Proceedings of the Eighth IEEE International Symposium on Cluster Computing and the Grid, Washington, DC, USA, 2008, pp. 548-553.
16. A. Ramakrishnan, G. Singh, H. Zhao, E. Deelman, R. Sakellariou, K. Vahi, K. Blackburn, D. Mayers, M. Samidi, Scheduling data-intensive workflows onto storage-constrained distributed resources, in: Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid, 2007, pp. 401-409. (Pubitemid 47334578)
17. W. Zhang, J. Cao, Y. Zhong, L. Liu, C. Wu, An integrated resource management and scheduling system for grid data streaming applications, in: Proceedings of the 2008, Ninth IEEE/ACM International Conference on Grid Computing, GRID '08, Washington, DC, USA, 2008, pp. 258-265.
18. J. Bent, D. Thain, A. Arpaci-Dusseau, R.H. Arpaci-Dusseau, M. Livny, Explicit control in a batch-aware distributed file system, in: Proceedings of Networked Systems Design and Implementation (NSDI), San Francisco, California, USA, 2004, pp. 365-378.
19. J. Bent, T.E. Denehy, M. Livny, A.C. Arpaci-Dusseau, R.H. Arpaci-Dusseau, Data-Driven Batch Scheduling, in: Data-Aware Distributed Computing 2009 (DADC09), 2009, Munich, Germany.
20. W. Zhang, J. Cao, Y. Zhong, L. Liu, C. Wu, Block-based concurrent and storage-aware data streaming for grid applications with lots of small files, in: Proceedings of the 2009, Ninth IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID '09, Washington, DC, USA, 2009, pp. 538-543.
21. A. Tanenbaum Modern Operating Systems 2001 Prentice Hall
22. J. Wang, H. Kuehl, and M. Sacchi Least-squares wave-equation AVP imaging of 3D common azimuth data Proceedings of the 73rd Annual International Meeting 2003 Society of Exploration Geophysicists
23. P. Blaha, K. Schwarz, G. Madsen, D. Kvasnicka, J. Luitz, WIEN2k: an augmented plane wave plus local orbitals program for calculating crystal properties, Tech. rep., Institute of Physical and Theoretical Chemistry, Vienna University of Technology, 2001.
24. R. Lake, J. Schaeffer, and P. Lu Solving large retrograde analysis problems using a network of workstations Advances in Computer Chess VII 1994 135 162
25. J. Schaeffer, and R. Lake Solving the game of checkers R.J. Nowakowski, Games of No Chance vol. 20 1996 Cambridge University Press
26. A. Rosenberg On scheduling mesh-structured computations for internet-based computing IEEE Transactions on Computers 53 9 2004 1176 1186
27. T. Glatard, J. Montagnat, X. Pennec, Grid-enabled workflows for data intensive medical applications, in: 18th IEEE Symposium on Computer-Based Medical Systems, Trinity College Dublin, Ireland, 2005, pp. 537-542. (Pubitemid 41542951)
28. P. Crandall, R. Aydt, A. Chien, D. Reed, Input/output characteristics of scalable parallel applications, in: Proceedings of the IEEE/ACM Conference on Supercomputing, San Diego, California, USA, 1995, pp. 59-89.
29. P. Hulith, The AMANDA experiment, in: Proceedings of the XVII International Conference on Neutrino Physics and Astrophysics, Helsinki, Finland, 1996.
30. A. Sum, J. de Pablo, Nautilus: molecular simulation code, Tech. rep., University of Wisconsin-Madison, Department of Chemical Engineering (2002).
31. P. Gburzynski, SMURPH, , access date: Oct. 2, 2012.
32. E. Coffman Computer and Job-Shop Scheduling Theory 1976 Wiley New York
33. S. Venugopal, and R. Buyya A set coverage-based mapping heuristic for scheduling distributed data-intensive applications on global grids Proceedings of the Seventh IEEE/ACM International Conference on Grid Computing(Grid06) 2006 IEEE CS Press
34. S.-D. Lang An extended Banker's algorithm for deadlock avoidance IEEE Transactions on Software Engineering 25 3 1999 428 432