Making work queue cluster-friendly for data intensive scientific applications

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2013, Pages

  Albrecht, Michael ^a   Rajan, Dinesh ^a   Thain, Douglas ^a  

^a University of Notre Dame   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA-INTENSIVE APPLICATION; HIERARCHICAL CONFIGURATION; MASTER-WORKER APPLICATIONS; MULTIPLE CONFIGURATIONS; PHYSICAL ARCHITECTURE; SCIENTIFIC APPLICATIONS; SOFTWARE AND HARDWARES; SYNTHETIC APPLICATION;

CLUSTER COMPUTING; MIDDLEWARE; MOLECULAR DYNAMICS; NETWORK ARCHITECTURE; PHYSICAL ADDRESSES;

REDUCTION;

EID: 84893571544     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTER.2013.6702628     Document Type: Conference Paper

References (11)

1. B. Abdul-Wahid, L. Yu, D. Rajan, H. Feng, E. Darve, D. Thain, and J. A. Izaguirre. Folding Proteins at 500 ns/hour with Work Queue. In 8th IEEE International Conference on eScience (eScience 2012), 2012.
2. M. Albrecht, P. Donnelly, P. Bui, and D. Thain. Makeflow: a Portable Abstraction for Data Intensive Computing on Clusters, Clouds, and Grids. In Workshop on Scalable Workflow Enactment Engines and Technologies (SWEET) at ACM SIGMOD, 2012.
3. Z. Balaton, G. Gombás, P. Kacsuk, A. Kornafeld, J. Kovács, A. Marosi, G. Vida, N. Podhorszki, and T. Kiss. Sztaki desktop grid: a modular and scalable way of building large computing grids. In Parallel and Distributed Processing Symposium, 2007. IPDPS 2007. IEEE International, pages 1-8. ieee, 2007.
4. J. Berthold, M. Dieterle, R. Loogen, and S. Priebe. Hierarchical masterworker skeletons. Practical Aspects of Declarative Languages, pages 248-264, 2008.
5. D. Borthakur. The hadoop distributed file system: Architecture and design. http://hadoop. apache.org/, 2007.
6. P. Bui, D. Rajan, B. Abdul-Wahid, J. Izaguirre, and D. Thain. Work Queue + Python: a Framework For Scalable Scientific Ensemble Applications. In Workshop on Python for High Performance and Scientific Computing (PyHPC) at the ACM/IEEE International Conference for High Performance Computing, Networking, Storage, and Analysis (Supercomputing), 2011.
7. D. Caromel, C. Delbé, A. Di Costanzo, M. Leyton, et al. Proactive: an integrated platform for programming and running applications on grids and p2p systems. Computational Methods in Science and Technology, 12, 2006.
8. M. Di Santo, F. Frattolillo, N. Ranaldo, W. Russo, and E. Zimeo. Programming metasystems with active objects. In Proceedings of the 5th International Workshop on Java for Parallel and Distributed Computing, 2003.
9. M. Di Santo, F. Frattolillo, W. Russo, and E. Zimeo. a component-based approach to build a portable and flexible middleware for metacomputing. Parallel Computing, 28(12):1789-1810, 2002. (Pubitemid 35412377)
10. C. Moretti, A. Thrasher, L. Yu, M. Olson, S. Emrich, and D. Thain. A Framework for Scalable Genome Assembly on Clusters, Clouds, and Grids. IEEE Transactions on Parallel and Distributed Systems, 23(12), 2012.
11. S. Priebe. Dynamic task generation and transformation within a nestable workpool skeleton. Euro-Par 2006 Parallel Processing, pages 615-624, 2006. (Pubitemid 44568432)