A workflow-aware storage system: An opportunity study

Proceedings - 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2012

Volumn , Issue , 2012, Pages 326-334

  Vairavanathan, Emalayan ^a   Al Kiswany, Samer ^a   Costa, Lauro Beltrão ^a   Zhang, Zhao ^b   Katz, Daniel S ^c   Wilde, Michael ^c   Ripeanu, Matei ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b UNIVERSITY OF CHICAGO   (United States)

^c ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

Large scale storage systems; workflow optimizations; workflow aware storage systems

Indexed keywords

DATA ACCESS PATTERNS; DATA LOCATION; DESIGN DECISIONS; DISTRIBUTED STORAGE SYSTEM; NFS SERVERS; PERFORMANCE BENEFITS; PERFORMANCE GAIN; RUNTIMES; STORAGE SYSTEMS; SYNTHETIC BENCHMARK; WORK-FLOWS; WORKFLOW OPTIMIZATION; WORKFLOW-BASED APPLICATIONS;

GRID COMPUTING; MULTIPROCESSING SYSTEMS; OPTIMIZATION;

BENCHMARKING;

EID: 84863702050     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGrid.2012.109     Document Type: Conference Paper

References (28)

1. modFTDock. [cited 2011; http://www.mybiosoftware.com/3d-molecularmodel/ 922.
2. A. C. Laity, N. Anagnostou, G. B. Berriman, J. C. Good, et al. Montage: An Astronomical Image Mosaic Service for the NVO. in Proceedings of Astronomical Data Analysis Software and Systems (ADASS). 2004.
3. Y. Chen, W. Chen, M. H. Cobb, and Y. Zhao, PTMap A sequence alignment software for unrestricted, accurate, and full-spectrum identification of post-translational modification sites. Proceedings of the National Academy of Sciences of the USA, 2009. 106(3).
4. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. in USENIX Symposium on Operating Systems Design and Implementation (OSDI). 2004.
5. I. Raicu, I. T. Foster, and Y. Zhao, Many-Task Computing for Grids and Supercomputers, in IEEE Workshop on Many-Task Computing on Grids and Supercomputers 2008.
6. I. Foster, M. Hategan, J. M. Wozniak, M. Wilde, et al., Swift: A language for distributed parallel scripting Journal of Parallel Computing, 2011.
7. E. Deelman, G. Singh, M.-H. Su, J. Blythe, et al., Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems. Journal of Scientific Programming, 2005. 13(3): p. 219-237.
8. E. Santos-Neto, S. Al-Kiswany, N. Andrade, S. Gopalakrishnan, et al. Beyond Search and Navigability: Custom Metadata Can Enable Cross-Layer Optimizations in Storage Systems. in ACM/IEEE International Symposium on High Performance Distributed Computing (HPDC) - Hot Topics Track. 2008.
9. J. Wozniak and M. Wilde. Case studies in storage access by loosely coupled petascale applications. in Petascale Data Storage Workshop. 2009.
10. J. Bent, D. Thain, A. C.Arpaci-Dusseau, R. H. Arpaci-Dusseau, et al. Explicit Control in a Batch-Aware Distributed File System. in Proceedings of the 1st USENIX Symposium on Networked Systems Design and Implementation (NSDI '04). 2004. San Francisco, California.
11. K. Shvachko, H. Kuang, S. Radia, and R. Chansler. The Hadoop Distributed File System. in IEEE Symposium on Mass Storage Systems and Technologies (MSST) 2010.
12. S. Al-Kiswany, A. Gharaibeh, and M. Ripeanu. The Case for Versatile Storage System. in Workshop on Hot Topics in Storage and File Systems (HotStorage). 2009.
13. I. Raicu, Y. Zhao, I. Foster, and A. Szalay, Accelerating Large-scale Data Exploration through Data Diffusion, in International Workshop on Data-Aware Distributed Computing. 2008.
14. A. Chervenak, E. Deelman, M. Livny, M.-H. Su, et al., Data placement for scientific applications in distributed environments in IEEE/ACM International Conference on Grid Computing. 2007
15. D. S. Katz, T. G. Armstrong, Z. Zhang, M. Wilde, et al., Many-Task Computing and Blue Waters, in Technical Report CI-TR-13-0911. Computation Institute, University of Chicago & Argonne National Laboratory. arXiv:1202.3943v1. 2012.
16. T. Shibata, S. Choi, and K. Taura, File-access patterns of data-intensive workflow applications and their implications to distributed filesystems, in International Symposium on High Performance Distributed Computing (HPDC). 2010.
17. S. Bharathi, A. Chervenak, E. Deelman, G. Mehta, et al., Characterization of Scientific Workflows, in Workshop on Workflows in Support of Large-Scale Science. 2008.
18. U. Yildiz, A. Guabtni, and A. H. H. Ngu, Towards scientific workflow patterns, in Workshop on Workflows in Support of Large-Scale Science. 2009.
19. G. A. Alvarez, E. Borowsky, S. Go, T. H. Romer, et al., Minerva: An automated resource provisioning tool for large-scale storage systems. ACM Transactions on Computer Systems (TOCS), 2001 19(4).
20. L. B. Costa and M. Ripeanu. Towards Automating the Configuration of a Distributed Storage System. in ACM/IEEE International Conference on Grid Computing (Grid). 2010.
21. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google File System. in 19th ACM Symposium on Operating Systems Principles. 2003. Lake George, NY.
22. K. Gupta, R. Jain, I. Koltsidas, H. Pucha, et al., GPFS-SNC: An enterprise storage framework for virtual-machine clouds IBM Journal of Research and Development 2011.
23. M. Rosenblum and J. K. Ousterhout. The Design and Implementation of a Log-Structured File System. in ACM Transactions on Computer Systems. February 1992.
24. M. Abd-El-Malek, W. V. C. II, C. Cranor, G. R. Ganger, et al. Ursa minor: versatile cluster-based storage. in FAST 2005.
25. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, et al. Falkon: a Fast and Light-weight tasK executiON framework. in SuperComputing. 2007.
26. I. Raicu, I. Foster, Y. Zhao, P. Little, et al., The Quest for Scalable Support of Data Intensive Workloads in Distributed Systems, in International symposium on High Performance Distributed Computing (HPDC). 2009.
27. Z. Zhang, D. Katz, M. Ripean, M. Wilde, et al. AME: An Anyscale Many-Task Computing Engine. in Workshop on Workflows in Support of Large-Scale Science. 2011.
28. FUSE, Filesystem in Userspace. [cited 2011; http://fuse.sourceforge.net/.