/Scratch as a cache: Rethinking HPC center scratch storage

Proceedings of the International Conference on Supercomputing

Volumn , Issue , 2009, Pages 350-359

  Monti, Henry M ^a   Butt, Ali R ^a   Vazhkudai, Sudharshan S ^b  

^a Virginia Polytechnic Institute and State University   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

HPC scratch management; Just in time staging; Offloading

Indexed keywords

CACHING POLICY; DATA POPULATION; DATA STAGING; FILE SYSTEMS; HIGH-PERFORMANCE COMPUTING; JOB SCHEDULING; JUST IN TIME; JUST-IN-TIME STAGING; OPERATING BUDGETS; SCIENTIFIC APPLICATIONS; STORAGE SYSTEMS;

INTELLIGENT CONTROL; MANAGEMENT; SUPERCOMPUTERS;

POPULATION STATISTICS;

EID: 70449707775     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1542275.1542325     Document Type: Conference Paper

References (28)

1. GUPFS - Global Unified Parallel File System. http://www.nersc.gov/ projects/GUPFS/, 2004.
2. NCCS.GOV File Systems. http://info.nccs.gov/computing-resources/jaguar/ file-systems, 2007.
3. National Center for Computational Sciences. http://www.nccs.gov/, 2008.
4. Pbs pro technical overview: Scheduling and file staging. https://secure.altair.com/sched-staging.html, 2008.
5. Spider. http://www.nccs.gov/2008/06/30/nccs-launches-new-file-management- system/, 2008.
6. S. Albers and M. Büttner. Integrated prefetching and caching in single and parallel disk systems. In Proc. 15th ACM SPAA, pages 24-39, Duluth, MN, June 2003.
7. J. Bent, D. Thain, A. Arpaci-Dusseau, R. Arpaci-Dusseau, and M. Livny. Explicit control in a batch aware distributed file system. In Proceedings of the First USENIX/ACM Conference on Networked Systems Design and Implementation, Mar. 2004.
8. P. Cao, E. W. Felten, A. R. Karlin, and K. Li. Implementation and performance of integrated application-controlled file caching, prefetching, and disk scheduling. ACM Transactions on Computer Systems, 14(4):311-343, 1996.
9. P. Carns, W. L. III, R. Ross, and R. Thakur. PVFS: A Parallel File System For Linux Clusters. In Proceedings of the 4th Annual Linux Showcase and Conference, 2000.
10. Cluster File Systems, Inc. Lustre: A scalable, high-performance file system. http://www.lustre.org/docs/whitepaper.pdf, 2002.
11. R. Coyne and R. Watson. The parallel i/o architecture of the high-performance storage system (hpss). In Proceedings of the IEEE MSS Symposium, 1995.
12. C. Gniady, A. R. Butt, and Y. C. Hu. Program-counter-based pattern classification in buffer caching. In Proc. USENIX OSDI, 2004.
13. Grid Infrastructure Group. TeraGrid, May 2007. http://www.teragrid.org/.
14. T. Kosar and M. Livny. Stork: Making data placement a first class citizen in the grid. In Proceedings of 24th IEEE International Conference on Distributed Computing Systems (ICDCS2004), 2004.
15. X. Ma, S. Vazhkudai, V. Freeh, T. Simon, T. Yang, and S. L. Scott. Coupling prefix caching and collective downloads for remote data access. In Proceedings of the ACM International Conference on Supercomputing, 2006.
16. N. Megiddo and D. S. Modha. ARC: A Self-tuning, Low Overhead Replacement Cache. In Proc. 2nd USENIX FAST, pages 115-130, San Francisco, CA, Mar. 2003.
17. Cluster resources inc. http://www.clusterresources.com/, 2008.
18. H. Monti, A. R. Butt, and S. S. Vazhkudai. Just-in-time staging of large input data for supercomputing jobs. In Proc. PDS Workshop at SC08, 2008.
19. H. Monti, A. R. Butt, and S. S. Vazhkudai. Timely offloading of result-data in hpc centers. In Proc. ACM ICS'08, 2008.
20. OpenPBS. Portable batch system. http://www.openpbs.org/.
21. R. H. Patterson, G. A. Gibson, E. Ginting, D. Stodolsky, and J. Zelenka. Informed prefetching and caching. In Proc. 15th ACM SOSP, pages 79-95, Copper Mountain, CO, Dec. 1995.
22. J. Plank, M. Beck, W. Elwasif, T. Moore, M. Swany, and R. Wolski. The Internet Backplane Protocol: Storage in the network. In Proceedings of the Network Storage Symposium, 1999.
23. F. Schmuck and R. Haskin. GPFS: a shared-disk file system for large computing clusters. In Proceedings of the First Conference on File and Storage Technologies, 2002.
24. D. Thain, S. S. J. Basney, and M. Livny. The kangaroo approach to data movement on the grid. In Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10), 2001.
25. B. Tierney, J. Lee, M. Holding, J. Hylton, and F. Drake. A network-aware distributed storage cache for data intensive environments. In Proceedings of the IEEE High Performance Distributed Computing conference (HPDC-8), 1999.
26. N. Tran and D. A. Reed. ARIMA time series modeling and forecasting for adaptive I/O prefetching. In Proc. 15th International Conference on Supercomputing, pages 473-485, Sorrento, Italy, June 2001.
27. R. Wolski, N. Spring, and J. Hayes. The Network Weather Service: A distributed resource performance forecasting service for metacomputing. Future Generation Computing Systems, 15(5):757-768, 1999.
28. Z. Zhang, C. Wang, S. S. Vazhkudai, X. Ma, G. Pike, J. Cobb, and F. Mueller. Optimizing center performance through coordinated data staging, scheduling and recovery. In Proc. SC, 2007.