DataStager: Scalable data staging services for petascale applications

Proc. 18th ACM International Symposium on High Performance Distributed Computing, HPDC 09, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conf., HPDC'09

Volumn , Issue , 2009, Pages 31-37

  Abbasi, Hasan ^a   Wolf, Matthew ^a   Eisenhauer, Greg ^a   Klasky, Scott ^b   Schwan, Karsten ^a   Zheng, Fang ^a  

^a Georgia Institute of Technology   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Datatap; GTC; I O; Staging; WARP; XT3; XT4

Indexed keywords

CHECK POINTING; DATA STAGING; EXPERIMENTAL EVALUATION; HIGH PERFORMANCE APPLICATIONS; INTELLIGENT DATA; OAK RIDGE NATIONAL LABORATORY; OUTPUT DATA; PETASCALE; PROCESSING TIME; RUNTIMES;

COMPUTER SCIENCE; OPTICAL TELESCOPES; WEAVING;

DIE CASTING;

EID: 70449650353     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1551609.1551618     Document Type: Conference Paper

References (37)

1. H. Abbasi, M. Wolf, and K. Schwan. LIVE Data Workspace: A Flexible, Dynamic and Extensible Platform for Petascale Applications. Cluster Computing, 2007. IEEE International, Sept. 2007.
2. N. Ali and M. Lauria. Improving the performance of remote i/o using asynchronous primitives. High Performance Distributed Computing, 2006 15th IEEE International Symposium on, pages 218-228, 0-0 0.
3. P. Beckman and S. Coghlan. ZeptoOS: the small Linux for big computers, 2005.
4. K. Bell, A. Chien, and M. Lauria. A high-performance cluster storage server. High Performance Distributed Computing, 2002. HPDC-11 2002. Proceedings. 11th IEEE International Symposium on, pages 311-320, 2002.
5. R. Bianchini, M. Crovella, L. Kontothanassis, and T. LeBlanc. Alleviating Memory Contention in Matrix Computations on Large-Scale Shared-Memory Multiprocessors. Technical report, DTIC, 1993.
6. J. Borrill, L. Oliker, J. Shalf, and H. Shan. Investigation Of Leading HPC I/O Performance Using A Scientific-Application Derived Benchmark. In Proceedings of the 2007 Conference on SuperComputing, SC07, 2007.
7. R. Brightwell, T. Hudson, R. Riesen, and A. B. Maccabe. The Portals 3.0 message passing interface. Technical report SAND99-2959, Sandia National Laboratories, December 1999.
8. R. Brightwell, B. Lawry, A. B. MacCabe, and R. Riesen. Portals 3.0: Protocol building blocks for low overhead communication. In IPDPS '02: Proceedings of the 16th International Parallel and Distributed Processing Symposium, page 268, Washington, DC, USA, 2002. IEEE Computer Society.
9. F. E. Bustamante, G. Eisenhauer, K. Schwan, and P. Widener. Efficient wire formats for high performance computing. In Proceedings of the 2000 ACM/IEEE conference on Supercomputing (CDROM), page 39. IEEE Computer Society, 2000.
10. Z. Cai, G. Eisenhauer, Q. He, V. Kumar, K. Schwan, and M. Wolf. Iq-services: network-aware middleware for interactive large-data applications. In MGC '04: Proceedings of the 2nd workshop on Middleware for grid computing, pages 11-16, New York, NY, USA, 2004. ACM Press.
11. P. H. Carns, W. B. Ligon III, R. B. Ross, and R. Thakur. PVFS: A parallel file system for linux clusters. In Proceedings of the 4th Annual Linux Showcase and Conference, pages 317-327, Atlanta, GA, 2000. USENIX Association.
12. Lustre: A scalable, high-performance file system. Cluster File Systems Inc. white paper, version 1.0, November 2002. http://www.lustre.org/docs/ whitepaper.pdf.
13. S. Dandamudi. Reducing hot-spot contention in shared-memory multiprocessor systems. Concurrency, IEEE [see also IEEE Parallel & Distributed Technology], 7(1):48-59, Jan-Mar 1999.
14. C. Ding, S. Dwarkadas, M. Huang, K. Shen, and J. Carter. Program phase detection and exploitation. Parallel and Distributed Processing Symposium, 2006. IPDPS 2006. 20th International, pages 8 pp.-, 25-29 April 2006.
15. C. Docan, M. Parashar, and S. Klasky. High speed asynchronous data transfers on the cray xt3. In Cray User Group Conference, 2007.
16. G. Eisenhauer. The evpath library. http://www.cc.gatech.edu/systems/ projects/EVPath.
17. G. Eisenhauer. Portable binary input/output. http://www.cc.gatech.edu/ systems/projects/PBIO.
18. G. Eisenhauer, F. Bustamente, and K. Schwan. Event services for high performance computing. In Proceedings of High Performance Distributed Computing (HPDC-2000), 2000.
19. M. K. Gardner, W.-c. Feng, J. S. Archuleta, H. Lin, and X. Ma. Parallel Genomic Sequence-Searching on an Ad-Hoc Grid: Experiences, Lessons Learned, and Implications. In ACM/IEEE SC|06: The International Conference on High-Performance Computing, Networking, Storage, and Analysis, Tampa, FL, November 2006. Best Paper Nominee.
20. S. Golestani. A stop-and-go queueing framework for congestion management. In SIGCOMM'90 Symposium, pages 8-18. ACM, September 1990.
21. R. Jain, K. K. Ramakrishnan, and D. M. Chiu. Congestion avoidance in computer networks with a connectionless network layer. Technical Report DEC-TR-506, Digital Equipment Corporation, MA, Aug. 1987.
22. D. Kotz. Disk-directed I/O for MIMD multiprocessors. ACM Transactions on Computer Systems, 15(1):41-47, February 1997.
23. R. Latham, N. Miller, R. Ross, and P. Carns. A next-generation parallel file system for linux clusters. LinuxWorld, 2(1), January 2004.
24. J. Lofstead, K. Schwan, S. Klasky, N. Podhorszki, and C. Jin. Flexible io and integration for scientific codes through the adaptable io system (adios). In Challenges of Large Applications in Distributed Environments (CLADE), 2008.
25. E. L. Miller and R. H. Katz. Input/output behavior of supercomputing applications. In Supercomputing '91: Proceedings of the 1991 ACM/IEEE conference on Supercomputing, pages 567-576, New York, NY, USA, 1991. ACM.
26. A. Nisar, W. keng Liao, and A. Choudhary. Scaling parallel i/o performance through i/o delegate and caching system. In SC '08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing, pages 1-12, Piscataway, NJ, USA, 2008. IEEE Press.
27. R. A. Oldfield, A. B. Maccabe, S. Arunagiri, T. Kordenbrock, R. R. sen, L. Ward, and P. Widener. Lightweight I/O for Scientific Applications. In Proc. 2006 IEEE Conference on Cluster Computing, Barcelona, Spain, September 2006.
28. R. A. Oldfield, P. Widener, A. B. Maccabe, L. Ward, and T. Kordenbrock. Efficient Data Movement for Lightweight I/O. In Proc. 2006 Workshop on high-performance I/O techniques and deployment of Very-Large Scale I/ O Systems (HiPerI/O 2006), Barcelona, Spain, September 2006.
29. L. Oliker, J. Carter, michael Wehner, A. Canning, S. Ethier, A. Mirin, G. Bala, D. Parks, Patrick Worley Shigemune Kitawaki, and Y. Tsuda. Leading computational methods on scalar and vector hec platforms. In Proceedings of SuperComputing 2005, 2005.
30. C. M. Patrick, S. Son, and M. Kandemir. Comparative evaluation of overlap strategies with study of i/o overlap in mpi-io. SIGOPS Oper. Syst. Rev., 42(6):43-49, 2008.
31. F. Schmuck and R. Haskin. GPFS: A Shared-Disk File System for Large Computing Clusters. Proceedings of the 1st USENIX Conference on File and Storage Technologies, 2002.
32. K. E. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett. Server-directed collective i/o in panda. In Supercomputing '95: Proceedings of the 1995 ACM/IEEE conference on Supercomputing (CDROM), page 57, New York, NY, USA, 1995. ACM.
33. S. Sinha and M. Parashar. Adaptive system sensitive partitioning of amr applications on heterogeneous clusters. Cluster Computing, 5(4):343-352, 2002.
34. N. Stone, D. Balog, B. Gill, B. Johan-SON, J. Marsteller, P. Nowoczynski, D. Porter, R. Reddy, J. Scott, D. Simmel, et al. PDIO: High-performance remote file I/O for Portals enabled compute nodes. Proceedings of the 2006 Conference on Parallel and Distributed Processing Techniques and Applications, Las Vegas, NV, June, 2006.
35. P. M. Widener, M. Wolf, H. Abbasi, M. Barrick, J. Lofstead, J. Pullikottil, G. Eisenhauer, A. Gavrilovska, S. Klasky, R. Oldfield, P. G. Bridges, A. B. Maccabe, and K. Schwan. Structured streams: Data services for petascale science environments. Technical Report TR-CS-2007-17, University of New Mexico, Albuquerque, NM, November 2007.
36. M. Wolf, H. Abbasi, B. Collins, D. Spain, and K. Schwan. Service augmentation for high end interactive data services. In IEEE International Conference on Cluster Computing (Cluster 2005), September 2005.
37. M. Wolf, Z. Cai, W. Huang, and K. Schwan. Smartpointers: Personalized scientific data portals in your hand. In Proceedings of the Proceedings of the IEEE/ACM SC2002 Conference, page 20. IEEE Computer Society, 2002.