Optimization techniques at the I/O forwarding layer

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2010, Pages 312-321

  Ohta, Kazuki ^a   Kimpe, Dries ^b   Cope, Jason ^b   Iskra, Kamil ^b   Ross, Robert ^b   Ishikawa, Yutaka ^c  

^a Preferred Infrastructure Inc   (Japan)

^b ARGONNE NATIONAL LABORATORY   (United States)

^c UNIVERSITY OF TOKYO   (Japan)

Author keywords

I O forwarding; Leadership class machines; Parallel file systems

Indexed keywords

COMPUTING FACILITIES; COMPUTING SYSTEM; FILE SYSTEMS; I/O BOTTLENECK; I/O FORWARDING; I/O SYSTEMS; LEADERSHIP-CLASS MACHINES; OPTIMIZATION TECHNIQUES; PARALLEL FILE SYSTEM; PARALLEL FILE SYSTEMS; SCIENTIFIC APPLICATIONS; SECOND OPTIMIZATION; STORAGE SYSTEMS; UNIVERSITY OF TOKYO;

CLUSTER COMPUTING; DATA TRANSFER; FILE ORGANIZATION; SUPERCOMPUTERS;

OPTIMIZATION;

EID: 78649473366     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTER.2010.36     Document Type: Conference Paper

References (31)

1. "Top500 list," http://www.top500.org/. [Online]. Available: http://www.top500.org/
2. V. S. ed., "ExaScale Software Study: Software Challenges in Extreme Scale Systems," September 2009.
3. E. Smirni and D. A. Reed, "Lessons from Characterizing the Input/Output Behavior of Parallel Scientific Applications," Perform. Eval., vol. 33, no. 1, pp. 27-44, 1998.
4. "HDF5," http://www.hdfgroup.org/HDF5/.
5. J. Li, W. keng Liao, A. N. Choudhary, R. B. Ross, R. Thakur, W. Gropp, and R. Latham, "Parallel netCDF: A Scientific High-Performance I/O Interface," CoRR, vol. cs.DC/0306048, 2003.
6. J. M. del Rosario, R. Bordawekar, and A. Choudhary, "Improved Parallel I/O via a Two-Phase Run-time Access Strategy," SIGARCH Comput. Archit. News, vol. 21, no. 5, pp. 31-38, 1993.
7. R. Thakur, W. Gropp, and E. Lusk, "Optimizing noncontiguous accesses in MPI - IO," Parallel Comput., vol. 28, no. 1, pp. 83-105, 2002.
8. P. H. Carns, W. B. L. III, R. B. Ross, and R. Thakur, "PVFS: A Parallel File System for Linux Clusters," in ALS'00: Proceedings of the 4th Annual Linux Showcase & Conference, Atlanta. Berkeley, CA: USENIX Association, 2000, pp. 28-28.
9. "Lustre file system," http://lustre.org/.
10. F. Schmuck and R. Haskin, "GPFS: A Shared-Disk File System for Large Computing Clusters," in Proceedings of the First Conference on File and Storage Technologies (FAST), Jan. 2002, pp. 231-244. [Online]. Available: citeseer.ist.psu.edu/schmuck02gpfs.html
11. "IOFSL: I/O Forwarding and Scalability Layer," http://www.iofsl.org/.
12. N. Ali, P. Carns, K. Iskra, D. Kimpe, S. Lang, R. Latham, R. Ross, L. Ward, and P. Sadayappan, "Scalable I/O Forwarding Framework for High-Performance Computing Systems," in IEEE Int'l Conference on Cluster Computing (Cluster 2009), September 2009.
13. K. Iskra, J. W. Romein, K. Yoshii, and P. Beckman, "ZOID: I/O-Forwarding Infrastructure for Petascale Architectures," in PPoPP '08: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. New York, NY, USA: ACM, 2008, pp. 153-162.
14. P. Carns, R. Ross, W. Ligon, and P. Wyckoff, "BMI: A Network Abstraction Layer for Parallel I/O," in In Proceedings of IPDPS05, CAC Workshop, 2005.
15. R. Thakur, W. Gropp, and E. Lusk, "On implementing MPI-IO portably and with high performance," in Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, 1999, pp. 23-32. [Online]. Available: citeseer.ist.psu.edu/article/ thakur99implementing.html
16. "libsysio," http://sourceforge.net/projects/libsysio.
17. T. P. Community, "Parallel virtual file system, version 2," http://www.pvfs.org/.
18. K. Ohta and Y. Ishikawa, "Scalable Parallel I/O Systems by Client-Side Optimizations," Master's thesis, University of Tokyo, March 2010.
19. Y. Qian, E. Barton, T. Wang, N. Puntambekar, and A. Dilger, "A novel network request scheduler for a large scale storage system," Computer Science - R&D, vol. 23, no. 3-4, pp. 143-148, 2009.
20. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, Third Edition. The MIT Press, 2009.
21. "T2K," http://www.open-supercomputer.org/.
22. "Bonnie++," http://www.coker.com.au/bonnie++/.
23. "MPICH2: High-performance and Widely Portable MPI," http://www.mcs.anl.gov/research/projects/mpich2/ .
24. P. Wong and R. F. V. der Wijngaart, "NAS Parallel Benchmark I/O Version 2.4, NAS Technical Report nas- 03-002, NASA Ames Research Center, Moffett Field, CA 94035-1000."
25. "Zeptoos," http://www.mcs.anl.gov/research/projects/zeptoos/.
26. Sandia National Laboratories, "Computational Plant," http://www.cs.sandia.gov/cplant.
27. H. Yu, R. K. Sahoo, C. Howson, G. Almasi, J. G. Castanos, M. Gupta, J. E. Moreira, J. J. Parker, T. E. Engelsiepen, R. Ross, R. Thakur, R. Latham, and W. D. Gropp, "High performance file I/O for the Blue Gene/L supercomputer," in International Symposium on High- Performance Computer Architecture, Feb. 2006.
28. D. Thain, T. Tannenbaum, and M. Livny, "Distributed computing in practice: The Condor experience." Concurrency - Practice and Experience, vol. 17, no. 2-4, pp. 323-356, 2005.
29. R. Ross, "Reactive Scheduling for Parallel I/O Systems," Ph.D. dissertation, Clemson University, 2000.
30. A. Lebre, G. Huard, Y. Denneulin, and P. Sowa, "I/O Scheduling Service for Multi-Application Clusters," in Proceedings of IEEE Cluster 2006, 2006.
31. K. Ohta, H. Matsuba, and Y. Ishikawa, "Improving parallel write by node-level request scheduling," in IEEE Int'l Symposium on Cluster Computing and the Grid (CCGrid 2009), May 2009.