InterferenceRemoval: Removing interference of disk access for MPI programs through data replication

Proceedings of the International Conference on Supercomputing

Volumn , Issue , 2010, Pages 223-232

  Zhang, Xuechen ^a   Jiang, Song ^a  

^a WAYNE STATE UNIVERSITY   (United States)

Author keywords

I O interference; MPI program; MPI IO

Indexed keywords

ACCESS PATTERNS; COLLECTIVE I/O; DATA REPLICATION; DATA SETS; DIFFERENT PROCESS; HARD DISKS; HIGH THROUGHPUT; I/O PERFORMANCE; ITS DATA; MPI PROGRAMS; MULTIPLE PROCESS; PARALLEL FILE SYSTEM;

COMPUTER SOFTWARE; DISKS (STRUCTURAL COMPONENTS); INTELLIGENT CONTROL; OPTIMIZATION; SOFTWARE ARCHITECTURE; THROUGHPUT;

MULTIPLE ACCESS INTERFERENCE;

EID: 77954708874     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1810085.1810116     Document Type: Conference Paper

References (25)

1. M. Bhadkamkar, J. Guerra, L. Useche, S. Burnett, J. Liptak, R. Rangaswami, and V. Hristidis, "BORG:Block-reORGanization for Self-optimizing Storage Systems", In Proceedings of the 7th USENIX Conference on File and Storage Technologies , San Fancisco, CA, 2009.
2. A. Ching, A. Choudhary, W. Liao, R. Ross, and W. Gropp, "Efficient Structured Data Access in Parallel File System", In Proceedings of IEEE International Conference on Cluster Computing, Hong Kong, China, 2003.
3. A. Ching, A. Choudhary, K. Coloma, and W. Liao, "Noncontiguous I/O Accesses Through MPI-IO", In Proceedings of IEEE International Symposium on Cluster, Cloud, and Grid Computing, Tokyo, Japan, 2003.
4. Cluster File Systems, Inc. Lustre. "Lustre: A scalable, robust, highly-available cluster file system", http://www.lustre.org/. Online-document, 2010.
5. H. Huang, W. Hung, and K. Shin, "FS2: Dynamic Data Replication in Free Disk Space for Improving Disk Performance and Energy Consumption", In Proceedings of ACM Symposium on Operating Systems Principles, Brighton, UK, 2005.
6. W. Hsu, A. Smith, H. Young, "The Automatic Improvement of Locality in Storage Systems", ACM Transactions on Computer Systems, Volume 23, Issue 4, Nov. 2006, Pages 424-473.
7. W. Hsu, A. Smith, H. Young, "The Automatic Improvement of Locality in Storage Systems", Technical Report CSD-03-1264, UC Berkeley, Jul. 2003.
8. Interleaved or Random (IOR) benchmarks, http://www.cs.dartmouth.edu/ pario/examples.html, Online-document, 2008.
9. S. Iyer and P. Druschel, "Anticipatory scheduling: A disk scheduling framework to overcome deceptive idleness in synchronous I/O", In Proceedings of ACM Symposium on Operating Systems Principles, Banff, Canada, 2001.
10. D. Kotz, "Disk-directed I/O for MIMD Multiprocessors.", ACM Transactions on Computer Systems, Volume 15, Issue 1, Feb. 1997, pages 41-74.
11. S. Liang, S. Jiang, and X. Zhang, "STEP:Sequentiality and Thrashing Detection Based Prefetching to Improve Performance of Networked Storage Servers.", In Proceedings of International Conference on Distributed Computing Systems, Toronto, Canada, 2007.
12. Mpi-tile-io Benchmark, http: www-unix.mcs.anl.gov/thakur/pio-benchmarks. html. Online-document, 2009.
13. M. Kandemir, S. Son, M. Karakoy, "Improving I/O Performance of Applications through Compiler-Directed Code Restructuring", In Proceedings of the 6th USENIX Conference on File and Storage Technologies, San Jose, CA, 2008.
14. MPICH2, Argonne National Laboratory, http://www.mcs.anl.gov/research-/ projects/mpich2/. Online-document, 2009.
15. NAS Parallel Benchmarks, NASA AMES Research Center, http://www.nas.nasa. gov/Software/NPB/. Online-document, 2009.
16. PVFS, http://www.pvfs.org. Online-document, 2010.
17. P. Pacheco, "Parallel Programming with MPI", Morgran Kaufmann Publishers, pages 137-178, 1997.
18. K. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett, "Server-directed collective I/O in Panda", In Proceedings of Supercomputing, San Diego, CA, 1995.
19. F. Schmuck and R. Haskin, "GPFS:A shared-disk file system for large computing clusters.", In Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, CA, 2002, Monterey, CA, USA.
20. R. Thakur, W. Gropp and E. Lusk, "Data Sieving and Collective I/O in ROMIO", In Proceedings of the 7th Symposium on the Frontiers of Massively Parallel Computation, Annapolis, MD, 1999.
21. S3aSim I/O Benchmark, http://wwwunix.mcs.anl.gov/thakur/s3asim.html. Online-document, 2009.
22. The DiskSim Simulation Environment(v4.0), Parallel Data Lab, http://www.pdl.cmu.edu/DiskSim/. Online-document, 2009.
23. Y. Wang and D. Kaeli, "Profile-Guided I/O Partitioning", In Proceedings of International Conference on Supercomputing, San Fancisco, CA, 2003.
24. C. Wang, Z. Zhang, X. Ma, S. Vazhkudai, and F. Mueller, "Improving the Availability of Supercomputer Job Input Data Using Temporal Replication", In Proceedings of International Supercomputing Conference, Hamburg, Germany, 2009.
25. X. Zhang, S. Jiang, and K. Davis, "Making Resonance a Common Case: A High-performance Implementation of Collective I/O on Parallel File Systems", In Proceedings of IEEE International Parallel & Distributed Processing Symposium, Rome, Italy, 2009.