A server-level adaptive data layout strategy for parallel file systems

Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012

Volumn , Issue , 2012, Pages 2095-2103

  Song, Huaiming ^a   Jin, Hui ^b   He, Jun ^b   Sun, Xian He ^b   Thakur, Rajeev ^c  

^a Dawning Information Industry   (China)

^b Illinois Institute of Technology   (United States)

^c ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

data layout optimization; equal depth histogram; parallel file system; Server level adaptive data layout; variable stripe size

Indexed keywords

DATA LAYOUT OPTIMIZATION; DATA LAYOUTS; EQUAL-DEPTH HISTOGRAM; PARALLEL FILE SYSTEM; VARIABLE STRIPE SIZE;

BANDWIDTH; DISTRIBUTED PARAMETER NETWORKS; GRAPHIC METHODS;

FILE ORGANIZATION;

EID: 84867423230     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPSW.2012.246     Document Type: Conference Paper

References (30)

1. "High-performance Storage Architecture and Scalable Cluster File System," Lustre File System White Paper, December 2007.
2. F. Schmuck and R. Haskin, "GPFS: A Shared-disk File System for Large Computing Clusters," in FAST '02: Proceedings of the 1st USENIX Conference on File and Storage Technologies. Berkeley, CA, USA: USENIX Association, 2002, p. 19.
3. G. Gibson, B. Welch, G. Goodson, and P. Corbett, "Parallel NFS Requirements and Design Considerations," Internet Draft, October 2004.
4. I. F. Haddad, "PVFS: A Parallel Virtual File System for Linux Clusters," Linux Journal, p. 5, 2000.
5. D. Nagle, D. Serenyi, and A. Matthews, "The Panasas Activescale Storage Cluster: Delivering Scalable High Band-width Storage," in SC '04: Proceedings of the 2004 ACM/IEEE Conference on Supercomputing. Washington, DC, USA: IEEE Computer Society, 2004, p. 53.
6. P. Lu and K. Shen, "Multi-layer Event Trace Analysis for Parallel I/O Performance Tuning," International Conference on Parallel Processing, vol. 0, p. 12, 2007.
7. A. Konwinski, J. Bent, J. Nunez, and M. Quist, "Towards an I/O Tracing Framework Taxonomy," in Proceedings of the 2nd International Workshop on Petascale Data Storage: held in conjunction with Supercomputing '07, ser. PDSW '07. New York, NY, USA: ACM, 2007, pp. 56-62.
8. S. Byna, Y. Chen, X.-H. Sun, R. Thakur, and W. Gropp, "Parallel I/O Prefetching Using MPI File Caching and I/O Signatures," in SC '08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing. Piscataway, NJ, USA: IEEE Press, 2008, pp. 1-12.
9. K. Vijayakumar, F. Mueller, X. Ma, and P. C. Roth, "Scalable I/O Tracing and Analysis," in PDSW '09: Proceedings of the 4th Annual Workshop on Petascale Data Storage. New York, NY, USA: ACM, 2009, pp. 26-31.
10. X.-H. Sun, Y. Chen, and Y. Yin, "Data Layout Optimization for Petascale File Systems," in PDSW '09: Proceedings of the 4th Annual Workshop on Petascale Data Storage. New York, NY, USA: ACM, 2009, pp. 11-15.
11. H. Song, Y. Yin, Y. Chen, and X.-H. Sun, "A Cost-intelligent Application-specific Data Layout Scheme for Parallel File Systems," in Proceedings of the 20th ACM International Symposium on High Performance Distributed Computing, ser. HPDC '11. 2011.
12. E. Smirni and D. A. Reed, "Workload Characterization of Input/Output Intensive Parallel Applications," in in Proceedings of the 9th International Conference on Modelling Techniques and Tools for Computer Performance Evaluation. Springer-Verlag, 1997, pp. 169-180.
13. T. M. Madhyastha and D. A. Reed, "Learning to Classify Parallel Input/Output Access Patterns," IEEE Transactions on Parallel and Distributed Systems, vol. 13, pp. 802-813, 2002.
14. F. Chen, D. A. Koufaty, and X. Zhang, "Survival of the Fittest: Making the Best Usage of SSDs in High-performance Storage Systems," in Proceedings of the 25th ACM International Conference on Supercomputing, ser. ICS '11. 2011.
15. N. Tran, D. A. Reed, and S. Member, "Automatic ARIMA Time Series Modeling for Adaptive I/O Prefetching," IEEE Transactions on Parallel and Distributed Systems, vol. 15, pp. 362-377, 2004.
16. X. Zhang and S. Jiang, "InterferenceRemoval: Removing Interference of Disk Access for MPI Programs through Data Replication," in Proceedings of the 24th International Conference on Supercomputing, 2010, pp. 223-232.
17. H. Song, Y. Yin, X.-H. Sun, R. Thakur, and S. Lang, "A Segment-level Adaptive Data Layout Scheme for Improved Load Balance in Parallel File Systems," in Proceedings of the 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, ser. CCGRID '11. 2011.
18. D. Kotz and N. Nieuwejaar, "File-system Workload on a Scientific Multiprocessor," IEEE Parallel and Distributed Technology, vol. 3, pp. 51-60, March 1995.
19. M. Seltzer, P. Chen, and J. Ousterhout, "Disk Scheduling Revisited," in In Proceedings of the USENIX Winter Technical Conference (USENIX Winter 90, 1990, pp. 313-324.
20. B. L. Worthington, G. R. Ganger, and Y. N. Patt, "Scheduling Algorithms for Modern Disk Drives," 1994, pp. 241-251.
21. C. R. Lumb, J. Schindler, G. R. Ganger, and D. F. Nagle, "Towards Higher Disk Head Utilization: Extracting Free Band-width from Busy Disk Drives," in Symposium on Operating Systems Design and Implementation. USENIX Association, 2000, pp. 87-102.
22. S. Rubin, R. Bodík, and T. Chilimbi, "An Efficient Profile-analysis Framework for Data-layout Optimizations," SIGPLAN Not., vol. 37, no. 1, pp. 140-153, 2002.
23. Y. Wang and D. Kaeli, "Profile-guided I/O Partitioning," in Proceedings of the 17th Annual International Conference on Supercomputing, ser. ICS '03. New York, NY, USA: ACM, 2003, pp. 252-260.
24. S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn, "Ceph: A Scalable, High-performance Distributed File System," in OSDI '06: Proceedings of the 7th Symposium on Operating Systems Design and Implementation, 2006, pp. 307-320.
25. J. Bent, G. Gibson, G. Grider, B. McClelland, P. Nowoczynski, J. Nunez, M. Polte, and M. Wingate, "PLFS: a Checkpoint Filesystem for Parallel Applications," in SC '09: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, New York, NY, USA: ACM, 2009, pp. 21:1-21:12.
26. "Application I/O Traces: Anonymous LANL App1." [Online]. Available: http://institutes.lanl.gov/plfs/maps/
27. P. B. Gibbons, Y. Matias, and V. Poosala, "Fast Incremental Maintenance of Approximate Histograms," in Proceedings of the 23rd International Conference on Very Large Data Bases, ser. VLDB '97. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1997, pp. 466-475.
28. S. W. Son, S. P. Muralidhara, O. Ozturk, M. Kandemir, I. Kolcu, and M. Karakoy, "Profiler and Compiler Assisted Adaptive I/O Prefetching for Shared Storage Caches," in Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques, ser. PACT '08. New York, NY, USA: ACM, 2008, pp. 112-121.
29. G. Yadgar, M. Factor, K. Li, and A. Schuster, "MC2: Multiple Clients on a Multilevel Cache," in Proceedings of the 2008 The 28th International Conference on Distributed Computing Systems, ser. ICDCS '08. Washington, DC, USA: IEEE Computer Society, 2008, pp. 722-730.
30. C. M. Patrick, M. Kandemir, M. Karaköy, S. W. Son, and A. Choudhary, "Cashing in on Hints for Better Prefetching and Caching in PVFS and MPI-IO," in Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, ser. HPDC '10. New York, NY, USA: ACM, 2010, pp. 191-202.