Efficient object storage journaling in a distributed parallel file system

Proceedings of FAST 2010: 8th USENIX Conference on File and Storage Technologies

Volumn , Issue , 2010, Pages 143-154

  Oral, Sarp ^a   Wang, Feiyi ^a   Dillow, David ^a   Shipman, Galen ^a   Miller, Ross ^a   Drokin, Oleg ^b  

^a OAK RIDGE NATIONAL LABORATORY   (United States)

^b ORNL Sun Microsystem Inc

Author keywords

[No Author keywords available]

Indexed keywords

DIGITAL STORAGE;

HARDWARE SOLUTIONS; LARGE SCALE PRODUCTIONS; LOCAL FILE SYSTEMS; LUSTRE FILE SYSTEMS; OVERALL EFFICIENCY; PARALLEL FILE SYSTEM; SCIENTIFIC APPLICATIONS; STORAGE SYSTEMS;

FILE ORGANIZATION;

EID: 85077022178     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. S. R. Alam, R. F. Barrett, M. R. Fahey, J. A. Kuehn, J. M. Larkin, R. Sankaran, and P. H. Worley. Cray XT4: An early evaluation for petascale scientific simulation. In Proceedings of the ACM/IEEE conference on High Performance Networking and Computing (SC07), Reno, NV, 2007.
2. N. Ali, P. Carns, K. Iskra, D. Kimpe, S. Lang, R. Latham, R. Ross, L. Ward, and P. Sadayappan. Scalable i/o forwarding framework for high-performance computing systems. In Proceedings of the IEEE International Conference on Cluster Computing, Aug, 2009.
3. M. Baker, S. Asami, E. Deprit, J. Ousterhout, and M. Seltzer. Non-volatile memory for fast, reliable file systems. In Proceedings of the 5th ASPLOS, pages 10-22, 1992.
4. R. Brightwell, K. Pedretti, and K. D. Underwood. Initial performance evaluation of the cray seastar interconnect. In HOTI'05: Proceedings of the 13th Symposium on High Performance Interconnects, pages 51-57, Washington, DC, USA, 2005. IEEE Computer Society.
5. Cray Inc. Cray XT5. http://cray.com/Products/XT/Systems/XT5.aspx.
6. Data Direct Networks. DDN S2A9900. http://www.ddn.com/9900.
7. Dell. Dell PowerEdge 1950 Server. http://www.dell.com/downloads/global/products/ pedge/en/1950_specs.pdf.
8. J. Dongarra, H. Meuer, and E. Strohmaier. Top500 November 2009 List. http://www.top500.org/lists/2009/ 11, 2008.
9. J. Dongarra, H. Meuer, and E. Strohmaier. Top500 supercomputing sites. http://www.top500.org, 2009.
10. G. R. Ganger and Y. N. Patt. Metadata update performance in file systems. In OSDI'94: Proceedings of the 1st USENIX conference on Operating Systems Design and Implementation, page 5, Berkeley, CA, USA, 1994. USENIX Association.
11. ioperformance.com. xdd performance benchmark, version 6.5. http://www.ioperformance.com, 2008.
12. S. Klasky. private communication, Sept. 2009.
13. A. Leventhal. Hybrid storage pools in the 7410. http://blogs.sun.com/ahl/entry/fishworks_launch.
14. A. Leventhal. Flash storage memory. Communications of the ACM, 51(7):47-51, 2008.
15. Z. Lin, T. S. Hahm, W. W. Lee, W. M. Tang,, and R. B. White. Turbulent transport reduction by zonal flows: Massively parallel simulations. Science, 18:1835-1837, 1988.
16. J. Lofstead, F. Zheng, S. Klasky, and K. Schwan. Adaptable, metadata rich io methods for portable high performance io. In In Proceedings of IPDPS'09, May 25-29, Rome, Italy, 2009.
17. R. Mendel and J. K. Ousterhout. The design and implementation of a log-structured file system. ACM Trans. Comput. Syst., 10(1):26-52, 1992.
18. D. A. Nowak and M. Seagar. ASCI terascale simulation: Requirements and deployments. http://www.ornl.gov/sci/optical/docs/Tutorial19991108Nowak.pdf.
19. Oak Ridge National Laboratory, National Center for Computational Sciences. Jaguar. http://www.nccs.gov/jaguar/.
20. B. Pawlowski, C. Juszczak, P. Staubach, C. Smith, D. Lebel, and D. Hitz. NFS Version 3 - Design and Implementation. Proceedings of the Summer 1994 USENIX Technical Conference, pages 137-152, 1994.
21. V. Prabhakaran, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Analysis and evolution of journaling file systems. In Proceedings of the Annual USENIX Technical Conference, May 2005.
22. B. Schroeder and G. A. Gibson. Understanding failures in petascale computers. Journal of Physics Conference Series, 78(1):012022-+, July 2007.
23. M. Seltzer, G. Ganger, K. McKusick, K. Smith, C. Soules, and C. Stein. Journaling versus Soft Updates: Asynchronous Meta-data Protection in File Systems. In Proceedings of the USENIX Technical Conference, pages 71-84, June 2000.
24. H. Shan and J. Shalf. Using IOR to analyze the I/O performance of XT3. In Proceedings of the 49th Cray User Group (CUG) Conference 2007, Seattle, WA, 2007.
25. G. Shipman. Spider and SION: Supporting the I/O Demands of a Peta-scale Environment. In Cray User Group Meeting, 2008.
26. G. Shipman, D. Dillow, S. Oral, and F. Wang. The spider center wide file system: From concept to reality. In Proceedings,Cray User Group (CUG) Conference, Atlanta, GA, May 2009.
27. Sun Microsystems. Lustre i/o kit, obdfilter-survey. http://manual.lustre.org/manual/LustreManual16_HTML/LustreIOKit.html.
28. Sun Microsystems Inc. Luste wiki. http://wiki.lustre.org, 2009.
29. Texas Memory Systems Inc. Ramsan-400. http://www.ramsan.com/products/ramsan-400.htm.
30. S. C. Tweedie. Journaling the Linux ext2fs Filesystem. In Proceedings of the fourth annual Linux expo, 1998.
31. F. Wang, S. Oral, G. Shipman, O. Drokin, T. Wang, and I. Huang. Understanding lustre filesystem internals. Technical Report ORNL/TM-2009/117, Oak Ridge National Lab., National Center for Computational Sciences, 2009.
32. W. Yu, S. Oral, S. Canon, J. Vetter, and R. Sankaran. Empirical analysis of a large-scale hierarchical storage system. In 14th European Conference on Parallel and Distributed Computing (Euro-Par 2008), 2008.
33. W. Yu, J. Vetter, and S. Oral. Performance characterization and optimization of parallel I/O on the Cray XT. In Proceedings of 22nd IEEE International Parallel and Distributed Processing Symposium (IPDPS'08), Miami, FL, 2008.