Making a case for distributed file systems at Exascale

LSAP'11 - Proceedings of the 3rd International Workshop on Large-Scale System and Application Performance

Volumn , Issue , 2011, Pages 11-18

  Raicu, Ioan ^a,b   Foster, Ian T ^b,c,d   Beckman, Pete ^b,c  

^a Illinois Institute of Technology   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

^c UNIVERSITY OF CHICAGO   (United States)

^d THE UNIVERSITY OF CHICAGO   (United States)

Author keywords

distributed file systems; exascale computing; many task computing; storage architecture

Indexed keywords

ACHILLES HEEL; BISECTION BANDWIDTH; COMPUTE RESOURCES; DISTRIBUTED FILE SYSTEMS; DISTRIBUTED STORAGE; EXASCALE COMPUTING; HIGH-END COMPUTING; MANY-CORE; MANY-TASK COMPUTING; MEAN TIME TO FAILURE; NON-VOLATILE MEMORIES; ORDERS OF MAGNITUDE; PETASCALE; POOR PERFORMANCE; POSITION PAPERS; SCIENTIFIC DISCOVERY; TORUS NETWORKS;

LARGE SCALE SYSTEMS; METADATA; NETWORK ARCHITECTURE;

INFORMATION MANAGEMENT;

EID: 79961097605     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1996029.1996034     Document Type: Conference Paper

References (110)

1. Top500 Supercomputer Sites, Performance Development, http://www.top500. org/lists/2010/11/performance-development, November 2010
2. T. Hey, S. Tansley, and K. Tolle. The Fourth Paradigm: Data-Intensive Scientific Discovery, Microsoft Research 2009
3. V. Sarkar, et al. "ExaScale Software Study: Software Challenges in Extreme Scale Systems", ExaScale Computing Study, DARPA IPTO, 2009
4. B. Obama. "A Strategy for American Innovation: Driving Towards Sustainable Growth and Quality Jobs", National Economic Council, 2009
5. F. Schmuck, R. Haskin, "GPFS: A Shared-Disk File System for Large Computing Clusters," FAST 2002
6. P. H. Carns, W. B. Ligon III, R. B. Ross, R. Thakur. "PVFS: A parallel file system for linux clusters", Proceedings of the 4th Annual Linux Showcase and Conference, 2000
7. P. Schwan. "Lustre: Building a file system for 1000-node clusters," Proc. of the 2003 Linux Symposium, 2003
8. I. Wladawsky-Berger. "Opinion - Challenges to exascale computing", International Science Grid this Week, April 2010; http://www.isgtw.org/feature/opinion-challengesexascale-computing
9. Wikipedia contributors. "Achilles' heel." Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 8 Jun. 2010. Web. 29 Jun. 2010
10. I. Raicu, Z. Zhang, M. Wilde, I. Foster, P. Beckman, K. Iskra, B. Clifford. "Toward Loosely Coupled Programming on Petascale Systems," IEEE SC 2008
11. Z. Zhang, A. Espinosa, K. Iskra, I. Raicu, I. Foster, M. Wilde. "Design and Evaluation of a Collective I/O Model for Loosely-coupled Petascale Programming", IEEE MTAGS08, 2008
12. E.N. Mootaz Elnozahy, et al. "System Resilience at Extreme Scale", Defense Advanced Research Project Agency (DARPA), 2007
13. S. Ghemawat, H. Gobioff, S.T. Leung. "The Google file system," 19th ACM SOSP, 2003
14. A. Bialecki, M. Cafarella, D. Cutting, O. O'Malley. "Hadoop: A Framework for Running Applications on Large Clusters Built of Commodity Hardware", 2005
15. J.S. Quarterman, S. Wilhelm, "UNIX, POSIX, and Open Systems: The Open Standards Puzzle", Addison-Wesley, Reading, MA, 1993
16. B. Bode, et al. "The Portable Batch Scheduler and the Maui Scheduler on Linux Clusters", Usenix, 4th Annual Linux Showcase & Conference, 2000
17. W. Gentzsch. "Sun Grid Engine: Towards Creating a Compute Power Grid," IEEE CCGrid, 2001
18. D. Thain, T. Tannenbaum, M. Livny, "Distributed Computing in Practice: The Condor Experience", Concurrency and Computation: Practice and Experience, 2005
19. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, M. Wilde. "Falkon: A Fast and Light-weight tasK executiON Framework," IEEE/ACM SC 2007
20. A. Birrell, M. Isard, C. Thacker, T. Wobber. "A design for high-performance flash disks", Operating Systems Review, 41(2):88-93, 2007
21. Intel News Release. "Intel, STMicroelectronics deliver industry's first phase change memory prototypes", http://www.intel.com/pressroom/ archive/releases/20080206corp.htm, 2008
22. W. Jiang, C. Hu, Y. Zhou, A. Kanevsky. "Are disks the dominant contributor for storage failures? A comprehensive study of storage subsystem failure characteristics", In USENIX Conference on File and Storage Technologies (FAST), pages 111-125, 2008
23. Intel Product Manual. "Intel® X25-E SATA Solid State Drive", http://download.intel.com/design/flash/nand/extreme/319984.pdf, 2009
24. H. Balakrishnan, M. F. Kaashoek, D. Karger, R. Morris, I. Stoica. "Looking up data in P2P systems", Communications of the ACM, 46(2):43-48, 2003
25. H. Stockinger, A. Samar, B. Allcock, I. Foster, K. Holtman, B. Tierney, "File and object replication in data grids," ACM HPDC-10, 2001
26. S. Podlipnig, L. Böszörmenyi. "A survey of Web cache replacement strategies", ACM Computing Surveys (CSUR), Volume 35 , Issue 4, Pages: 374 - 398, 2003
27. "Filesystem in Userspace", http://fuse.sourceforge.net/, 2011
28. W. Vogels, "Eventually consistent," ACM Queue, 2008.
29. I. Raicu, I. Foster, Y. Zhao. "Many-Task Computing for Grids and Supercomputers", IEEE MTAGS08, 2008
30. FusionFS: Fusion distributed File System, http://datasys.cs.iit.edu/ projects/FusionFS/index.html, 2011
31. ZHT: Zero-Hop Distributed Hash Table for High-End Computing, http://datasys.cs.iit.edu/projects/ZHT/index.html, 2011
32. P. Maymounkov, D. Mazieres. "Kademlia: A Peer-to-peer Information System Based on the XOR Metric", In Proceedings of IPTPS, 2002
33. S. Ratnasamy, P. Francis, M. Handley, R. Karp, S. Schenker, "A scalable content-addressable network," in Proceedings of SIGCOMM, pp. 161-172, 2001
34. I. Stoica, R. Morris, D. Karger, M.F. Kaashoek, H. Balakrishnan, "Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications", ACM SIGCOMM, pp. 149-160, 2001
35. A. Rowstron and P. Druschel, "Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems," in Proceedings of Middleware, pp. 329-350, 2001
36. B.Y. Zhao, L. Huang, J. Stribling, S.C. Rhea, A.D. Joseph, and J.D. Kubiatowicz. "Tapestry: A Resilient Global-Scale Overlay for Service Deployment", IEEE Journal on Selected Areas in Communication, VOL. 22, NO. 1, 2004
37. B. Fitzpatrick. "Distributed caching with memcached." Linux Journal, 2004(124):5, 2004
38. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, W. Vogels. "Dynamo: Amazon's Highly Available Key- Value Store." SIGOPS Operating Systems Review, 2007
39. H. Shen, C. Xu, and G. Chen. Cycloid: A Scalable Constant-Degree P2P Overlay Network. Performance Evaluation, 63(3):195-216, 2006 (Pubitemid 41705253)
40. Ketama, http://www.audioscrobbler.net/development/ketama/, 2011
41. Riak, https://wiki.basho.com/display/RIAK/Riak, 2011
42. Maidsafe-DHT, http://code.google.com/p/maidsafe-dht/, 2011
43. C-MPI, http://c-mpi.sourceforge.net/, 2011
44. M. Wilde, I. Raicu, A. Espinosa, Z. Zhang, B. Clifford, M. Hategan, K. Iskra, P. Beckman, I. Foster. "Extreme-scale scripting: Opportunities for large task-parallel applications on petascale computers", SciDAC09, 2009
45. C. Docan, M. Parashar, S. Klasky. "DataSpaces: An Interaction and Coordination Framework for Coupled Simulation Workflows", ACM HPDC 2010
46. H. Stern. "Managing NFS and NIS". O'Reilly & Associates, Inc., 1991
47. P.J. Braam. "The Coda distributed file system", Linux Journal, #50, 1998
48. D. Nagle, D. Serenyi, A. Matthews. "The panasas activescale storage cluster: Delivering scalable high bandwidth storage". In SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing, 2004
49. Microsoft Inc. "Distributed File System", http://www.microsoft. com/windowsserversystem/dfs/default.mspx, 2011
50. GlusterFS, http://www.gluster.com/, 2011
51. Isilon Systems. "OneFS", http://www.isilon.com/, 2011
52. "POHMELFS: Parallel Optimized Host Message Exchange Layered File System", http://www.ioremap.net/projects/pohmelfs/, 2011
53. F. Hupfeld, T. Cortes, B. Kolbeck, E. Focht, M. Hess, J. Malo, J. Marti, J. Stender, E. Cesario. "XtreemFS - a case for object-based storage in Grid data management". VLDB Workshop on Data Management in Grids, 2007
54. Y. Gu, R. Grossman, A. Szalay, A. Thakar. "Distributing the Sloan Digital Sky Survey Using UDT and Sector," e- Science 2006
55. CloudStore, http://kosmosfs.sourceforge.net/, 2011
56. S.A. Weil, S.A. Brandt, E.L. Miller, D.D.E. Long, C. Maltzahn. "Ceph: A scalable, highperformance distributed file system". In Proceedings of the 7th OSDI, 2006
57. W. Xiaohui, W.W. Li, O. Tatebe, X. Gaochao, H. Liang, J. Jiubin. "Implementing Data Aware Scheduling in Gfarm Using LSF Scheduler Plugin Mechanism," GCA05, 2005
58. X. Wei, Li Wilfred W., T. Osamu, G. Xu, L. Hu, J. Ju. "Integrating Local Job Scheduler - LSF with Gfarm," ISPA05, vol. 3758/2005, 2005
59. MooseFS, http://www.moosefs.org/, 2011
60. D. Thain, C. Moretti, J. Hemmes. "Chirp: A Practical Global Filesystem for Cluster and Grid Computing," JGC, Springer, 2008
61. S. Al-Kiswany, A. Gharaibeh, M. Ripeanu. "The Case for a Versatile Storage System", Workshop on Hot Topics in Storage and File Systems (HotStorage'09), 2009
62. P. Druschel, A. Rowstron. "Past: Persistent and anonymous storage in a peer-to-peer networking environment". In Proceedings of the 8th IEEE Workshop on Hot Topics in Operating Systems (HotOS), 2001
63. Circle, http://savannah.nongnu.org/projects/circle/, 2011
64. J. Ousterhout, et al. "The case for RAMclouds: Scalable high-performance storage entirely in DRAM". In Operating system review, 2009
65. J. Dean, S. Ghemawat. "MapReduce: Simplified Data Processing on Large Clusters," OSDI 2004
66. A.L. Chervenak, N. Palavalli, S. Bharathi, C. Kesselman, R. Schwartzkopf, "The Replica Location Service", IEEE HPDC, 2004
67. A. Chervenak, R. Schuler. "The Data Replication Service", Technical Report, USC ISI, 2006
68. K. Ranganathan I. Foster, "Simulation Studies of Computation and Data Scheduling Algorithms for Data Grids", Journal of Grid Computing, V1(1) 2003
69. T. Kosar. "A New Paradigm in Data Intensive Computing: Stork and the Data-Aware Schedulers," IEEE CLADE 2006
70. D.E. Culler, A. Arpaci-Dusseau, R. Arpaci-Dusseau, B. Chun, S. Lumetta, A. Mainwaring, R. Martin, C. Yoshikawa, F. Wong. "Parallel computing on the berkeley now", Symposium on Parallel Processing, 1997
71. R. Arpaci-Dusseau. "Run-time adaptation in river", ACM Transactions on Computer Systems, 21(1):36-86, 2003
72. Y. Zhao, I. Raicu, I. Foster, M. Hategan, V. Nefedova, M. Wilde. "Realizing Fast, Scalable and Reliable Scientific Computations in Grid Environments", Grid Computing Research Progress, Nova Publisher 2008
73. M. Branco, "DonQuijote - Data Management for the ATLAS Automatic Production System", Computing in High Energy and Nuclear Physics (CHEP04), 2004
74. D.L. Adams, K. Harrison, C.L. Tan. "DIAL: Distributed Interactive Analysis of Large Datasets", Computing in High Energy and Nuclear Physics (CHEP 06), 2006
75. A. Chervenak, et al. "High-Performance Remote Access to Climate Simulation Data: A Challenge Problem for Data Grid Technologies", Parallel Computing, Special issue on High performance computing with geographical data, 2003
76. M. Beynon, T.M. Kurc, U.V. Catalyurek, C. Chang, A. Sussman, J.H. Saltz. "Distributed Processing of Very Large Datasets with DataCutter", Parallel Computing, Vol. 27, No. 11, pp. 1457-1478, 2001
77. D.T. Liu, M.J. Franklin. "The Design of GridDB: A Data-Centric Overlay for the Scientific Grid", VLDB04, pp. 600-611, 2004
78. H. Andrade, T. Kurc, A. Sussman, J. Saltz. "Active Semantic Caching to Optimize Multidimensional Data Analysis in Parallel and Distributed Environments", Parallel Computing Journal, 2007
79. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. von Laszewski, I. Raicu, T. Stef-Praun, M. Wilde. "Swift: Fast, Reliable, Loosely Coupled Parallel Computation," IEEE Workshop on Scientific Workflows 2007
80. M. Bancroft, J. Bent, E. Felix, G. Grider, J. Nunez, S. Poole, R. Ross, E. Salmon, L. Ward. "HEC FSIO 2008 Workshop Report", High End Computing Interagency Working Group (HECIWG), Sponsored File Systems and I/O Workshop HEC FSIO, 2009
81. T. Hacker. "CAREER: Aon - An Integrative Approach to Petascale Fault Tolerance", NSF CCF CAREER Award, 2010
82. R. Vuduc. "CAREER: Autotuning Foundations for Exascale Computing", NSF CCF CAREER Award, 2010
83. J. Wang. "CAREER: Data-Intensive HPC Analytics: A Systems Approach Through Extended Interfaces, Data Restructuring and Data-centric Scheduling", NSF CCF CAREER Award, 2010
84. A. Butt. "CAREER: A Scalable Hierarchical Framework for High-Performance Data Storage", NSF CCF CAREER Award, 2008
85. K. Shen. "CAREER: System Support for Data-Intensive Online Applications", NSF CCF CAREER Award, 2005
86. A. Sirer. "CAREER: Building Robust, High-Performance Infrastructure Services Through Informed Resource Tradeoffs", NSF CNS CAREER Award, 2006
87. T. Kosar. "CAREER: Data-aware Distributed Computing for Enabling Large-scale Collaborative Science", NSF CNS CAREER Award, 2009
88. D. Thain. "CAREER: Data Intensive Grid Computing on Active Storage Clusters", NSF CNS CAREER Award, 2007
89. J. A. Chandy, "Active Storage Networks for High End Computing", NSF HECURA Award, 2006
90. J. Chandy. "Active Object Storage to Enable Scalable and Reliable Parallel File System", NSF HECURA Award, 2009
91. A. Maccabe, K. Schwann. "Petascale I/O for High End Computing", NSF HECURA Award, 2006
92. W. Ligon. "Improving Scalability in Parallel File Systems for High End Computing", NSF HECURA Award, 2006
93. A. Choudhary, M. Kandemir. "Scalable I/O Middleware and File System Optimizations for High-Performance Computing", NSF HECURA Award, 2006
94. X. Ma, A. Sivasubramaniam, Y. Zhou. "Applicationadaptive I/O Stack for Data-intensive Scientific Computing", NSF HECURA Award, 2006
95. K. Shen. "Concurrent I/O Management for Cluster-based Parallel Storages", NSF HECURA Award, 2006
96. H. Jiang, Yifeng Zhu. "SAM∧2 Toolkit: Scalable and Adaptive Metadata Management for High-End Computing", NSF HECURA Award, 2006
97. Y. Zhu, J. Hong. "A New Semantic-Aware Metadata Organization for Improved File-System Performance and Functionality in High-End Computing", NSF HECURA Award, 2009
98. T. Li, X. He, T. Zhang. "Cross-Layer Exploration of Non- Volatile Solid-State Memories to Achieve Effective I/O Stack for High-Performance Computing Systems", NSF HECURA Award, 2009
99. A. Szalay, H. Huang. "Balanced Scalable Architectures for Data-Intensive Supercomputing", NSF HECURA Award, 2009
100. C. Leiserson. "Microdata Storage Systems for High-End Computing", NSF HECURA Award, 2006
101. G.R. Gao. "A High Throughput Massive I/O Storage Hierarchy for PETA-scale High-end Architectures", NSF CPA Award 2007
102. R. Arpaci-Dusseau. "HaRD: The Wisconsin Hierarchically-Redundant, Decoupled Storage Project", NSF HECURA Award, 2009
103. V. Sarkar, J. Dennis, G. Gao. "Programming Models and Storage System for High Performance Computation with Many-Core Processors", NSF HECURA Award, 2009
104. I. Raicu. "Many-Task Computing: Bridging the Gap between High Throughput Computing and High Performance Computing", Doctorate Dissertation, University of Chicago, 2009
105. I. Raicu. "Many-Task Computing: Bridging the Gap between High Throughput Computing and High Performance Computing", VDM Verlag Dr. Muller Publisher, 2009
106. I. Raicu, et al. "Middleware Support for Many-Task Computing", Cluster Computing, The Journal of Networks, Software Tools and Applications, 2010
107. M. Wilde, I. Foster, K. Iskra, P. Beckman, Z. Zhang, A. Espinosa, M. Hategan, B. Clifford, I. Raicu. "Parallel Scripting for Applications at the Petascale and Beyond", IEEE Computer Nov. 2009 Special Issue on Extreme Scale Computing, 2009
108. I. Raicu, I. Foster, Y. Zhao, P. Little, C. Moretti, A. Chaudhary, D. Thain. "The Quest for Scalable Support of Data Intensive Workloads in Distributed Systems", ACM HPDC, 2009
109. I. Raicu, I. Foster, A. Szalay, G. Turcu. "AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis", TeraGrid Conference, 2006
110. I. Foster, Y. Zhao, I. Raicu, S. Lu. "Cloud Computing and Grid Computing 360-Degree Compared", IEEE GCE, 2008