Abstract storage: Moving file format-specific abstractions into petabyte-scale storage systems

Proceedings of the IEEE International Symposium on High Performance Distributed Computing

Volumn , Issue , 2009, Pages 31-39

  Buck, Joe B ^a   Watkins, Noah ^a   Maltzahn, Carlos ^a   Brandt, Scott A ^a  

^a University of California Santa Cruz   (United States)

Author keywords

Active storage; Data intensive computing; Structured storage

Indexed keywords

ACTIVE STORAGE; APPLICATION-SPECIFIC; DATA PARTITIONING; DATA TRANSPORT; DATA-INTENSIVE COMPUTING; DATA-INTENSIVE WORKLOADS; DISTRIBUTED COMPUTATIONS; FILE FORMATS; GENERAL SOLUTIONS; HIGH-END COMPUTING; IN-SITU; IN-SITU DATA; ORDERS OF MAGNITUDE; PERFORMANCE MANAGEMENT; STORAGE NODES; STORAGE SYSTEMS;

ABSTRACTING; DISTRIBUTED COMPUTER SYSTEMS; IN SITU PROCESSING; INDEXING (MATERIALS WORKING);

DATA PROCESSING;

EID: 77953592428     PISSN: 10828907     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1552280.1552284     Document Type: Conference Paper

References (31)

1. Amazon simpleDB. http://aws.amazon.com/simpledb/.
2. Apache couchDB. http://couchdb.apache.org/.
3. Cassandra project. http://incubator.apache.org/cassandra/.
4. A. Acharya, M. Uysal, and J. Saltz. Active disks: programming model, algorithms and evaluation. SIGPLAN Not., 33(11):81-91, 1998.
5. D. O. Bigelow, S. Iyer, T. Kaldewey, R. C. Pineiro, A. Povzner, S. A. Brandt, R. A. Golding, T. M. Wong, and C. Maltzahn. End-to-end performance management for scalable distributed storage. In PDSW'07: Proceedings of the 2nd international workshop on Petascale data storage, pages 30-34, New York, NY, USA, 2007. ACM.
6. P. J. Braam. The Lustre storage architecture. http://www.lustre.org/ documentation.html, Aug. 2004. Cluster File Systems, Inc.
7. S. A. Brandt, C. Maltzahn, A. Povzner, R. Pineiro, A. Shewmaker, and T. Kaldewey. An integrated model for performance management in a distributed system. In 4th International Workshop on Operating Systems Platforms for Embedded Real-Time Applications (OSPERT 2008), Prague, Czeck Republic, July 2008.
8. P. H. Carns, W. B. Ligon III, R. B. Ross, and R. Thakur. PVFS: A parallel file system for linux clusters. In Proceedings of the 4th Annual Linux Showcase and Conference, pages 317-327, Atlanta, GA, 2000. USENIX Association.
9. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. t E. Gruber. Bigtable: A distributed storage system for structured data. In osdi06, Seattle, WA, November 2006.
10. P. F. Corbett and D. G. Feitelson. The vesta parallel file system. ACM Trans. Comput. Syst., 14(3):225-264, 1996.
11. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. In OSDI'04: Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, pages 137-150, Berkeley, CA, USA, 2004. USENIX Association.
12. I. C. for Information Technology Standards. SCSI Object-Based Storage Device Commands - 3 (OSD-3). Project Proposal for a new INCITS Standard T10/08-331r1, International Committee for Information Technology Standards, September 11 2008.
13. R. Grimm, M. M. Swift, and H. M. Levy. Revisiting structured storage: A transactional record store. Technical report, University of Washington, http://pages.cs.wisc.edu/swift/papers/tr00-04-01.pdf, 2000.
14. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In EuroSys'07: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, pages 59-72, New York, NY, USA, 2007. ACM.
15. T. Kaldewey, A. Povzner, T. Wong, R. Golding, S. A. Brandt, and C. Maltzahn. Virtualizing disk performance. In The IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS 2008), St. Louis, Missouri, April 2008. Springer Journal of Real-Time Systems Award for Best Student Paper.
16. K. Keeton, D. A. Patterson, and J. M. Hellerstein. A case for intelligent disks (idisks). SIGMOD Rec., 27(3):42-52, 1998.
17. P. Kogge, K. Bergman, S. Borkar, D. Campbell, W. Carson, W. Dally, M. Denneau, P. Franzon, W. Harrod, K. Hill, J. Hiller, S. Karp, S. Keckler, D. Klein, R. Lucas, M. Richards, A. Scarpelli, S. Scott, A. Snavely, T. Sterling, R. S. Williams, and K. Yellick. Exascale computing study: Technology challenges in achieving exascale systems. Technical Report TR-2008-13, DARPA, September 28 2008.
18. J. F. Lofstead, S. Klasky, K. Schwan, N. Podhorszki, and C. Jin. Flexible io and integration for scientific codes through the adaptable io system (adios). In CLADE'08: Proceedings of the 6th international workshop on Challenges of large applications in distributed environments, pages 15-24, New York, NY, USA, 2008. ACM.
19. J. MacCormick, N. Murphy, M. Najork, C. A. Thekkath, and L. Zhou. Boxwood: abstractions as the foundation for storage infrastructure. In OSDI'04: Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, pages 8-8, Berkeley, CA, USA, 2004. USENIX Association.
20. J. Piernas, J. Nieplocha, and E. J. Felix. Evaluation of active storage strategies for the lustre parallel file system. In SC'07: Proceedings of the 2007 ACM/IEEE conference on Supercomputing, pages 1-10, New York, NY, USA, 2007. ACM.
21. A. Povzner, T. Kaldewey, S. Brandt, R. Golding, T. M. Wong, and C. Maltzahn. Efficient guaranteed disk request scheduling with fahrrad. SIGOPS Oper. Syst. Rev., 42(4):13-25, 2008.
22. A. Povzner, T. Kaldewey, S. A. Brandt, R. Golding, T. Wong, and C. Maltzahn. Efficient guaranteed disk request scheduling with fahrrad. In Eurosys 2008, Glasgow, Scottland, March 31 - April 4 2008.
23. E. Riedel, G. A. Gibson, and C. Faloutsos. Active storage for large-scale data mining and multimedia. In VLDB'98: Proceedings of the 24rd International Conference on Very Large Data Bases, pages 62-73, San Francisco, CA, USA, 1998. Morgan Kaufmann Publishers Inc.
24. R. Ross. Meeting the needs of computational science at extreme scale. slides presented at University of California Santa Cruz, January 2009.
25. F. Schmuck and R. Haskin. Gpfs: A shared-disk file system for large computing clusters. In FAST'02: Proceedings of the 1st USENIX Conference on File and Storage Technologies, page 19, Berkeley, CA, USA, 2002. USENIX Association.
26. M. Sivathanu, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Evolving rpc for active storage. In ASPLOS-X: Proceedings of the 10th international conference on Architectural support for programming languages and operating systems, pages 264-276, New York, NY, USA, 2002. ACM.
27. M. Stonebraker, J. Becla, D. Dewitt, K.-T. Lim, D. Maier, O. Ratzesberger, and S. Zdonik. Requirements for science data bases and SciDB. In CIDR Perspectives 2009, January 2009.
28. S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn. Ceph: a scalable, high-performance distributed file system. In OSDI'06: Proceedings of the 7th symposium on Operating systems design and implementation, pages 307-320, Berkeley, CA, USA, 2006. USENIX Association.
29. B. Welch, M. Unangst, Z. Abbasi, G. Gibson, B. Mueller, J. Small, J. Zelenka, and B. Zhou. Scalable performance of the panasas parallel file system. In FAST'08: Proceedings of the 6th USENIX Conference on File and Storage Technologies, pages 1-17, Berkeley, CA, USA, 2008. USENIX Association.
30. R. Wickremesinghe, J. Chase, and J. Vitter. Distributed computing with load-managed active storage. High Performance Distributed Computing, 2002. HPDC-11 2002. Proceedings. 11th IEEE International Symposium on, pages 13-23, 2002.
31. P. Widener, M. Wolf, H. Abbasi, M. Barrick, J. Lofstead, J. Pullikottil G. Eisenhauer, A. Gavrilovsk, S. Klasky, R. Oldfield, P. G. Bridges, A. B. Maccabe, and K. Schwan. Structured streams: Data services for petascale science environments. Technical Report TR-CS-2007-17, University of New Mexico, November 2007.