HYDRAstor: A scalable secondary storage

Proceedings of the 7th USENIX Conference on File and Storage Technologies, FAST 2009

Volumn , Issue , 2009, Pages 197-210

  Dubnicki, Cezary ^a   Gryz, Leszek ^a   Heldt, Lukasz ^a   Kaczmarczyk, Michal ^a   Kilian, Wojciech ^a   Strzelczak, Przemyslaw ^a   Szczepkowski, Jerzy ^a   Ungureanu, Cristian ^a   Welnicki, Michal ^a  

^a NEC LABORATORIES AMERICA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION PROGRAMMING INTERFACES (API); DIRECTED GRAPHS; FILE ORGANIZATION; SECONDARY RECOVERY;

AUTOMATIC RECOVERY; COMMERCIAL IMPLEMENTATION; DIRECTED ACYCLIC GRAPH (DAG); DISTRIBUTED HASH TABLE; DUPLICATE ELIMINATION; ENTERPRISE MARKET; FAILURE RESILIENCY; PROGRAMMING MODELS;

DIGITAL STORAGE;

EID: 79955995661     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (43)

1. Plasmon and Pegasus. Archival Storage Total Cost of Ownership Analysis, 2005. http://www.pegasus-afs.com/PDFs/WhitePapers/Archive Storage TCO Report.pdf.
2. Vtf open, 2005. http://www.diligent.com/products:VTF-Open-2.
3. Overland storage unveils reo 9500d all-in-one deduplicating vtl appliance, 2007. http://www.overlandstorage.com.
4. EMC Corp. EMC Centera: content addressed storage system, 2008. http://www.emc.com/products/family/emc-centerafamily.htm?-openfolder=platform.
5. ABD-EL-MALEK, M., II, W. V. C., CRANOR, C., GANGER, G. R., HENDRICKS, J., KLOSTERMAN, A. J., MESNIER, M. P., PRASAD, M., SALMON, B., SAMBASIVAN, R. R., SINNAMOHIDEEN, S., STRUNK, J. D., THERESKA, E., WACHS, M., AND WYLIE, J. J. Ursa minor: Versatile cluster-based storage. In FAST (2005).
6. ADYA, A., BOLOSKY, W., CASTRO, M., CHAIKEN, R., CER-MAK, G., DOUCEUR, J., HOWELL, J., LORCH, J., THEIMER, M., AND WATTENHOFER, R. Farsite: Federated, available, and reliable storage for an incompletely trusted environment, 2002.
7. AJMANI, S., LISKOV, B., AND SHRIRA, L. Modular software upgrades for distributed systems. In European Conference on Object-Oriented Programming (ECOOP) (July 2006).
8. BINDEL, D., CHEN, Y., EATON, P., GEELS, D., GUMMADI, R., RHEA, S., WEATHERSPOON, H., WEIMER, W., WELLS, C., ZHAO, B., AND KUBIATOWICZ, J. Oceanstore: An extremely wide-area storage system. Tech. rep., Berkeley, CA, USA, 1999.
9. BLÖMER, J., KALFANE, M., KARPINSKI, M., KARP, R., LUBY, M., AND ZUCKERMAN, D. An xor-based erasure-resilient coding scheme. Tech. Rep. TR-95-048, International Computer Science Institute, August 1995.
10. DABEK, F., KAASHOEK, M. F., KARGER, D., MORRIS, R., AND STOICA, I. Wide-area cooperative storage with cfs. SIGOPS Oper. Syst. Rev. 35, 5 (2001), 202–215.
11. DRUSCHEL, P., AND ROWSTRON, A. PAST: A large-scale, persistent peer-to-peer storage utility. In HotOS VIII (Schloss Elmau, Germany, May 2001), pp. 75–80.
12. DUBNICKI, C., UNGUREANU, C., AND KILIAN, W. FPN: A Distributed Hash Table for Commercial Applications. In Proceedings of the Thirteenth International Symposium on High-Performance Distributed Computing (HPDC-13 2004) (Honolulu, Hawaii, June 2004), pp. 120–128.
13. EL MALEK, M. A., II, W. V. C., CRANOR, C., GANGER, G. R., HENDRICKS, J., KLOSTERMAN, A. J., MESNIER, M., PRASAD, M., SALMON, O., SAMBASIVAN, R. R., SINNAMOHIDEEN, S., STRUNK, J. D., THERESKA, E., WACHS, M., AND WYLIE, J. J. Early experiences on the journey towards self-* storage. IEEE Data Eng. Bulletin (2006).
14. ELLARD, D., AND MEGQUIER, J. Disp: Practical, efficient, secure and fault-tolerant distributed data storage. Trans. Storage 1, 1 (2005), 71–94.
15. GHEMAWAT, S., GOBIOFF, H., AND LEUNG, S.-T. The google file system. In SOSP (2003), pp. 29–43.
16. GUNAWI, H. S., AGRAWAL, N., ARPACI-DUSSEAU, A. C., ARPACI-DUSSEAU, R. H., AND SCHINDLER, J. Deconstructing Commodity Storage Clusters. In Proceedings of the 32nd International Symposium on Computer Architecture (Madison, WI, June 2005).
17. GUPTA, S. D. Network Magazine. Addressing Storage Management Challenges, 2002. http://www.networkmagazineindia.com/200212/cover1.shtml.
18. HAEBERLEN, A., MISLOVE, A., AND DRUSCHEL, P. Glacier: highly durable, decentralized storage despite massive correlated failures. In NSDI’05: Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation (Berkeley, CA, USA, 2005), USENIX Association, pp. 143–158.
19. LEE, E. K., AND THEKKATH, C. A. Petal: Distributed virtual disks. In Proceedings of the Seventh International Conference on Architectural Support for Programming Languages and Operating Systems (Cambridge, MA, 1996), pp. 84–92.
20. MAYMOUNKOV, P., AND MAZIERES, D. Kademlia: A peer-to-peer information system based on the xor metric. In In Proceedings of IPTPS02 (Cambridge, USA, March 2002).
21. MERRILL, D. Hitachi Data Systes. Storage Economics: Identifying and Reducing Operating Expenses in the Storage Infrastructure, 2003. http://www.hds.com/pdf/StorageEconomicsWHP153.pdf.
22. MESNIER, M., GANGER, G. R., AND RIEDEL, E. Object-based storage. IEEE Communications Magazine 41 (2003), 84–90.
23. NEC Corporation. HYDRAstor Grid Storage System, 2008. http://www.hydrastor.com.
24. QUINLAN, S., AND DORWARD, S. Venti: a new approach to archival storage. In First USENIX conference on File and Storage Technologies (Monterey,CA, 2002).
25. RATNASAMY, S., FRANCIS, P., HANDLEY, M., KARP, R., AND SCHENKER, S. A scalable content-addressable network. In SIGCOMM’01: Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications (New York, NY, USA, 2001), ACM, pp. 161–172.
26. RHEA, S., COX, R., AND PESTEREV, A. Fast, inexpensive content-addressed storage in foundation. In Proceedings of the 2008 USENIX Annual Technical Conference (Berkeley, CA, USA, 2008), USENIX Association, pp. 143–156.
27. ROWSTRON, A., AND DRUSCHEL, P. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. Lecture Notes in Computer Science 2218 (2001), 329+.
28. SAITO, Y., FROLUND, S., VEITCH, A., MERCHANT, A., AND SPENCE, S. Fab: building distributed enterprise disk arrays from commodity components. SIGOPS Oper. Syst. Rev. 38, 5 (2004), 48–58.
29. SHAPIRO, M. A survey of distributed garbage collection techniques. In In Proceedings of the 1995 International Workshop on Memory Management (1995), Springer-Verlag, pp. 211–249.
30. SOULES, C. A. N., APPAVOO, J., HUI, K., WISNIEWSKI, R. W., SILVA, D. D., GANGER, G. R., KRIEGER, O., STUMM, M., AUSLANDER, M., OSTROWSKI, M., ROSENBURG, B., AND XENIDIS, J. System support for online reconfiguration, June 2003.
31. STOICA, I., MORRIS, R., KARGER, D., KAASHOEK, M. F., AND BALAKRISHNAN, H. Chord: A scalable peer-to-peer lookup service for internet applications. In Proceedings of the 2001 conference on applications, technologies, architectures, and protocols for computer communications (2001), ACM Press, pp. 149–160.
32. STORER, M. W., GREENAN, K. M., MILLER, E. L., AND VORUGANTI, K. Pergamum: replacing tape with energy efficient, reliable, disk-based archival storage. In FAST’08: Proceedings of the 6th USENIX Conference on File and Storage Technologies (Berkeley, CA, USA, 2008), USENIX Association, pp. 1–16.
33. STRUNK, J. D., AND GANGER, G. R. A human organization analogy for self-* systems. In In First Workshop on Algorithms and Architectures for Self-Managing Systems, in conjunction with Federated Computing Research Conference (2003), pp. 1–6.
34. TANG, H., GULBEDEN, A., ZHOU, J., STRATHEARN, W., YANG, T., AND CHU, L. A self-organizing storage cluster for parallel data-intensive applications. In SC’04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing (Washington, DC, USA, 2004), IEEE Computer Society, p. 52.
35. THERESKA, E., SALMON, O., STRUNK, J., WACHS, M., EL MALEK, M. A., LOPEZ, J., AND GANGER, G. R. Stardust: Tracking activity in a distributed storage system. In ACM SIG-METRICS Conference on Measurement and Modeling of Computer Systems (Saint-Malo (2006), ACM Press, pp. 3–14.
36. WEATHERSPOON, H., AND KUBIATOWICZ, J. Erasure coding vs. replication: A quantitative comparison. In IPTPS’01: Revised Papers from the First International Workshop on Peer-to-Peer Systems (London, UK, 2002), Springer-Verlag, pp. 328–338.
37. WEIL, S. A., BRANDT, S. A., MILLER, E. L., LONG, D. D. E., AND MALTZAHN, C. Ceph: A scalable, high-performance distributed file system. In OSDI’06: 7th USENIX Symposium on Operating Systems Design and Implementation (Berkeley, CA, USA, 2006), USENIX Association, pp. 307–320.
38. WEIL, S. A., LEUNG, A. W., BRANDT, S. A., AND MALTZAHN, C. Rados: a scalable, reliable storage service for petabyte-scale storage clusters. In PDSW (2007), G. A. Gibson, Ed., ACM Press, pp. 35–44.
39. WELCH, B., UNANGST, M., ABBASI, Z., GIBSON, G., AND MUELLER, B. Scalable performance of the panasas parallel file system. In FAST’08: Proceedings of the 6th USENIX Conference on File and Storage Technologies (Berkeley, CA, USA, 2008), USENIX Association, pp. 17–33.
40. YOU, L. L., POLLACK, K. T., AND LONG, D. D. E. Deep store: An archival storage system architecture. In ICDE’05: Proceedings of the 21st International Conference on Data Engineering (Washington, DC, USA, 2005), IEEE Computer Society, pp. 804–8015.
41. ZHANG, Z., LIN, S., LIAN, Q., AND JIN, C. Repstore: A self-managing and self-tuning storage backend with smart bricks. In ICAC (2004), pp. 122–129.
42. ZHAO, B. Y., HUANG, L., STRIBLING, J., RHEA, S. C., JOSEPH, A. D., AND KUBIATOWICZ, J. D. Tapestry: A resilient global-scale overlay for service deployment. IEEE Journal on Selected Areas in Communications 22 (2004), 41–53.
43. ZHU, B., LI, K., AND PATTERSON, H. Avoiding the disk bottleneck in the data domain deduplication file system. In FAST’08: Proceedings of the 6th USENIX Conference on File and Storage Technologies (Berkeley, CA, USA, 2008), USENIX Association, pp. 1–14.