Reliable and randomized data distribution strategies for large scale storage systems

18th International Conference on High Performance Computing, HiPC 2011

Volumn , Issue , 2011, Pages

  Miranda, Alberto ^a   Effert, Sascha ^b   Kang, Yangwook ^c   Miller, Ethan L ^c   Brinkmann, Andre ^b   Cortes, Toni ^a,d  

^a BARCELONA SUPERCOMPUTING CENTER   (Spain)

^b UNIVERSITY OF PADERBORN   (Germany)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

DATA DISTRIBUTION; DATA ITEMS; DISTRIBUTION STRATEGIES; DYNAMIC ENVIRONMENTS; LARGE-SCALE STORAGE SYSTEMS; RULE BASED; SCALABLE STORAGE; SIMPLE++; STORAGE POOL; STORAGE SOLUTIONS;

VIRTUAL STORAGE;

EID: 84858020286     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/HiPC.2011.6152745     Document Type: Conference Paper

References (26)

1. S. Amarasinghe, D. Campbell, W. Carlson, A. Chien, W. Dally, E. Elnohazy, M. Hall, et al. Exascale software study: Software challenges in extreme scale systems. Technical report, sponsored by DARPA IPTO in the context of the ExaScale Computing Study, 2010.
2. A. Azagury, V. Dreizin, M. Factor, E. Henis, D. Naor, Y. Rinetzky, O. Rodeh, J. Satran, A. Tavory, and L. Yerushalmi. Towards an object store. In Proceedings of the 20th IEEE Conference on Mass Storage Systems and Technologies (MSST), pages 165-176, 2003.
3. M. Blaum, J. Brady, J. Bruck, and J. Menon. EVENODD: an optimal scheme for tolerating double disk failures in RAID architectures. In Proceedings of the 21st International Symposium on Computer Architecture (ISCA), pages 245-254, Apr. 1994.
4. A. Brinkmann and S. Effert. Redundant data placement strategies for cluster storage environments. In Proceedings of the 12th International Conference On Principles Of DIstributed Systems (OPODIS), Dec. 2008.
5. A. Brinkmann, S. Effert, F. Meyer auf der Heide, and C. Scheideler. Dynamic and Redundant Data Placement. In Proceedings of the 27th IEEE International Conference on Distributed Computing Systems (ICDCS), Toronto, Canada, June 2007.
6. A. Brinkmann, M. Heidebuer, F. Meyer auf der Heide, U. Rückert, K. Salzwedel, and M. Vodisek. V: Drive -costs and benefits of an out-of-band storage virtualization system. In Proceedings of the 21st IEEE Conference on Mass Storage Systems and Technologies (MSST), pages 153-157, 2004.
7. A. Brinkmann, K. Salzwedel, and C. Scheideler. Efficient, distributed data placement strategies for storage area networks. In Proceedings of the 12th ACM Symposium on Parallel Algorithms and Architectures (SPAA), pages 119-128, 2000.
8. A. Brinkmann, K. Salzwedel, and C. Scheideler. Compact, adaptive placement schemes for non-uniform distribution requirements. In Proceedings of the 14th ACM Symposium on Parallel Algorithms and Architectures (SPAA), pages 53-62, Winnipeg, Manitoba, Canada, 2002.
9. P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong, and S. Sankar. Row-diagonal parity for double disk failure correction. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies (FAST), pages 1-14, San Francisco, CA, 2004.
10. T. Cortes and J. Labarta. Extending heterogeneity to RAID level 5. In Proceedings of the USENIX Annual Technical Conference, pages 119- 132, Boston, Massachusetts, June 2001.
11. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon's highly available key-value store. ACM SIGOPS Operating Systems Review, 41(6):205-220, 2007. (Pubitemid 351429377)
12. A. Devulapalli, D. Dalessandro, and P. Wyckoff. Data Structure Consistency Using Atomic Operations in Storage Devices. In Proceedings of the 5th International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI), pages 65-73, Baltimore, USA, 2008.
13. J. Gonzalez and T. Cortes. Distributing Orthogonal Redundancy on Adaptive Disk Arrays. In Proceedings of the International Conference on Grid computing, high-performAnce and Distributed Applications (GADA), Monterrey, Mexico, Nov. 2008.
14. R. J. Honicky and E. L. Miller. A fast algorithm for online placement and reorganization of replicated data. In Proceedings of the 17th IEEE International Parallel and Distributed Processing Symposium (IPDPS), Nice, France, Apr. 2003.
15. R. J. Honicky and E. L. Miller. Replication Under Scalable Hashing: A Family of Algorithms for Scalable Decentralized Data Distribution. In Proceedings of the 18th IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2004.
16. N. L. Johnson and S. Kotz. Urn Models and Their Applications. John Wiley and Sons, New York, 1977.
17. D. Karger, E. Lehman, T. Leighton, M. Levine, D. Lewin, and R. Panigrahy. Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web. In Proceedings of the 29th ACM Symposium on Theory of Computing (STOC), pages 654-663, El Paso, Texas, USA, 1997.
18. M. Mense and C. Scheideler. Spread: An adaptive scheme for redundant and fair storage in dynamic heterogeneous storage systems. In Proceedings of the 19th ACM-SIAM Symposium on Discrete Algorithms (SODA), San Francisco, California, Jan. 2008.
19. M. Mitzenmacher. The Power of Two Choices in Randomized Load Balancing. PhD thesis, Computer Science Department, University of California at Berkeley, 1996.
20. D. A. Patterson, G. Gibson, and R. H. Katz. A Case for Redundant Arrays of Inexpensive Disks (RAID). In Proceedings of the 1988 ACM Conference on Management of Data (SIGMOD), pages 109-116, June 1988.
21. P. Sanders. Reconciling simplicity and realism in parallel disk models. In Proceedings of the 12th ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 67-76. SIAM, Philadelphia, PA, 2001.
22. C. Schindelhauer and G. Schomaker. Weighted distributed hash tables. In Proceedings of the 17th ACM Symposium on Parallel Algorithms and Architectures (SPAA), pages 218-227, Las Vegas, Nevada, USA, July 2005. (Pubitemid 43206574)
23. I. Stoica, R. Morris, D. Liben-Nowell, D. Karger, M. Kaashoek, F. Dabek, and H. Balakrishnan. Chord: a scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Transactions on Networking, 11(1):17-32, 2003.
24. S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn. Ceph: A scalable, high-performance distributed file system. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI), pages 307-320, Seattle, WA, USA, Nov. 2006.
25. S. A. Weil, S. A. Brandt, E. L. Miller, and C. Maltzahn. CRUSH: Controlled, Scalable And Decentralized Placement Of Replicated Data. In Proceedings of the ACM/IEEE Conference on Supercomputing, Tampa, FL, Nov. 2006.
26. W. Zheng and G. Zhang. Fastscale: Accelerate raid scaling by minimizing data migration. In Proceedings of the 9th USENIX Conference on File and Storage Technologies (FAST), Feb. 2011.