CDRM: A cost-effective dynamic replication management scheme for cloud storage cluster

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2010, Pages 188-196

  Wei, Qingsong ^a   Veeravalli, Bharadwaj ^b   Gong, Bozhao ^b   Zeng, Lingfang ^c   Feng, Dan ^c  

^a DATA STORAGE INSTITUTE   (Singapore)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^c HUAZHONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

Author keywords

Cost effective; Dynamic replication management; HDFS; Load balance

Indexed keywords

CLOUD STORAGES; DATA AVAILABILITY; DATA REPLICATION; DISTRIBUTED FILE SYSTEMS; DYNAMIC REPLICATION MANAGEMENT; EFFECTIVE DYNAMICS; HDFS; IS COSTS; LOAD BALANCE; LOAD-BALANCING; NODE CAPACITY; REPLICA PLACEMENT;

BLOCKING PROBABILITY; CLUSTER COMPUTING; COST EFFECTIVENESS; FILE ORGANIZATION; NETWORK MANAGEMENT;

COSTS;

EID: 78649459815     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTER.2010.24     Document Type: Conference Paper

References (20)

1. Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung, The Google File System, Proceedings of 19th ACM Symposium on Operating Systems Principles(SOSP 2003), New York, USA, October, 2003.
2. Jeffrey Dean and Sanjay Ghemawat. Mapreduce: Simplified data processing on large clusters. pages 137-150, December 2004.
3. The Apache Software Foundation. Hadoop. http://hadoop.apache.org/core/, 2009.
4. Amazon-S3. Amazon simple storage service (amazon s3). http://www.amazon.com/s, 2009.
5. Philip H. Carns, Walter B. Ligon, III, Robert B. Ross, and Rajeev Thakur. Pvfs: A parallel file system for linux clusters. Proceedings of the 4th Annual Linux Showcase and Conference, pages 317 - 327. USENIX.
6. Peter Honeyman, Dean Hildebrand, Dean Hildebrand, Lee Ward, and Lee Ward. Large files, small writes, and pnfs. Proceedings of the 20thACM International Conference on Supercomputing, pages 116-124, 2006.
7. Frank Schmuck and Roger Haskin. Gpfs: A shared-disk file system for large computing clusters. Proceedings of the 2002 Conference on File and Storage Technologies (FAST), pages 231-244, 2002.
8. Garth A. Gibson, David F. Nagle, Khalil Amiri, Jeff Butler, Fay W. Chang, Howard Gobioff, Charles Hardin, Erik Riedel, David Rochberg, and Jim Zelenka. A cost-effective, high-bandwidth storage architecture. SIGPLAN Not., 33(11):92-103, 1998.
9. Dan Feng, Lingjun Qin. Adaptive Object Placement in Object-Based Storage Systems with Minimal Blocking Probability. Proceeding of the 20th international conference on Advanced Information Networking and Applications (AINA'06), 2006.
10. C. Cunha, A. Bestavros, and M. Crovella, Characteristics of WWW Client-Based Traces, Technical Report 1995-010, Boston Univ., 1995.
11. Lustre: A Scalable, High-performance File System. Whitepaper, Cluster File System, Inc. http://www.lustre.org/docs/lustre.pdf.
12. Hai Huang, Wanda Hung, Kang G. Shin: FS2: dynamic data replication in free disk space for improving disk performance and energy consumption. Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP 2005), Brighton, UK, October, 2005.
13. Sage A. Weil Scott A. Brandt Ethan L. Miller Darrell D. E. Long, Ceph: A Scalable, High-Performance Distributed File System, Proceeding of 7th conference on operating system design and implementation (OSDI'06), November, 2006.
14. Thanasis Loukopoulos, Ishfaq Ahmad and Dimitris Papadias, An Overview of Data Replication on the Internet, Proceedings of the International Symposium on Parallel Architectures, Algorithms and Networks (ISPAN.02).
15. K. Ranganathan and I. Foster, Identifying Dynamic Replication Strategies for a High Performance Data Grid, presented at International Workshop on Grid Computing, Denver, CO, 2001.
16. Napster project home page. http://ww.napster.com.
17. FreeNet home page. http://freenet.sourceforge.net.
18. http://now.cs.berkeley.edu/Xfs/AuspexTraces/auspex.html
19. K.Ranganathan, A.Iamnitchi and I.Foster, Improving Data Availability through Dynamic Model-Driven Replication in Large Peer-to-Peer Communities, Proceedings of the Workshop on Global and P2P Computing on Large Scale Distributed Systems, Berlin, May 2002.
20. Sage Weil, Scott A. Brandt, Ethan L. Miller, Carlos Maltzahn, CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data, Proceedings of SC '06, November 2006.