Joint latency and cost optimization for erasure-coded data center storage

Performance Evaluation Review

Volumn 42, Issue 2, 2014, Pages 3-14

  Xiang, Yu ^a   Lan, Tian ^a   Aggarwal, Vaneet ^b   Chen, Yih Farn R ^b  

^a GEORGE WASHINGTON UNIVERSITY   (United States)

^b AT AND T LABS RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FORWARD ERROR CORRECTION; MULTIPROCESSING SYSTEMS;

CONVEX APPROXIMATION; COST OPTIMIZATION; DISTRIBUTED DATA; DISTRIBUTED STORAGE SYSTEM; LATENCY REDUCTION; SCHEDULING POLICIES; SERVICE TIME DISTRIBUTION; THREE DIMENSIONS;

DIGITAL STORAGE;

EID: 84908217731     PISSN: 01635999     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2667522.2667524     Document Type: Conference Paper

References (46)

1. A. D. Luca and M. Bhide, "Storage virtualization for dummies, Hitachi Data Systems Edition", John and Wiley Publishing, 2009.
2. Amazon S3, "Amazon Simple Storage Service", available online at http://aws.amazon.com/s3/.
3. Sathiamoorthy, Maheswaran, et al. "Xoring elephants: Novel erasure codes for big data." Proceedings of the 39th international conference on Very Large Data Bases. VLDB Endowment, 2013.
4. Fikes, Andrew. "Storage architecture and challenges." Talk at the Google Faculty Summit, available online at http://bit.ly/nUylRW, 2010.
5. A. G. Dimakis, K. Ramchandran, Y. Wu, C. Suh, "A Survey on Network Codes for Distributed Storage", arXiv:1004.4438, Apr. 2010
6. A. Fallahi and E. Hossain, "Distributed and energy-Aware MAC for differentiated services wireless packet networks: a general queuing analytical framework", IEEE CS, CASS, ComSoc, IES, SPS, 2007.
7. F. Baccelli, A. Makowski, and A. Shwartz, "The fork-join queue and related systems with synchronization constraints: stochastic ordering and computable bounds, Advances in Applied Probability, pp. 629660, 1989.
8. A. S. Alfa, "Matrix-geometric solution of discrete time MAP/PH/1 priority queue", Naval research logistics, vol. 45, 00. 23-50, 1998.
9. J. H. Kim and J. K. Lee, "Performance of carrier sense multiple access with collision avoidance in wireless LANs", Proc. IEEE IPDS., 1998.
10. E. Ziouva and T. Antoankopoulos, "CSMA/CA Performance under high traffic conditions: throughput and delay analysis", Computer Comm, vol. 25, pp. 313-321, 2002.
11. C. Angllano, R. Gaeta and M. Grangetto, "Exploiting Rateless Codes in Cloud Storage Systems", IEEE Transactions on Parallel and Distributed Systems, Pre-print 2014.
12. N. E. Taylor and Z. G. Ives, "Reliable storage and querying for collaborative data sharing systems", IEEE ICED Conference, 2010.
13. R. Rosemark and W. C. Lee, "Decentralizing query processing in sensor networks", Proceedings of the second MobiQuitous: networking and services, 2005
14. Dimakis, Alexandros D G, "Distributed data storage in sensor networks using decentralized erasure codes", Signals, Systems and Computers, 2004. Conference Record of the Thirty-Eighth Asilomar., 2004.
15. R. Rojas-Cessa, L. Cai and T. Kijkanjanarat, "Scheduling memory access on a distributed cloud storage network", IEEE 21st annual WOCC, 2012.
16. M. K. Aguilera, R. Janakiraman, L. Xu, "Using Erasure Codes Efficiently for Storage in a Distributed System", Proceedings of the 2005 International Conference on DSN, pp. 336-345, 2005.
17. S. Chen, K. R. Joshi and M. A. Hiltunem, "Link Gradients: Predicting the Impact of Network Latency on Multi-Tier Applications", Proc. IEEE INFOCOM, 2009.
18. Q. Lv, P. Cao, E. Cohen, K. Li and S. Shenker, "Search and replication in unstructured peer-to-peer networks", Proceedings of the 16th ICS, 2002.
19. H. Kameyam and Y. Sato, "Erasure Codes with Small Overhead Factor and Their Distributed Storage Applications", CISS'07. 41st Annual Conference, 2007.
20. H. Y. Lin, and W. G. Tzeng, "A Secure Decentralized Erasure Code for Distributed Networked Storage", Parallel and Distributed Systems, IEEE Transactions, 2010.
21. W. Luo, Y. Wang and Z. Shen, "On the impact of erasure coding parameters to the reliability of distributed brick storage systems", Cyber-Enabled Distributed Computing and Knowledge Discovery, International Conference, 2009.
22. J. Li, "Adaptive Erasure Resilient Coding in Distributed Storage", Multimedia and Expo, 2006 IEEE International Conference, 2006.
23. K. V. Rashmi, N. Shah, and V. Kumar, "Enabling node repair in any erasure code for distributed storage", Proceedings of IEEE ISIT, 2011.
24. X. Wang, Z. Xiao, J. Han and C. Han, "Reliable Multicast Based on Erasure Resilient Codes over InfiniBand", Communications and Networking in China, First International Conference, 2006.
25. S. Mochan and L. Xu, "Quantifying Benefit and Cost of Erasure Code based File Systems." Technical report available at http://nisl.wayne.edu/Papers/Tech/cbefs.pdf, 2010.
26. H. Weatherspoon and J. D. Kubiatowicz, "Erasure Coding vs. Replication: A Quantitative Comparison. " In Proceedings of the First IPTPS, 2002
27. A. Abdelkefi and J. Yuming, "A Structural Analysis of Network Delay", Ninth Annual CNSR, 2011.
28. A. B. Downey, "The structural cause of file size distributions", Proceedings of Ninth International Symposium on MASCOTS, 2011.
29. F. Paganini, A. Tang, A. Ferragut and L. L. H. Andrew, "Network Stability Under Alpha Fair Bandwidth Allocation With General File Size Distribution", IEEE Transactions on Automatic Control, 2012.
30. P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong and S. Sankar, "Row-diagonal parity for double disk failure correction", In Proceedings of the 3rd USENIX FAST', pp. 1-14, 2004.
31. B. Calder, J. Wang, A. Ogus, N. Nilakantan, A. Skjolsvold, S. McKelvie, Y. Xu, S. Srivastav, J. Wu, H. Simitci, et al., "Windows azure storage: A highly available cloud storage service with strong consistency", In Proceedings of the Twenty-Third ACM SOSP, pages 143-157, 2011.
32. O. Khan, R. Burns, J. Plank, W. Pierce, and C. Huang, "Rethinking erasure codes for cloud file systems: Minimizing I/O for recovery and degraded reads", In Proceedings of FAST, 2012.
33. L. Huang, S. Pawar, H. Zhang, and K. Ramchandran, "Codes can reduce queueing delay in data centers", in Proc. IEEE ISIT, 2012.
34. G. Ananthanarayanan, S. Agarwal, S. Kandula, A. Greenberg, and I. Stoica, "Scarlett: Coping with skewed content popularity in MapReduce", Proceedings of ACM EuroSys, 2011.
35. M. Bramson, Y. Lu, and B. Prabhakar, "Randomized load balancing with general service time distributions", Proceedings of ACM Sigmetrics, 2010.
36. Y. Xiang, T. Lan, V. Aggarwal, Y. Chen, "Joint Latency and Cost Optimization for Erasure-coded Data Center Storage", Full paper available at http://arxiv.org/pdf/1404.4975.pdf, 2014.
37. Y. Lu, Q. Xie, G. Kliot, A. Geller, J. Larus, and A. Greenberg, "Joinidle-queue: A novel load balancing algorithm for dynamically scalable web services", 29th IFIPPERFORMANCE, 2010.
38. D. Bertsimas and K. Natarajan, "Tight bounds on Expected Order Statistics", Probability in the Engineering and Informational Sciences, 2006.
39. L. Huang, S. Pawar, H. Zhang and K. Ramchandran, "Codes Can Reduce Queueing Delay in Data Centers", Journals CORR, vol. 1202. 1359, 2012.
40. S. Boyd and L. Vandenberghe, "Convex Optimization", Cambridge University Press, 2005.
41. G. Joshi, Y. Liu, and E. Soljanin, "On the Delay-Storage Trade-off in Content Download from Coded Distributed Storage Systems", arXiv:1305.3945v1, May 2013.
42. B. Warner, Z. Wilcox-O'Hearn and R. Kinninmont, "Tahoe-LAFS docs", available online at https://tahoe-lafs.org/trac/tahoe-lafs.
43. N. Shah, K. Lee, and K. Ramachandran, "The MDS queue: analyzing latency performance of codes and redundant requests", arXiv:1211.5405, Nov. 2012.
44. L. T. Hoai An and P. D. Tao, "The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Non-convex Optimization Problems", Annals of Operations Research, vol. 133, Issue 1-4, pp. 23-46, Jan 2005.
45. MOSEK, "MOSEK: High performance software for large-scale LP, QP, SOCP, SDP and MIP", available online at http://www.mosek.com/.
46. T. Angell, "The Farkas-Minkowski Theorem". Lecture nodes available online at www.math.udel.edu/~angell/Opt/farkas.pdf, 2002.