The SCADS director: Scaling a distributed storage system under stringent performance requirements

Proceedings of FAST 2011: 9th USENIX Conference on File and Storage Technologies

Volumn , Issue , 2011, Pages 163-176

  Trushkowsky, Beth ^a   Bodík, Peter ^a   Fox, Armando ^a   Franklin, Michael J ^a   Jordan, Michael I ^a   Patterson, David A ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MULTIPROCESSING SYSTEMS; RESOURCE ALLOCATION;

CLOUD COMPUTING ENVIRONMENTS; DISTRIBUTED STORAGE SYSTEM; DYNAMIC RESOURCE ALLOCATIONS; ECONOMIC INCENTIVE; LATENCY MEASUREMENTS; PERFORMANCE REQUIREMENTS; SERVICE LEVEL OBJECTIVE; WORKLOAD PATTERNS;

DIGITAL STORAGE;

EID: 85077073185     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. H. Amur, J. Cipar, V. Gupta, G. R. Ganger, M. A. Kozuch, and K. Schwan. Robust and Flexible Power-Proportional Storage. In SoCC: ACM Symposium on Cloud Computing, 2010.
2. E. Anderson, M. Hobbs, K. Keeton, S. Spence, M. Uysal, and A. Veitch. Hippodrome: Running Circles Around Storage Administration. In FAST: Conference on File and Storage Technologies, 2002.
3. Apache. Cassandra. incubator.apache.org/cassandra, 2010.
4. Apache. HBase. hadoop.apache.org/hbase, 2010.
5. M. Arlitt and T. Jin. Workload Characterization of the 1998 World Cup Web Site. Technical Report HPL-1999-35R1, HP Labs, 1999.
6. M. Armbrust, A. Fox, D. Patterson, N. Lanham, H. Oh, B. Trushkowsky, and J. Trutna. SCADS: Scale-independent Storage for Social Computing Applications. In Conference on Innovative Data Systems Research (CIDR), 2009.
7. K. J. Åström. Introduction to Stochastic Control Theory. Academic Press, 1970.
8. H. Balakrishnan, M. F. Kaashoek, D. Karger, R. Morris, and I. Stoica. Looking Up Data in P2P Systems. Communications of the ACM, 46(2), February 2003.
9. P. Barford and M. Crovella. Generating Representative Web Workloads for Network and Server Performance Evaluation. In Proceedings of the ACM SIGMETRICS Joint International Conference, 1998.
10. P. Bodík et al. Combining Visualization and Statistical Analysis to Improve Operator Confidence and Efficiency for Failure Detection and Localization. In International Conference on Autonomic Computing (ICAC), 2005.
11. P. Bodík, A. Fox, M. J. Franklin, M. I. Jordan, and D. Patterson. Characterizing, Modeling, and Generating Workload Spikes for Stateful Services. In SoCC: ACM Symposium on Cloud Computing, 2010.
12. P. Bodík, R. Griffith, C. Sutton, A. Fox, M. I. Jordan, and D. A. Patterson. Automatic Exploration of Datacenter Performance Regimes. In Proceedings of the 1st Workshop on Automated Control for Datacenters and Clouds, 2009.
13. P. Bodík, R. Griffith, C. Sutton, A. Fox, M. I. Jordan, and D. A. Patterson. Statistical Machine Learning Makes Automatic Control Practical for Internet Datacenters. In Workshop on Hot Topics in Cloud Computing (HotCloud), 2009.
14. F. Chang et al. Bigtable: A Distributed Storage System for Structured Data. In OSDI: Symposium on Operating Systems Design and Implementation, 2006.
15. J. Chase, D. C. Anderson, P. N. Thakar, A. M. Vahdat, and R. P. Doyle. Managing Energy and Server Resources in Hosting Centers. In Symposium on Operating Systems Principles (SOSP), 2001.
16. J. Chen, G. Soundararajan, and C. Amza. Autonomic Provisioning of Backend Databases in Dynamic Content Web Servers. In International Conference on Autonomic Computing (ICAC), 2006.
17. B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. PNUTS: Yahoo!'s Hosted Data Serving Platform. In Proceedings of the International Conference on Very Large Databases (VLDB), 2008.
18. B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking cloud serving systems with YCSB. In SoCC: ACM Symposium on Cloud Computing, 2010.
19. J. Dean. Evolution and Future Directions of Large-scale Storage and Computation Systems at Google. In SoCC: ACM Symposium on Cloud Computing, 2010.
20. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon's Highly Available Key-value Store. In Symposium on Operating Systems Principles (SOSP), 2007.
21. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google File System. In Symposium on Operating Systems Principles (SOSP), 2003.
22. J. L. Hellerstein, V. Morrison, and E. Eilebrecht. Optimizing Concurrency Levels in the .NET ThreadPool: A Case Study of Controller Design and Implementation. In Workshop on Feedback Control Implementation and Design in Computing Systems and Networks, 2008.
23. D. Kusic et al. Power and Performance Management of Virtualized Computing Environments Via Lookahead Control. In International Conference on Autonomic Computing (ICAC), 2008.
24. W. LeFebvre. CNN.com: Facing a world crisis. www.tcsa.org/lisa2001/cnn.txt, 2001.
25. H. C. Lim, S. Babu, and J. S. Chase. Automated Control for Elastic Storage. In International Conference on Autonomic Computing (ICAC), 2010.
26. X. Liu, J. Heo, L. Sha, and X. Zhu. Adaptive Control of Multi-Tiered Web Applications Using Queueing Predictor. Network Operations and Management Symposium, April 2006.
27. C. Lu, G. A. Alvarez, and J. Wilkes. Aqueduct: Online Data Migration with Performance Guarantees. In FAST: Conference on File and Storage Technologies, 2002.
28. D. Narayanan, A. Donnelly, E. Thereska, S. Elnikety, and A. I. T. Rowstron. Everest: Scaling Down Peak Loads Through I/O Off-Loading. In OSDI: Symposium on Operating Systems Design and Implementation, 2008.
29. A. Rabkin and R. H. Katz. Chukwa: A System for Reliable Large-scale Log Collection. Master's thesis, EECS Department, University of California, Berkeley, Mar 2010.
30. J. A. Rossiter. Model Based Predictive Control: A Practical Approach. CRC Press, 2003.
31. J. Rothschild. High Performance at Massive Scale - Lessons Learned at Facebook. http://cns.ucsd.edu/lecturearchive09.shtml#Roth, October 2009.
32. G. Soundararajan, C. Amza, and A. Goel. Database Replication Policies for Dynamic Content Applications. In EuroSys: ACM SIGOPS/EuroSys European Conference on Computer Systems, 2006.
33. J. D. Strunk, E. Thereska, C. Faloutsos, and G. R. Ganger. Using Utility to Provision Storage Systems. In FAST: Conference on File and Storage Technologies, 2008.
34. G. Tesauro, N. Jong, R. Das, and M. Bennani. A Hybrid Reinforcement Learning Aproach to Autonomic Resource Allocation. In International Conference on Autonomic Computing (ICAC), 2006.
35. E. Thereska, A. Donnelly, and D. Narayanan. Sierra: A Power-Proportional, Distributed storage System. Technical Report MSR-TR-2009-153, Microsoft, 2009.
36. B. Urgaonkar, P. Shenoy, A. Chandra, and P. Goyal. Dynamic Provisioning of Multi-tier Internet Applications. In International Conference on Autonomic Computing (ICAC), 2005.
37. V. V. Vazirani. Approximation Algorithms. Springer, 2003.
38. L. Yin, S. Uttamchandani, M. Korupolu, K. Voruganti, and R. Katz. SMART: An Integrated Multi-Action Advisor for Storage Systems. In USENIX Annual Technical Conference, 2006.