End-to-end performance isolation through virtual datacenters

Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2014

Volumn , Issue , 2014, Pages 233-248

  Angel, Sebastian ^b   Ballani, Hitesh ^a   Karagiannis, Thomas ^a   O'Shea, Greg ^a   Thereska, Eno ^a  

^a MICROSOFT RESEARCH   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DIGITAL STORAGE; SYSTEMS ANALYSIS;

APPLICATION PERFORMANCE; CENTRALIZED CONTROLLERS; CHANGING WORKLOAD; END-TO-END PERFORMANCE; END-TO-END THROUGHPUT; RESOURCES BASED; STORAGE SERVERS; VIRTUAL DATACENTER;

PULSARS;

EID: 85076933822     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (74)

1. Amazon AWS Products. http://aws.amazon.com/products/.
2. Amazon DynamoDB capacity unit. http://aws.amazon.com/dynamodb/faqs/#What_is_a_readwrite_capacity_unit.
3. Bluelock: Virtual data centers. http://www.bluelock.com/ virtual-datacenters.
4. Hyper-V resource allocation. http://technet.microsoft.com/en-us/library/cc742470.aspx.
5. Iometer. http://iometer.org.
6. VMware vCloud: Organization virtual data center. http://kb.vmware.com/kb/1026320.
7. vSphere resource management guide. https://www.vmware.com/support/pubs/ vsphere- esxi- vcenter- server- pubs.html.
8. WANProxy. http://wanproxy.org.
9. M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In Proceedings of the ACM SIGCOMM Conference, Aug. 2008.
10. M. Alizadeh, A. G. Greenberg, D. A. Maltz, J. Padhye, P. Patel, B. Prabhakar, S. Sengupta, and M. Sridharan. Data center TCP (DCTCP). In Proceedings of the ACM SIGCOMM Conference, Aug. 2010.
11. H. Ballani, P. Costa, T. Karagiannis, and A. Rowstron. The price is right: Towards location-independent costs in datacenters. In Proceedings of the ACM Workshop on Hot Topics in Networks (HotNets), Nov. 2011.
12. H. Ballani, P. Costa, T. Karagiannis, and A. Rowstron. Towards predictable datacenter networks. In Proceedings of the ACM SIGCOMM Conference, Aug. 2011.
13. H. Ballani, D. Gunawardena, and T. Karagiannis. Network sharing in multi-tenant datacenters. Technical Report MSR-TR-2012-39, MSR, 2012.
14. H. Ballani, K. Jang, T. Karagiannis, C. Kim, D. Gunawaradena, and G. O'Shea. Chatty tenants and the cloud network sharing problem. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), Apr. 2013.
15. T. Benson, A. Anand, A. Akella, and M. Zhang. Understanding data center traffic characteristics. In Proceedings of the ACM SIGCOMM Workshop on Research on Enterprise Networking (WREN), Aug. 2009.
16. A. Bhattacharya, D. Culler, E. Friedman, A. Ghodsi, S. Shenker, and I. Stoica. Hierarchical scheduling for diverse datacenter workloads. In Proceedings of the ACM Symposium on Cloud Computing (SOCC), Oct. 2013.
17. J.-P. Billaud and A. Gulati. hClock: Hierarchical QoS for packet scheduling in a hypervisor. In Proceedings of the ACM European Conference on Computer Systems (EuroSys), Apr. 2013.
18. O. Biran, A. Corradi, M. Fanelli, L. Foschini, A. Nus, D. Raz, and E. Silvera. A stable network-aware VM placement for cloud systems. In Proceedings of the IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May 2012.
19. D. Dolev, D. G. Feitelson, J. Y. Halpern, R. Kupferman, and N. Linial. No justified complaints: On fair sharing of multiple resources. In Proceedings of the Innovations in Theoretical Computer Science (ITCS) Conference, Aug. 2012.
20. H. Dreger, A. Feldman, V. Paxson, and R. Sommer. Predicting the resource consumption of network intrusion detection systems. In Proceedings of the International Symposium on Research in Attacks, Intrusions and Defenses (RAID), Sept. 2008.
21. M. Feldman, K. Lai, and L. Zhang. A price-anticipating resource allocation mechanism for distributed shared clusters. In Proceedings of the ACM Conference on Electronic Commerce (EC), June 2005.
22. S. Floyd. HighSpeed TCP for large congestion windows, Dec. 2003. RFC 3649. http://www.ietf.org/rfc/rfc3649.txt.
23. A. Ghodsi, V. Sekar, M. Zaharia, and I. Stoica. Multi-resource fair queueing for packet processing. In Proceedings of the ACM SIGCOMM Conference, Aug. 2012.
24. A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker, and I. Stoica. Dominant resource fairness: Fair allocation of multiple resource types. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), Mar. 2011.
25. D. Ghoshal, R. S. Canon, and L. Ramakrishnan. I/O performance of virtualized cloud environments. In Proceedings of the International Workshop on Data Intensive Computing in the Clouds (DataCloud), May 2011.
26. A. Gulati, I. Ahmad, and C. A. Waldspurger. PARDA: proportional allocation of resources for distributed storage access. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST), Feb. 2009.
27. A. Gulati, A. Merchant, and P. J. Varman. mClock: Handling throughput variability for hypervisor IO scheduling. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), Oct. 2010.
28. C. Guo, G. Lu, H. J. Wang, S. Yang, C. Kong, P. Sun, W. Wu, and Y. Zhang. SecondNet: A data center network virtualization architecture with bandwidth guarantees. In Proceedings of the International Conference on emerging Networking EXperiments and Technologies (CoNEXT), Nov. 2010.
29. D. Gupta, L. Cherkasova, R. Gardner, and A. Vahdat. Enforcing performance isolation across virtual machines in XEN. In Proceedings of the ACM/IFIP/USENIX International Middleware Conference, Dec. 2006.
30. A. Gutman and N. Nisan. Fair allocation without trade. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS), June 2012.
31. Z. Hill, J. Li, M. Mao, A. Ruiz-Alvarez, and M. Humphrey. Early observations on the performance of Windows Azure. In Proceedings of the International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC), June 2010.
32. IEEE Computer Society. Virtual bridged local area networks. Amendment 13: Congestion notification, Apr. 2010. IEEE Std. 802.1Qau-2010.
33. A. Iosup, N. Yigitbasi, and D. Epema. On the performance variability of cloud services. Technical Report PDS-2010-002, Delft University, 2010.
34. D. Irwin, J. Chase, L. Grit, and A. Yumerefendi. Self-recharging virtual currency. In Proceedings of the ACM SIGCOMM Workshop on the Economics of Peer-to-Peer Systems (P2PECON), Aug. 2005.
35. V. Jalaparti, H. Ballani, P. Costa, T. Karagiannis, and A. Rowstron. Bridging the tenant-provider gap in cloud services. In Proceedings of the ACM Symposium on Cloud Computing (SOCC), 2012.
36. V. Jeyakumar, M. Alizadeh, D. Mazières, B. Prabhakar, C. Kim, and A. Greenberg. EyeQ: Practical network performance isolation at the edge. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), Apr. 2013.
37. C. Jin, D. X. Wei, and S. H. Low. FAST TCP: Motivation, architecture, algorithms, performance. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Mar. 2004.
38. C. Joe-Wong, S. Sen, T. Lan, and M. Chiang. Multi-resource allocation: Fairness-efficiency tradeoffs in a unifying framework. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Apr. 2012.
39. S. Kandula, S. Sengupta, A. Greenberg, P. Patel, and R. Chaiken. The nature of data center traffic: Measurements & analysis. In Proceedings of the ACM SIGCOMM Conference on Internet Measurement (IMC), Nov. 2009.
40. I. Kash, A. D. Procaccia, and N. Shah. No agent left behind: Dynamic fair division of multiple resources. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS), May 2013.
41. L. T. Kou and G. Markowsky. Multidimensional bin packing algorithms. IBM Journal of Research and Development, 21(5), Sept. 1977.
42. K. LaCurts, J. C. Mogul, H. Balakrishnan, and Y. Turner. Cicada: Introducing predictive guarantees for cloud networks. In Proceedings of the USENIX Workshop on Hot Topics in Cloud Computing (HotCloud), June 2014.
43. J. Lee, Y. Turner, M. Lee, L. Popa, S. Banarjee, J.-M. Kang, and P. Sharma. Application-driven bandwidth guarantees in datacenters. In Proceedings of the ACM SIGCOMM Conference, Aug. 2014.
44. S. Lee, R. Panigraphy, V. Prabhakaran, V. Ramasubramanian, K. Talwar, L. Uyeda, and U. Wieder. Validating heuristics for virtual machines consolidation. Technical Report MSR-TR-2011-9, Microsoft Research, 2011.
45. W. Leinberger, G. Karypis, and V. Kumar. Multi-capacity bin packing algorithms with applications to job scheduling under multiple constraints. In Proceedings of the International Conference on Parallel Processing (ICPP), Sept. 1999.
46. A. Li, X. Yang, S. Kandula, and M. Zhang. CloudCmp: Comparing public cloud providers. In Proceedings of the ACM SIGCOMM Conference on Internet Measurement (IMC), Nov. 2010.
47. K. Maruyama, S. K. Chang, and D. T. Tang. A general packing algorithm for multidimensional resource requirements. International Journal of Computer and Information Sciences, 6(2), 1977.
48. D. C. Parkes, A. D. Procaccia, and N. Shah. Beyond dominant resource fairness: Extensions, limitations, and indivisibilities. In Proceedings of the ACM Conference on Electronic Commerce (EC), June 2012.
49. P. Patel, D. Bansal, L. Yuan, A. Murthy, A. Greenberg, D. A. Maltz, R. Kern, H. Kumar, M. Zikos, H. Wu, C. Kim, and N. Karri. Ananta: Cloud scale load balancing. In Proceedings of the ACM SIGCOMM Conference, Aug. 2013.
50. J. T. Piao and J. Yan. A network-aware virtual machine placement and migration approach in cloud computing. In Proceedings of the International Conference on Grid and Cloud Computing (GCC), Nov. 2010.
51. L. Popa, G. Kumar, M. Chowdhury, A. Krishnamurthy, S. Ratnasamy, and I. Stoica. FairCloud: Sharing the network in cloud computing. In Proceedings of the ACM SIGCOMM Conference, Aug. 2012.
52. L. Popa, P. Yalagandula, S. Banarjee, J. C. Mogul, Y. Turner, and J. R. Santos. ElasticSwitch: Practical work-conserving bandwidth guarantees for cloud computing. In Proceedings of the ACM SIGCOMM Conference, Aug. 2013.
53. K. Ramakrishnan, S. Floyd, and D. Black. The addition of explicit congestion notification (ECN) to IP, Sept. 2001. RFC 3168. http://www.ietf.org/rfc/rfc3168.txt.
54. N. Roy, J. S. Kinnebrew, N. Shankaran, G. Biswas, and D. C. Schmidt. Toward effective multi-capacity resource allocation in distributed real-time embedded systems. In Proceedings of the IEEE Symposium on Object/Component/Service-oriented Real-time Distributed Computing (ISORC), May 2008.
55. J. Schad, J. Dittrich, and J.-A. Quiané-Ruiz. Runtime measurements in the cloud: observing, analyzing, and reducing variance. In Proceedings of the International Conference on Very Large Data Bases (VLDB), Sept. 2010.
56. V. Sekar, S. Ratnasamy, M. K. Reiter, N. Egi, and G. Shi. The middlebox manifesto: Enabling innovation in middlebox deployment. In Proceedings of the ACM Workshop on Hot Topics in Networks (HotNets), Nov. 2011.
57. D. Shah and D. Wischik. Principles of resource allocation in networks. In Proceedings of the ACM SIGCOMM Education Workshop, May 2011.
58. S. Shepler, B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M. Eisler, and D. Noveck. Network file system (NFS) version 4 protocol, Apr. 2003. RFC3530. http://www.ietf.org/rfc/rfc3530.txt.
59. J. Sherry, S. Hasan, C. Scott, A. Krishnamurthy, S. Ratnasamy, and V. Sekar. Making middleboxes someone else's problem: Network processing as a cloud service. In Proceedings of the ACM SIGCOMM Conference, Aug. 2012.
60. A. Shieh, S. Kandula, A. Greenberg, C. Kim, and B. Saha. Sharing the data center network. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), Mar. 2011.
61. D. Shue and M. J. Freedman. From application requests to Virtual IOPs: Provisioned key-value storage with Libra. In Proceedings of the ACM European Conference on Computer Systems (EuroSys), Apr. 2014.
62. D. Shue, M. J. Freedman, and A. Shaikh. Performance isolation and fairness for multi-tenant cloud storage. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), Oct. 2012.
63. I. Stoica, H. Abdel-Wahab, and A. Pothen. A microeconomic scheduler for parallel computers. In Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, 1994.
64. I. E. Sutherland. A futures market in computer time. Communications of the ACM, 11(6), June 1968.
65. A. S. Tanenbaum and D. J. Wetherall. Computer Networks. Prentice Hall, 5th edition, Oct. 2010.
66. E. Thereska, H. Ballani, G. O'Shea, T. Karagiannis, A. Rowstron, T. Talpey, R. Black, and T. Zhu. IOFlow: A software-defined storage architecture. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), Nov. 2013.
67. E. Thereska, A. Donnelly, and D. Narayanan. Sierra: Practical power-proportionality for data center storage. In Proceedings of the ACM European Conference on Computer Systems (EuroSys), Apr. 2011.
68. C. A. Waldspurger. Memory resource management in VMware ESX server. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), Dec. 2002.
69. C. A. Waldspurger and W. E. Weihl. Lottery scheduling: Flexible proportional-share resource management. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), Nov. 1994.
70. E. Walker. Benchmarking Amazon EC2 for high-performance scientific computing. USENIX;login:, 33(5), 2008.
71. G. Wang and T. S. E. Ng. The impact of virtualization on network performance of Amazon EC2 data center. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Apr. 2010.
72. H. Wang and P. J. Varman. Balancing fairness and efficiency in tiered storage systems with bottleneck-aware allocation. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST), Feb. 2014.
73. D. Xie, N. Ding, Y. C. Hu, and R. Kompella. The only constant is change: Incorporating time-varying network reservations in data centers. In Proceedings of the ACM SIGCOMM Conference, Aug. 2012.
74. L. Xu, K. Harfoush, and I. Rhee. Binary increase congestion control (BIC) for fast long-distance networks. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Mar. 2004.