A Simple Congestion-Aware Algorithm for Load Balancing in Datacenter Networks

IEEE/ACM Transactions on Networking

Volumn 25, Issue 6, 2017, Pages 3670-3682

  Shafiee, Mehrnoosh ^a   Ghaderi, Javad ^a  

^a Columbia University ^*  (United States)

Author keywords

datacenter network.; load balancing; Markov chains; online algorithms; routing algorithms

Indexed keywords

BANDWIDTH; COMPUTATIONAL COMPLEXITY; COSTS; FUNCTIONS; MARKOV PROCESSES; RESOURCE ALLOCATION; ROUTING ALGORITHMS; TOPOLOGY; TRAFFIC CONGESTION;

BANDWIDTH REQUIREMENT; DATA CENTER NETWORKS; END-TO-END DATA FLOW; EQUAL-COST-MULTI-PATHS; EXTENSIVE SIMULATIONS; ON-LINE ALGORITHMS; SOURCE-DESTINATION PAIRS; TRAFFIC CONDITIONS;

COMPLEX NETWORKS;

EID: 85030753496     PISSN: 10636692     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNET.2017.2751251     Document Type: Conference Paper

References (49)

1. M. Shafiee and J. Ghaderi, "A simple congestion-aware algorithm for load balancing in datacenter networks, " Proc. 35th Annu. IEEE Int. Conf. Comput. Commun. (INFOCOM), Apr. 2016, pp. 1-9.
2. M. Al-Fares, A. Loukissas, A. Vahdat, "A scalable, commodity data center network architecture, " ACM SIGCOMM Comput. Commun. Rev., vol. 38, no. 4, pp. 63-74, 2008.
3. T. Benson, A. Anand, A. Akella, M. Zhang, "MicroTE: Fine grained traffic engineering for data centers, " Proc. 7th Conf. Emerg. Netw. Experim. Technol., 2011, Art. no. 8.
4. M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, A. Vahdat, "Hedera: Dynamic flow scheduling for data center networks, " Proc. NSDI, vol. 10. 2010, p. 19.
5. C. Raiciu, S. Barre, C. Pluntke, A. Greenhalgh, D. Wischik, M. Handley, "Improving datacenter performance and robustness with multipath TCP, " ACM SIGCOMM Comput. Commun. Rev., vol. 41, no. 4, pp. 266-277, Aug. 2011.
6. S. Kandula, D. Katabi, S. Sinha, A. Berger, "Dynamic load balancing without packet reordering, " ACM SIGCOMM Comput. Commun. Rev., vol. 37, no. 2, pp. 51-62, 2007.
7. A. Greenberg, et al., "VL2: A scalable and flexible data center network, " ACM SIGCOMM Comput. Commun. Rev., vol. 39, no. 4, pp. 51-62, 2009.
8. C. Guo, et al., "BCube: A high performance, server-centric network architecture for modular data centers, " ACM SIGCOMM Comput. Commun. Rev., vol. 39, no. 4, pp. 63-74, 2009.
9. M. Bradonjíc, I. Saniee, I. Widjaja, "Scaling of capacity and reliability in data center networks, " ACM SIGMETRICS Perform. Eval. Rev., vol. 42, no. 2, pp. 46-48, 2014.
10. A. Singla, C.-Y. Hong, L. Popa, P. B. Godfrey, "Jellyfish: Networking data centers, randomly, " Proc. NSDI, vol. 12. 2012, p. 17.
11. S. Kandula, S. Sengupta, A. Greenberg, P. Patel, R. Chaiken, "The nature of data center traffic: Measurements & analysis, " Proc. 9th ACM SIGCOMM Conf. Internet Meas. Conf., 2009, pp. 202-208.
12. A. Dixit, P. Prakash, Y. C. Hu, R. R. Kompella, "On the impact of packet spraying in data center networks, " Proc. IEEE INFOCOM, Apr. 2013, pp. 2130-2138.
13. S. Even, A. Itai, A. Shamir, "On the complexity of time table and multi-commodity flow problems, " Proc. 16th Annu. Symp. Found. Comput. Sci., Oct. 1975, pp. 184-193.
14. G. M. Guisewite and P. M. Pardalos, "Minimum concave-cost network flow problems: Applications, complexity, algorithms, " Ann. Oper. Res., vol. 25, no. 1, pp. 75-99, 1990.
15. Y. Dinitz, N. Garg, M. X. Goemans, "On the single-source unsplittable flow problem, " Proc. 39th Annu. Symp. Found. Comput. Sci., 1998, pp. 290-299.
16. R. N. Mysore, et al., "PortLand: A scalable fault-tolerant layer 2 data center network fabric, " ACM SIGCOMM Comput. Commun. Rev., vol. 39, no. 4, pp. 39-50, 2009.
17. J. Cao, et al., "Per-packet load-balanced, low-latency routing for closbased data center networks, " Proc. 9th ACM Conf. Emerg. Netw. Experim. Technol., 2013, pp. 49-60.
18. P. Gill, N. Jain, N. Nagappan, "Understanding network failures in data centers: Measurement, analysis, implications, " ACM SIGCOMM Comput. Commun. Rev., vol. 41, no. 4, pp. 350-361, Aug. 2011.
19. M. Chiesa, G. Kindler, M. Schapira, "Traffic engineering with Equal-Cost-MultiPath: An algorithmic perspective, " IEEE/ACM Trans. Netw., vol. 25, no. 2, pp. 779-792, Apr. 2017.
20. S. Sen, D. Shue, S. Ihm, M. J. Freedman, "Scalable, optimal flow routing in datacenters via local link balancing, " Proc. 9th ACM Conf. Emerg. Netw. Experim. Technol., 2013, pp. 151-162.
21. J. W. Jiang, T. Lan, S. Ha, M. Chen, M. Chiang, "Joint VM placement and routing for data center traffic engineering, " Proc. IEEE INFOCOM, Mar. 2012, pp. 2876-2880.
22. K. He, et al., "Presto: Edge-based load balancing for fast datacenter networks, " Proc. ACM Conf. Special Interest Group Data Commun., 2015, pp. 465-478.
23. R. Gallager, "A minimum delay routing algorithm using distributed computation, " IEEE Trans. Commun., vol. COM-25, no. 1, pp. 73-85, Jan. 1977.
24. N. Michael and A. Tang, "HALO: Hop-by-hop adaptive link-state optimal routing, " IEEE/ACM Trans. Netw., vol. 23, no. 6, pp. 1862-1875, Dec. 2015.
25. D. Xu, M. Chiang, J. Rexford, "Link-state routing with hop-by-hop forwarding can achieve optimal traffic engineering, " IEEE/ACM Trans. Netw., vol. 19, no. 6, pp. 1717-1730, Dec. 2011.
26. R. W. Rosenthal, "A class of games possessing pure-strategy Nash equilibria, " Int. J. Game Theory, vol. 2, no. 1, pp. 65-67, 1973.
27. N. Nisan, T. Roughgarden, E. Tardos, V. V. Vazirani, Algorithmic Game Theory, vol. 1. Cambridge, U.K.: Cambridge Univ. Press, 2007.
28. T. Roughgarden, Selfish Routing and the Price of Anarchy, vol. 174. Cambridge, MA, USA: MIT Press, 2005.
29. P. Key, L. Massoulié, D. Towsley, "Path selection and multipath congestion control, " Proc. IEEE 26th IEEE Int. Conf. Comput. Commun. (INFOCOM), May 2007, pp. 143-151.
30. J. G. Wardrop, "Road paper. Some theoretical aspects of road traffic research, " Proc. Inst. Civil Eng., vol. 1, no. 3, pp. 325-362, 1952.
31. D. Applegate and E. Cohen, "Making intra-domain routing robust to changing and uncertain traffic demands: Understanding fundamental tradeoffs, " Proc. Conf. Appl., Technol., Architectures, Protocols Comput. Commun., 2003, pp. 313-324.
32. H. Wang, et al., "COPE: Traffic engineering in dynamic networks, " ACM SIGCOMM Comput. Commun. Rev., vol. 36, no. 4, pp. 99-110, 2006.
33. M. Bienkowski, M. Korzeniowski, H. Räcke, "A practical algorithm for constructing oblivious routing schemes, " Proc. 15th Annu. ACM Symp. Parallel Algorithms Architectures, 2003, pp. 24-33.
34. N. McKeown, et al., "OpenFlow: Enabling innovation in campus networks, " ACM SIGCOMM Comput. Commun. Rev., vol. 38, no. 2, pp. 69-74, Apr. 2008.
35. M. Casado, et al., "Rethinking enterprise network control, " IEEE/ACM Trans. Netw., vol. 17, no. 4, pp. 1270-1283, Aug. 2009.
36. M. Alizadeh, et al., "CONGA: Distributed congestion-aware load balancing for datacenters, " Proc. ACM Conf. SIGCOMM, 2014, pp. 503-514.
37. A. L. Stolyar, "An infinite server system with general packing constraints, " Oper. Res., vol. 61, no. 5, pp. 1200-1217, 2013.
38. M. Chiesa, G. Kindler, M. Schapira, "Traffic engineering with Equal-Cost-MultiPath: An algorithmic perspective, " Proc. IEEE INFOCOM, Apr. 2014, pp. 1590-1598.
39. B. Fortz and M. Thorup, "Internet traffic engineering by optimizing OSPF weights, " Proc. 19th Annu. Joint Conf. IEEE Comput. Commun. Soc. (INFOCOM), vol. 2. Mar. 2000, pp. 519-528.
40. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
41. D. Ersoz, M. S. Yousif, C. R. Das, "Characterizing network traffic in a cluster-based, multi-tier data center, " Proc. 27th Int. Conf. Distrib. Comput. Syst. (ICDCS), 2007, p. 59.
42. M. Grant and S. Boyd. (Mar. 2014). CVX: MATLAB Software for Disciplined Convex Programming, Version 2.1. [Online]. Available: Http://cvxr.com/cvx
43. J. Diáz, M. J. Serna, N. C. Wormald, "Bounds on the bisection width for random d-regular graphs, " Theor. Comput. Sci., vol. 382, no. 2, pp. 120-130, 2007.
44. B. Bollobás, "Random graphs, " in Modern Graph Theory. New York, NY, USA: Springer, 1998, pp. 215-252.
45. M. Mitzenmacher, "The power of two choices in randomized load balancing, " IEEE Trans. Parallel Distrib. Syst., vol. 12, no. 10, pp. 1094-1104, Oct. 2001.
46. A. L. Stolyar and Y. Zhong, "Asymptotic optimality of a greedy randomized algorithm in a large-scale service system with general packing constraints, " Queueing Syst., vol. 79, no. 2, pp. 117-143, 2015.
47. J. Ghaderi, Y. Zhong, R. Srikant, "Asymptotic optimality of BestFit for stochastic bin packing, " ACM SIGMETRICS Perform. Eval. Rev., vol. 42, no. 2, pp. 64-66, 2014.
48. P. Billingsley, Convergence of Probability Measures, 2nd ed. New York, NY, USA: Wiley, 1999.
49. S. N. Ethier and T. G. Kurtz, Markov Processes: Characterization and Convergence, vol. 282. Hoboken, NJ, USA: Wiley, 2009.