Data Transfer Scheduling for Maximizing Throughput of Big-Data Computing in Cloud Systems

IEEE Transactions on Cloud Computing

Volumn 6, Issue 1, 2018, Pages 87-98

  Xie, Ruitao ^a   Jia, Xiaohua ^a  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Big data computing; Data center; Data transfer scheduling; Throughput maximization

Indexed keywords

APPROXIMATION ALGORITHMS; DATA COMMUNICATION SYSTEMS; DATA TRANSFER; DISTRIBUTED COMPUTER SYSTEMS; INTEGER PROGRAMMING; SCHEDULING; THROUGHPUT;

APPROXIMATION RATIOS; COMPUTING APPLICATIONS; DATA CENTERS; EXTENSIVE SIMULATIONS; MIXED INTEGER PROGRAMMING; NEAR-OPTIMAL SOLUTIONS; PARALLEL PROCESSING; THROUGHPUT MAXIMIZATION;

BIG DATA;

EID: 85043505925     PISSN: None     EISSN: 21687161     Source Type: Journal    
DOI: 10.1109/TCC.2015.2464808     Document Type: Article

References (21)

1. J. Dean and S. Ghemawat, "Mapreduce: Simplified data processing on large clusters," Commun. ACM, vol. 51, no. 1, pp. 107-113, Jan. 2008.
2. D. Borthakur. (2008). "HDFS architecture guide," [Online]. Available: http://hadoop.apache.org/docs/r1.2.1/hdfs design.pdf
3. C. Guo, G. Lu, D. Li, H. Wu, X. Zhang, Y. Shi, C. Tian, Y. Zhang, and S. Lu, "BCube: A high performance, Server-centric network architecture for modular data centers," in Proc. SIGCOMM, 2009, pp. 63-74.
4. A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta, "VL2: A scalable and flexible data center network," in Proc. SIGCOMM, 2009, pp. 51-62.
5. C. Hopps. (2000). "Analysis of an equal-cost multi-path algorithm," RFC 2992, IETF, [Online]. Available: https://tools.ietf. org/html/rfc2992
6. M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat, "Hedera: Dynamic flow scheduling for data center networks," in Proc. 7th USENIX Conf. Netw. Syst. Des. Implementation, 2010, pp. 19-19.
7. M. Chowdhury, M. Zaharia, J. Ma, M. I. Jordan, and I. Stoica, "Managing data transfers in computer clusters with orchestra," in Proc. SIGCOMM, 2011, pp. 98-109.
8. J. L. Gross, J. Yellen, and P. Zhang, Handbook of Graph Theory, 2nd ed. London, U.K.: Chapman & Hall, 2013.
9. R. J. Vanderbei, Linear Programming: Foundations and Extensions. New York, NY, USA: Springer, 1996.
10. I. G. Optimization. (2015). Gurobi optimizer reference manual [Online]. Available: http://www.gurobi.com
11. A. Dixit, P. Prakash, and R. R. Kompella, "On the efficacy of finegrained traffic splitting protocolsin data center networks," in Proc. SIGCOMM, 2011, pp. 430-431.
12. F. P. Tso and D. Pezaros, "Improving data center network utilization using near-optimal traffic engineering," IEEE Trans. Parallel Distrib. Syst., vol. 24, no. 6, pp. 1139-1148, Jun. 2013.
13. T. Benson, A. Anand, A. Akella, and M. Zhang, "MicroTE: Fine grained traffic engineering for data centers," in Proc. 7th Conf. Emerging Netw. Exp. Technol., 2011, pp. 8:1-8:12.
14. W. Cui and C. Qian, "DiFS: Distributed flow scheduling for adaptive routing in hierarchical data center networks," in Proc. 10th ACM/IEEE Symp. Archit. Netw. Commun. Syst., 2014, pp. 53-64.
15. M. Alizadeh, S. Yang, M. Sharif, S. Katti, N. McKeown, B. Prabhakar, and S. Shenker, "pFabric: Minimal near-optimal datacenter transport," in Proc. SIGCOMM, 2013, pp. 435-446.
16. F. R. Dogar, T. Karagiannis, H. Ballani, and A. Rowstron, "Decentralized task-aware scheduling for data center networks," in Proc. SIGCOMM, 2014, pp. 431-442.
17. H. H. E. AL-Mistarihi and C. H. Yong, "On fairness, optimizing replica selection in data grids," IEEE Trans. Parallel Distrib. Syst., vol. 20, no. 8, pp. 1102-1111, Aug. 2009.
18. R. M. Rahman, R. Alhajj, and K. Barker, "Replica selection strategies in data grid," J. Parallel Distrib. Comput., vol. 68, no. 12, pp. 1561-1574, 2008.
19. V. Valancius, B. Ravi, N. Feamster, and A. C. Snoeren, "Quantifying the benefits of joint content and network routing," in Proc. ACM SIGMETRICS/Int. Conf. Meas. Modeling Comput. Syst., 2013, pp. 243-254.
20. S. Narayana, W. Jiang, J. Rexford, and M. Chiang, "Joint server selection and routing for Geo-replicated services," in Proc. IEEE/ ACM 6th Int. Conf. Utility Cloud Comput., 2013, pp. 423-428.
21. H. Xu and B. Li, "Joint request mapping and response routing for Geo-distributed cloud services," in Proc. INFOCOM, 2013, pp. 854-862.