A MapReduce-supported network structure for data centers

Concurrency and Computation: Practice and Experience

Volumn 24, Issue 12, 2012, Pages 1271-1295

  Ding, Zeliu ^a   Guo, Deke ^a   Liu, Xue ^b   Luo, Xueshan ^a   Chen, Guihai ^c  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

^b MCGILL UNIVERSITY   (Canada)

^c SHANGHAI JIAO TONG UNIVERSITY   (China)

Author keywords

data center network (DCN); MapReduce; structure

Indexed keywords

DATA HANDLING; FORESTRY; STRUCTURE (COMPOSITION); TOPOLOGY;

COMPREHENSIVE EVALUATION; DATA CENTER NETWORKS; DISTRIBUTED DATA PROCESSING; GENERAL APPLICATIONS; MAP-REDUCE; MATHEMATICAL ANALYSIS; NETWORK STRUCTURES; TOPOLOGICAL PROPERTIES;

TREES (MATHEMATICS);

DATA PROCESSING; FORESTRY; MATHEMATICS; STRUCTURES; TREES;

EID: 84864663832     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.1791     Document Type: Article

References (23)

1. Greenberg A, Hamilton JR, Jain N, Kandula S, Kim C, Lahiri P, Maltz DA, Patel P, Sengupta S,. VL2: a scalable and flexible data center network. ACM SIGCOMM Computer Communication Review 2009; 39 (4): 51-62.
2. Greenberg A, Hamilton J, Maltz DA, Patel P,. The cost of a cloud research problems in data center networks. ACM SIGCOMM Computer Communication Review 2009; 39 (1): 68-73.
3. Guo C, Wu H, Tan K, Shiy L, Zhang Y, Lu S,. DCell: a scalable and fault-tolerant network structure for data centers. Proceedings of the ACM SIGCOMM, Aug. 2008;; 75-86.
4. Ghemawat S, Gobioff H, Leung ST,. The Google file system. Proceedings of the ACM 19th Symposium on Operating Systems Principles, Dec. 2003; 29-43. (Pubitemid 40929686)
5. Borthakur D,. The Hadoop distributed file system: architecture and design. http://hadoop.apache.org/core/docs/current/hdfsdesign.pdf [accessed on March 2010].
6. Chang F, Dean J, Ghemawat S, Hsieh WC, Wallach DA, Burrows M, Chandra T, Fikes A, Gruber RE,. Bigtable: a distributed storage system for structured data. Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI), Nov. 2006; 205-218.
7. Isard M, Budiu M, Yu Y, Birrell A, Fetterly D,. Dryad: distributed data-parallel programs from sequential building blocks. Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems, Jun. 2007; 59-72. (Pubitemid 47281575)
8. Dean J, Ghemawat S,. MapReduce: simplified data processing on large clusters. Proceedings of the 6th Symposium on Operating System Design and Implementation (OSDI), Dec. 2004; 137-150.
9. Al-Fares M, Loukissas A, Vahdat A,. A scalable, commodity data center network architecture. Proceedings of the ACM SIGCOMM, Aug. 2008; 63-74.
10. Li D, Guo C, Wu H, Tan K, Zhang Y, Lu S,. FiConn: using backup port for server interconnection in data centers. Proceedings of the IEEE INFOCOM, Apr. 2009; 2276-2285.
11. Guo C, Lu G, Li D, Wu H, Zhang X, Shi Y, Tian C, Zhang Y, Lu S,. BCube: a high performance, server-centric network architecture for modular data centers. Proceedings of the ACM SIGCOMM, Aug. 2009; 63-74.
12. Sandholm T, Lai K,. MapReduce optimization using regulated dynamic prioritization. Proceedings of the ACM 11th Joint International Conference on Measurement and Modeling of Computer Systems, Jun. 2009; 299-310.
13. Kruijf M, Sankaralingam K,. MapReduce for the cell B.E. architecture. IBM Journal of Research and Development 2009; 53 (5): 747-758.
14. Ranger C, Raghuraman R, Penmetsa A, Bradski G, Kozyrakis C,. Evaluating MapReduce for multi-core and multiprocessor systems. Proceedings of the 13th IEEE International Symposium on High-Performance Computer Architecture (HPCA), Feb. 2007; 13-24. (Pubitemid 47208148)
15. Hsu LH, Lin CK,. Graph Theory and Interconnection Networks. CRC Press: Boca Raton, 2009.
16. Cohen J,. Graph twiddling in a MapReduce world. IEEE Computing in Science and Engineering 2009; 2 (4): 29-41.
17. Abts D, Marty MR, Wells PM, Klausler P, Liu H,. Energy proportional datacenter networks. Proceedings of the ACM 37th Annual International Symposium on Computer Architecture (ISCA), Jul. 2010; 338-347.
18. Guo D, Chen T, Li D, Liu Y, Liu X, Chen G,. BCN: expansible network structures for data centers using hierarchical compound graphs. Proceedings of the IEEE INFOCOM, Apr. 2011. Mini conference.
19. Bastoul C, Feautrier P,. Improving data locality by chunking. Springer Lecture Notes in Computer Science 2003; 2622: 320-334.
20. Kumar A, Manjunath D, Kuri J,. Communication Networking an Analytical Approach. Elsevier Inc.: Amsterdam, 2004.
21. Dean J, Ghemawat S,. MapReduce: a flexible data processing tool. Communications of the ACM 2010; 53 (1): 72-77.
22. Weisstein E,. Floyd-Warshall Algorithm. Wolfram MathWorld, 2009. http://mathworld.wolfram.com/Floyd-WarshallAlgorithm.html [accessed on November 2009].
23. Rao L, Liu X, Xie L, Liu W,. Minimizing electricity cost: optimization of distributed Internet data centers in a multi-electricity-market environment. Proceedings of the IEEE INFOCOM, Mar. 2010.