Transparent and flexible network management for big data processing in the cloud

5th USENIX Workshop on Hot Topics in Cloud Computing, HotCloud 2013

Volumn , Issue , 2013, Pages

  Das, Anupam ^a   Lumezanu, Cristian ^b   Zhang, Yueping ^b   Singh, Vishal ^b   Jiang, Guofei ^b   Yu, Curtis ^c  

^a UIUC   (United States)

^b NEC LABORATORIES AMERICA   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION PROGRAMS; BIG DATA; CLOUD COMPUTING; DATA HANDLING; NETWORK MANAGEMENT; SOFTWARE AGENTS;

APPLICATION NETWORKS; APPLICATION SERVERS; DATA MOVEMENTS; DATA PROCESSING APPLICATIONS; DECISION ENGINES; HIGH UTILIZATIONS; NETWORK MANAGEMENT FRAMEWORKS; PROCESSING TIME;

INFORMATION MANAGEMENT;

EID: 85084162147     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. Big Data and the Enterprise: Network Design and Considerations. Cisco white paper, 2011.
2. ABU-LIBDEH, H., COSTA, P., ROWSTRON, A., O’SHEA, G., AND DONNELLY, A. Symbiotic Routing in Future Data Centers. In ACM Sigcomm (2010).
3. AL-FARES, M., RADHAKRISHNAN, S., RAGHAVAN, B., HUANG, N., AND VAHDAT, A. Hedera: Dynamic Flow Scheduling for Data Center Networks. In USENIX NSDI (2010).
4. ANANTHANARAYANAN, G., KANDULA, S., GREENBERG, A., STOICA, I., LU, Y., SAHA, B., AND HARRIS, E. Reining in the outliers in Map-Reduce clusters using Mantri. In USENIX OSDI (2010).
5. BAZZAZ, H. H., TEWARI, M., WANG, G., PORTER, G., NG, T. E., ANDERSEN, D. G., KAMINSKY, M., KOZUCH, M. A., AND VAHDAT, A. Switching the optical divide: Fundamental challenges for hybrid electrical/optical datacenter networks. In SOCC (2011).
6. CHEN, K., SINGLA, A., SINGH, A., RAMACHANDRAN, K., XU, L., ZHANG, Y., WEN, X., AND CHEN, Y. OSA: An optical switching architecture for data center networks with unprecedented flexibility. In USENIX NSDI (2012).
7. CHEN, Y., ALSPAUGH, S., AND KATZ, R. Interactive analytical processing in big data systems: A cross-industry study of MapReduce workloads. In VLDB (2012).
8. CHOWDHURY, M., ZAHARIA, M., MA, J., JORDAN, M. I., AND STOICA, I. Managing data transfers in computer clusters with orchestra. In ACM Sigcomm (2011).
9. COSTA, P., DONNELLY, A., ROWSTRON, A., AND O’SHEA, G. Camdoop: Exploiting In-network Aggregation for Big Data Applications. In USENIX NSDI (2012).
10. FARRINGTON, N., PORTER, G., RADHAKRISHNAN, S., BAZZAZ, H. H., SUBRAMANYA, V., FAINMAN, Y., PAPEN, G., AND VAHDAT, A. Helios: A Hybrid Electrical/Optical Switch Architecture for Modular Data Centers. In ACM Sigcomm (2010).
11. FONSECA, R., PORTER, G., KATZ, R. H., SHENKER, S., AND STOICA, I. X-trace: A pervasive network tracing framework. In USENIX NSDI (2007).
12. GHODSI, A., ZAHARIA, M., HINDMAN, B., KONWINSKI, A., SHENKER, S., AND STOICA, I. Dominant resource fairness: Fair allocation of multiple resource types. In USENIX NSDI (2011).
13. GREENBERG, A., JAIN, N., KANDULA, S., KIM, C., LAHIRI, P., MALTZ, D., PATEL, P., AND SENGUPTA, S. VL2: A Scalable and Flexible Data Center Network. In ACM Sigcomm (2009).
14. GUDE, N., KOPONEN, T., PETTIT, J., PFAFF, B., CASADO, M., MCKEOWN, N., AND SHENKER, S. NOX: Towards an Operating System for Networks. SIGCOMM Comput. Commun. Rev. 38, 3 (2008), 105–110.
15. GUO, C., LU, G., LI, D., WU, H., ZHANG, X., SHI, Y., TIAN, C., ZHANG, Y., AND LU, S. BCube: A High Performance, Server-centric Network Architecture for Modular Data Centers. In ACM Sigcomm (2009).
16. Hadoop. http://hadoop.apache.org/.
17. ISARD, M., PRABHAKARAN, V., CURREY, J., WIEDER, U., TALWAR, K., AND GOLDBERG, A. Quincy: Fair scheduling for distributed computing clusters. In SOSP (2009).
18. PHANISHAYEE, A., KREVAT, E., VASUDEVAN, V., ANDERSEN, D. G., GANGER, G. R., GIBSON, G. A., AND SESHAN, S. Measurement and analysis of TCP throughput collapse in cluster-based storage systems. In FAST (2008).
19. SHIEH, A., KANDULA, S., GREENBERG, A., KIM, C., AND SAHA, B. Sharing the data center network. In USENIX NSDI (2011).
20. TOOTOONCHIAN, A., GHOBADI, M., AND GANJALI, Y. OpenTM: Traffic Matrix Estimator for OpenFlow Networks. In PAM (2010).
21. WANG, G., ANDERSEN, D. G., KAMINSKY, M., PAPAGIANNAKI, K., NG, T. E., KOZUCH, M., AND RYAN, M. c-Through: Part-time Optics in Data Centers. In ACM Sigcomm (2010).
22. WANG, G., NG, T. E., AND SHAIKH, A. Programming Your Network at Run-time for Big Data Applications. In HotSDN (2012).
23. YU, C., LUMEZANU, C., SINGH, V., ZHANG, Y., JIANG, G., AND MADHYASTHA, H. V. Monitoring network utilization with zero measurement cost. In PAM (2013).
24. ZAHARIA, M., BORTHAKUR, D., SARMA, J. S., ELMELEEGY, K., SHENKER, S., AND STOICA, I. Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling. In Eurosys (2010).