HadoopWatch: A first step towards comprehensive traffic forecasting in cloud computing

Proceedings - IEEE INFOCOM

Volumn , Issue , 2014, Pages 19-27

  Peng, Yang ^a   Chen, Kai ^a   Wang, Guohui ^b   Bai, Wei ^a   Ma, Zhiqiang ^a   Gu, Lin ^a  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^b FACEBOOK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FILE ORGANIZATION;

APPLICATION PERFORMANCE; BIG DATA APPLICATIONS; FILE SYSTEMS; HADOOP FRAMEWORKS; TIME ADVANCE; TRAFFIC DEMANDS; TRAFFIC FORECASTING; VIRTUAL MACHINES;

BIG DATA;

EID: 84904437213     PISSN: 0743166X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/INFOCOM.2014.6847920     Document Type: Conference Paper

References (35)

1. HBase, http://hbase.apache.org/.
2. M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In ACM SIGCOMM '08.
3. M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat. Hedera: Dynamic flow scheduling for data center networks. In NSDI '10.
4. H. H. Bazzaz, M. Tewari, G. Wang, G. Porter, T. S. E. Ng, D. G. Andersen, M. Kaminsky, M. A. Kozuch, and A. Vahdat. Switching the optical divide: fundamental challenges for hybrid electrical/optical datacenter networks. In Proceedings of the 2nd ACM Symposium on Cloud Computing, SOCC '11.
5. T. Benson, A. Anand, A. Akella, and M. Zhang. MicroTE: fine grained traffic engineering for data centers. In Proceedings of the Seventh COnference on emerging Networking EXperiments and Technologies, CoNEXT '11.
6. K. Chen, A. Singlay, A. Singhz, K. Ramachandranz, L. Xuz, Y. Zhangz, X. Wen, and Y. Chen. OSA: an optical switching architecture for data center networks with unprecedented flexibility. In NSDI '12.
7. Y. Chen, S. Alspaugh, D. Borthakur, and R. Katz. Energy efficiency for large-scale mapreduce workloads with significant interactive analysis. In EuroSys '12.
8. A. Curtis, W. Kim, and P. Yalagandula. Mahout: Low-overhead datacenter traffic management using end-host-based elephant detection. In INFOCOM '11.
9. A. R. Curtis, J. C. Mogul, J. Tourrilhes, P. Yalagandula, P. Sharma, and S. Banerjee. Devoflow: scaling flow management for high-performance networks. In ACM SIGCOMM '11.
10. A. Das, C. Lumezanu, Y. Zhang, V. Singh, G. Jiang, and C. Yu. Transparent and flexible network management for big data processing in the cloud. In Proceedings of the 5th USENIX conference on Hot topics in cloud computing, HotCloud '13.
11. J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. In OSDI '04.
12. N. Farrington, G. Porter, S. Radhakrishnan, H. H. Bazzaz, V. Subramanya, Y. Fainman, G. Papen, and A. Vahdat. Helios: A hybrid electrical/optical switch architecture for modular data centers. In ACM SIGCOMM '10.
13. A. Feldmann, N. Kammenhuber, O. Maennel, B. Maggs, R. De Prisco, and R. Sundaram. A methodology for estimating interdomain web traffic demand. In ACM IMC '04.
14. A. D. Ferguson, A. Guha, J. Place, R. Fonseca, and S. Krishnamurthi. Participatory networking. In Proceedings of the 2nd USENIX conference on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services, Hot-ICE'12.
15. R. Fonseca, G. Porter, R. H. Katz, S. Shenker, and I. Stoica. X-Trace: a pervasive network tracing framework. In NSDI' 07.
16. Q. Fu, J.-G. Lou, Y. Wang, and J. Li. Execution anomaly detection in distributed systems through unstructured log analysis. In Proceedings of the 2009 Ninth IEEE International Conference on Data Mining, ICDM '09.
17. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google file system. In SOSP '03.
18. A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: A scalable and flexible data center network. In ACM SIGCOMM '09.
19. S. Huang, J. Huang, J. Dai, T. Xie, and B. Huang. The HiBench benchmark suite: Characterization of the mapreduce-based data analysis. In 2010 IEEE 26th International Conference on Data Engineering Workshops, ICDEW '10.
20. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In EuroSys '07.
21. R. Love. Kernel korner: intro to Inotify. Linux J., 2005.
22. D. L. Mills. RFC 1305. Network Time Protocol (Version 3). 1992.
23. M. Roughan, M. Thorup, and Y. Zhang. Traffic engineering with estimated traffic matrices. In ACM IMC '03.
24. K. Shvachko, H. Kuang, S. Radia, and R. Chansler. The Hadoop Distributed File System. In Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies, MSST '10.
25. J. Tan, X. Pan, S. Kavulya, R. Gandhi, and P. Narasimhan. Mochi: visual log-analysis based tools for debugging Hadoop. In Proceedings of the first USENIX conference on Hot topics in cloud computing, HotCloud '09.
26. B. Vamanan, J. Hasan, and T. Vijaykumar. Deadline-aware datacenter TCP (D2TCP). In ACM SIGCOMM '12.
27. R. van Renesse and F. B. Schneider. Chain replication for supporting high throughput and availability. In OSDI '04.
28. G. Wang, D. G. Andersen, M. Kaminsky, K. Papagiannaki, T. E. Ng, M. Kozuch, and M. Ryan. c-Through: Part-time optics in data centers. In ACM SIGCOMM '10.
29. G. Wang, T. E. Ng, and A. Shaikh. Programming your network at runtime for big data applications. In Proceedings of the first workshop on Hot topics in software defined networks, HotSDN '12.
30. W. Wang, K. Zhu, L. Ying, J. Tan, and L. Zhang. Map task scheduling in MapReduce with data locality: Throughput and heavy-traffic optimality. In INFOCOM '13.
31. K. C. Webb, A. C. Snoeren, and K. Yocum. Topology switching for data center networks. In Proceedings of the 11th USENIX conference on Hot topics in management of internet, cloud, and enterprise networks and services, Hot-ICE '11.
32. C. Wilson, H. Ballani, T. Karagiannis, and A. Rowtron. Better never than late: meeting deadlines in datacenter networks. In ACM SIGCOMM '11.
33. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In EuroSys '10.
34. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: cluster computing with working sets. In Proceedings of the 2nd USENIX conference on Hot topics in cloud computing, HotCloud '10.
35. X. Zhang, E. Tune, R. Hagmann, R. Jnagal, V. Gokhale, and J. Wilkes. CPI2: CPU performance isolation for shared compute clusters. In EuroSys '13.