Evaluating I/O aware network management for scientific workflows on networked clouds

Proc. of NDM 2013: 3rd Int. Workshop on Network-Aware Data Management - Held in Conjunction with SC 2013: The Int. Conference for High Performance Computing, Networking, Storage and Analysis

Volumn , Issue , 2013, Pages

  Mandal, Anirban ^a   Ruth, Paul ^a   Baldin, Ilya ^a   Xin, Yufeng ^a   Castillo, Claris ^a   Rynge, Mats ^b   Deelman, Ewa ^b  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

^b ISI USC   (United States)

Author keywords

I O performance; Networked clouds; Performance evaluation; Performance monitoring; Scientific workflows

Indexed keywords

APPLICATION PERFORMANCE; DATA-INTENSIVE WORKLOADS; INFRASTRUCTURE PLANNING; PERFORMANCE EVALUATION; PERFORMANCE MONITORING; REAL TIME MONITORING; SCIENTIFIC WORKFLOWS; WORKFLOW EXECUTION;

DIGITAL STORAGE; EXPERIMENTS; INFORMATION MANAGEMENT; INFRASTRUCTURE AS A SERVICE (IAAS); TESTBEDS;

NETWORK MANAGEMENT;

EID: 84892912414     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2534695.2534698     Document Type: Conference Paper

References (32)

1. Amazon Elastic Compute Cloud (Amazon EC2). http://www.amazon.com/ec2.
2. I. Baldine, Y. Xin, A. Mandal, P. Ruth, A. Yumerefendi, and J. Chase. Exogeni: A multi-domain infrastructure-as-a-service testbed. In 8th International ICST Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities (TRIDENTCOM 2012), 2012.
3. J. Chase, L.Grit, D.Irwin, V.Marupadi, P.Shivam, and A.Yumerefendi. Beyond virtual data centers: Toward an open resource control architecture. In Selected Papers from the International Conference on the Virtual Computing Initiative (ACM Digital Library), May 2007.
4. J. S. Chase, D. E. Irwin, L. E. Grit, J. D. Moore, and S. E. Sprenkle. Dynamic Virtual Clusters in a Grid Site Manager. In Proceedings of the Twelfth International Symposium on High Performance Distributed Computing (HPDC), June 2003.
5. E. Deelman, G. Juve, M. Malawski, and J. Nabrzyski. Hosted science: Managing computational workflows in the cloud. Parallel Processing Letters, 23(2), 2013.
6. E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. B. Berriman, J. Good, et al. Pegasus: A framework for mapping complex scientific workflows onto distributed systems. Scientific Programming, 13(3):219-237, 2005.
7. Eucalyptus Systems. http://www.eucalyptus.com/.
8. ExoGENI Wiki. http://wiki.exogeni.net.
9. Y. Fu, J. Chase, B. Chun, S. Schwab, and A. Vahdat. SHARP: An Architecture for Secure Resource Peering. In Proceedings of the 19th ACM Symposium on Operating System Principles, October 2003.
10. D. Ghoshal, R. S. Canon, and L. Ramakrishnan. I/o performance of virtualized cloud environments. In Proceedings of the second international workshop on Data intensive computing in the clouds, DataCloud-SC '11, pages 71-80, New York, NY, USA, 2011. ACM.
11. A. Iosup, S. Ostermann, N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema. Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans. Parallel Distrib. Syst., 22(6):931-945, June 2011.
12. D. Irwin, J. S. Chase, L. Grit, A. Yumerefendi, D. Becker, and K. G. Yocum. Sharing Networked Resources with Brokered Leases. In Proceedings of the USENIX Technical Conference, June 2006.
13. K. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H. J. Wasserman, and N. Wright. Performance analysis of high performance computing applications on the amazon web services cloud. In Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on, pages 159-168, 2010.
14. J. C. Jacob, D. S. Katz, G. B. Berriman, J. C. Good, A. C. Laity, E. Deelman, C. Kesselman, G. Singh, M. Su, T. A. Prince, and R. Williams. Montage a grid portal and software toolkit for science grade astronomical image mosaicking. Int. J. Comput. Sci. Eng., 4(2):73-87, July 2009.
15. G. Juve and E. Deelman. Scientific workflows and clouds. Crossroads, 16(3):14-18, Mar. 2010.
16. G. Juve and E. Deelman. Scientific workflows in the cloud. In M. Cafaro and G. Aloisio, editors, Grids, Clouds and Virtualization, Computer Communications and Networks, pages 71-91. Springer London, 2011.
17. G. Juve, E. Deelman, G. B. Berriman, B. P. Berman, and P. Maechling. An evaluation of the cost and performance of scientific workflows on amazon ec2. J. Grid Comput., 10(1):5-21, 2012.
18. Z. Li, L. O'Brien, H. Zhang, and R. Cai. A factor framework for experimental design for performance evaluation of commercial cloud services. In CloudCom, pages 169-176. IEEE, 2012.
19. X. Lin, Y. Mao, F. Li, and R. Ricci. Towards fair sharing of block storage in a multi-tenant cloud. In Proceedings of the 4th USENIX conference on Hot Topics in Cloud Ccomputing, HotCloud'12, pages 15-15, Berkeley, CA, USA, 2012. USENIX Association.
20. Linux iotop. http://guichaz.free.fr/iotop/.
21. M. Malawski, G. Juve, E. Deelman, and J. Nabrzyski. Cost-and deadline-constrained provisioning for scientific workflow ensembles in iaas clouds. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12, pages 22:1-22:11, Los Alamitos, CA, USA, 2012. IEEE Computer Society Press.
22. A. Mandal, I. Baldine, Y. Xin, P. Ruth, and C. Heerman. Enabling persistent queries for cross-aggregate performance monitoring. Technical Report TR-13-01, Renaissance Computing Institute, 2013, http://www.renci.org/wpcontent/ uploads/2013/04/TR-13-01.pdf.
23. A. Mandal, Y. Xin, I. Baldine, P. Ruth, C. Heerman, J. Chase, V. Orlikowski, and A. Yumerefendi. Provisioning and evaluating multi-domain networked clouds for hadoop-based applications. In Cloud Computing Technology and Science (CloudCom), 2011 IEEE Third International Conference on, pages 690-697, 2011.
24. D. Milojicic, I. M. Llorente, and R. S. Montero. Opennebula: A cloud management tool. IEEE Internet Computing, 15(2):11-14, 2011.
25. OpenStack Cloud Software. http://openstack.org.
26. S. Ostermann, A. Iosup, N. Yigitbasi, R. Prodan, T. Fahringer, and D. H. J. Epema. A performance analysis of ec2 cloud computing services for scientific computing. In D. R. Avresky, M. Diaz, A. Bode, B. Ciciani, and E. Dekel, editors, CloudComp, volume 34 of Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, pages 115-131. Springer, 2009.
27. X. Pu, L. Liu, Y. Mei, S. Sivathanu, Y. Koh, and C. Pu. Understanding performance interference of i/o workload in virtualized cloud environments. In Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing, CLOUD '10, pages 51-58, Washington, DC, USA, 2010. IEEE Computer Society.
28. L. Ramakrishnan, L. Grit, A. Iamnitchi, D. Irwin, A. Yumerefendi, and J. Chase. Toward a Doctrine of Containment: Grid Hosting with Adaptive Resource Control. In Supercomputing (SC06), November 2006.
29. J. Shafer. I/o virtualization bottlenecks in cloud computing today. In Proceedings of the 2nd conference on I/O virtualization, WIOV'10, pages 5-5, Berkeley, CA, USA, 2010. USENIX Association.
30. R. Tudoran, A. Costan, G. Antoniu, and L. Bougé. A performance evaluation of azure and nimbus clouds for scientific applications. In Proceedings of the 2nd International Workshop on Cloud Computing Platforms, CloudCP '12, pages 4:1-4:6, New York, NY, USA, 2012. ACM.
31. J.-S. Vöckler, G. Juve, E. Deelman, M. Rynge, and B. Berriman. Experiences using cloud computing for a scientific workflow application. In Proceedings of the 2nd international workshop on Scientific cloud computing, ScienceCloud '11, pages 15-24, New York, NY, USA, 2011. ACM.
32. J. Wang and I. Altintas. Early cloud experiences with the kepler scientific workflow system. Procedia Computer Science, 9(0):1630-1634, 2012.