Performance analysis of HPC applications in the cloud

Future Generation Computer Systems

Volumn 29, Issue 1, 2013, Pages 218-229

  Expósito, Roberto R ^a   Taboada, Guillermo L ^a   Ramos, Sabela ^a   Touriño, Juan ^a   Doallo, Ramón ^a  

^a UNIVERSITY OF A CORUÑA   (Spain)

Author keywords

Amazon EC2 Cluster Compute platform; Cloud computing; High Performance Computing; MPI; OpenMP

Indexed keywords

10-GIGABIT ETHERNET; AMAZON EC2; APPLICATION SCALABILITY; COMMUNICATION MIDDLEWARE; COMMUNICATION PERFORMANCE; COMPUTATIONAL POWER; COST-EFFICIENCY; HIGH PERFORMANCE COMPUTING; HIGH PERFORMANCE COMPUTING (HPC); HIGH-SPEED; MPI; MULTI-THREADING; NAS PARALLEL BENCHMARKS; NETWORK COMMUNICATIONS; OPENMP; PERFORMANCE ANALYSIS; PERFORMANCE BOTTLENECKS; SHARED MEMORIES; VIRTUAL MACHINES; VIRTUALIZATIONS; VIRTUALIZED ENVIRONMENT;

APPLICATION PROGRAMMING INTERFACES (API); CLOUD COMPUTING; COMMUNICATION; INTERCONNECTION NETWORKS; MESSAGE PASSING; MIDDLEWARE; MULTITASKING; VIRTUAL REALITY;

SCALABILITY;

EID: 84864058807     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2012.06.009     Document Type: Article

References (42)

1. R. Buyya, C.S. Yeo, S. Venugopal, J. Broberg, and I. Brandic Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility Future Generation Computer Systems 25 6 2009 599 616
2. L. Rodero-Merino, L.M. Vaquero, V. Gil, F. Galán, J. Fontán, R.S. Montero, and I.M. Llorente From infrastructure delivery to service management in clouds Future Generation Computer Systems 26 8 2010 1226 1240
3. Amazon Web Services LLC, Amazon elastic compute cloud (Amazon EC2), last visited: May 2012. http://aws.amazon.com/ec2.
4. C. Evangelinos, C.N. Hill, Cloud computing for parallel scientific HPC applications: feasibility of running coupled atmosphere-ocean climate models on Amazon's EC2, in: Proc. 1st Workshop on Cloud Computing and Its Applications, CCA'08, Chicago, IL, USA, 2008, pp. 1-6.
5. E. Walker Benchmarking Amazon EC2 for high-performance scientific computing;login: The usenix journal 33 5 2008 18 23
6. J. Ekanayake, G.C. Fox, High performance parallel computing with clouds and cloud technologies, in: Proc. 1st Intl. Conference on Cloud Computing, CLOUDCOMP'09, Munich, Germany, 2009, pp. 20-38.
7. Amazon Web Services LLC, High performance computing using amazon EC2, last visited: May 2012. http://aws.amazon.com/ec2/hpc-applications/.
8. D.H. Bailey The NAS parallel benchmarks International Journal of High Performance Computing Applications 5 3 1991 63 73
9. H. Jin, and R.F. Van der Wijngaart Performance characteristics of the multi-zone NAS parallel benchmarks Journal of Parallel and Distributed Computing 66 2006 674 685
10. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, A. Warfield, Xen and the art of virtualization, in: Proc. 19th ACM Symposium on Operating Systems Principles, SOSP'03, Bolton Landing, NY, USA, 2003, pp. 164-177.
11. A. Whitaker, M. Shaw, S.D. Gribble, Denali: lightweight virtual machines for distributed and networked applications, Technical Report 02-02-01, University of Washington, USA, 2002.
12. D. Abramson Intel virtualization technology for directed I/O Intel Technology Journal 10 3 2006 179 192
13. AMD virtualization technology (AMD-V), last visited: May 2012. http://sites.amd.com/us/business/it-solutions/virtualization/Pages/amd-v.aspx.
14. D. Ghoshal, R.S. Canon, L. Ramakrishnan, I/O performance of virtualized cloud environments, in: Proc. 2nd Intl. Workshop on Data Intensive Computing in the Clouds, DataCloud-SC'11, Seattle, WA, USA, 2011, pp. 71-80.
15. K. Fraser, S. Hand, R. Neugebauer, I. Pratt, A. Warfield, M. Williamson, Safe hardware access with the xen virtual machine monitor, in: Proc. 1st Workshop on Operating System and Architectural Support for the On-Demand IT Infrastructure, OASIS'04, Boston, MA, USA, 2004.
16. A. Menon, A.L. Cox, W. Zwaenepoel, Optimizing network virtualization in Xen, in: Proc. USENIX'06 Annual Technical Conference, Boston, MA, USA, 2006, p. 2.
17. J.R. Santos, Y. Turner, G. Janakiraman, I. Pratt, Bridging the gap between software and hardware techniques for I/O virtualization, in: Proc. USENIX'08 Annual Technical Conference, Boston, MA, USA, 2008, pp. 29-42.
18. H. Raj, K. Schwan, High performance and scalable I/O virtualization via self-virtualized devices, in: Proc. 16th IEEE Intl. Symposium on High-Performance Distributed Computing, HPDC'07, Monterey, CA, USA, 2007, pp. 179-188.
19. K. Mansley, G. Law, D. Riddoch, G. Barzini, N. Turton, S. Pope, Getting 10 Gb/s from Xen: safe and fast device access from unprivileged domains, in: Proc. Workshop on Virtualization/Xen in High-Performance Cluster and Grid Computing, VHPC'07, Rennes, France, 2007, pp. 224-233.
20. B.-A. Yassour, M. Ben-Yehuda, O. Wasserman, Direct device assignment for untrusted fully-virtualized virtual machines, Tech. Rep., 2008.
21. PCI SIG. I/O virtualization, last visited: May 2012. http://www.pcisig. com/specifications/iov/.
22. J. Liu, Evaluating standard-based self-virtualizing devices: a performance study on 10 GbE NICs with SR-IOV support, in: Proc. 24th IEEE Intl. Parallel and Distributed Processing Symposium, IPDPS'10, Atlanta, GA, USA, 2010, pp. 1-12.
23. A. Gordon, N. Amit, N. Har'El, M. Ben-Yehuda, A. Landau, D. Tsafrir, A. Schuster, ELI: bare-metal performance for I/O virtualization, in: Proc. 17th Intl. Conference on Architectural Support for Programming Languages Operating Systems, ASPLOS'12, London, UK, 2012.
24. C. Vecchiola, S. Pandey, R. Buyya, High-performance cloud computing: a view of scientific applications, in: Proc. 10th Intl. Symposium on Pervasive Systems, Algorithms, and Networks, ISPAN'09, Kaoshiung, Taiwan, ROC, 2009, pp. 4-16.
25. G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B.P. Berman, P. Maechling, Data sharing options for scientific workflows on Amazon EC2, in: Proc. 22th ACM/IEEE Conference on Supercomputing, SC'10, New Orleans, LA, USA, 2010, pp. 1-9.
26. A. Iosup, S. Ostermann, N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema Performance analysis of cloud computing services for many-tasks scientific computing IEEE Transactions on Parallel and Distributed Systems 22 2011 931 945
27. S. Gogouvitis, K. Konstanteli, S. Waldschmidt, G. Kousiouris, G. Katsaros, A. Menychtas, D. Kyriazis, and T. Varvarigou Workflow management for soft real-time interactive applications in virtualized environments Future Generation Computer Systems 28 1 2012 193 209
28. J. Napper, P. Bientinesi, Can cloud computing reach the TOP500? in: Proc. Combined Workshops on UnConventional High Performance Computing Workshop Plus Memory Access Workshop, UCHPC-MAW'09, Ischia, Italy, 2009, pp. 17-20.
29. K.R. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H.J. Wasserman, N.J. Wright, Performance analysis of high performance computing applications on the amazon web services cloud, in: Proc. 2nd IEEE Intl. Conference on Cloud Computing Technology and Science, CloudCom'10, Indianapolis, USA, 2010, pp. 159-168.
30. A.G. Carlyle, S.L. Harrell, P.M. Smith, Cost-effective HPC: the community or the cloud? in: Proc. 2nd IEEE Intl. Conference on Cloud Computing Technology and Science, CloudCom'10, Indianapolis, USA, 2010, pp. 169-176.
31. N. Regola, J.C. Ducom, Recommendations for virtualization technologies in high performance computing, in: Proc. 2nd IEEE Intl. Conference on Cloud Computing Technology and Science, CloudCom'10, Indianapolis, USA, 2010, pp. 409-416.
32. L. Ramakrishnan, R.S. Canon, K. Muriki, I. Sakrejda, N.J. Wright, Evaluating interconnect and virtualization performance for high performance computing, in: Proc. 2nd Intl. Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems, PMBS'11, Seattle, WA, USA, 2011, pp. 1-2.
33. Y. Zhai, M. Liu, J. Zhai, X. Ma, W. Chen, Cloud versus in-house cluster: evaluating amazon cluster compute instances for running MPI applications, in: Proc. 23th ACM/IEEE Conference on Supercomputing, SC'11, State of the Practice Reports, Seattle, WA, USA, 2011, pp. 1-10.
34. V. Mauch, M. Kunze, M. Hillenbrand, High performance cloud computing, Future Generation Computer Systems (2012), http://dx.doi.org/10.1016/j.future. 2012.03.011.
35. G. Neiger, A. Santoni, F. Leung, D. Rodgers, and R. Uhlig Intel virtualization technology: hardware support for efficient processor virtualization Intel Technology Journal 10 3 2006 167 178
36. AMD I/O virtualization technology (IOMMU) specification, last visited: May 2012. http://support.amd.com/us/ProcessorTechDocs/34434-IOMMU-Rev1.262-11- 09.pdf.
37. Open MPI Website, last visited: May 2012. http://www.open-mpi.org/.
38. MPICH2 Website, last visited: May 2012. http://www.mcs.anl.gov/research/ projects/mpich2/.
39. G.L. Taboada, J. Touriño, and R. Doallo F-MPJ: scalable java message-passing communications on parallel systems Journal of Supercomputing 60 1 2012 117 140
40. M. Baker, B. Carpenter, MPJ: a proposed Java message passing API and environment for high performance computing, in: Proc. 2nd International Workshop on Java for Parallel and Distributed Computing, JavaPDC'00, Cancun, Mexico, 2000, pp. 552-559.
41. D.A. Mallón, G.L. Taboada, J. Touriño, R. Doallo, NPB-MPJ: NAS parallel benchmarks implementation for message-passing in java, in: Proc. 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing, PDP'09, Weimar, Germany, 2009, pp. 181-190.
42. W. Voorsluys, S.K. Garg, R. Buyya, Provisioning spot market cloud resources to create cost-effective virtual clusters, in: Proc. 11th Intl. Conference on Algorithms and Architectures for Parallel Processing, ICA3PP'11, Melbourne, Australia, 2011, pp. 395-408.