Designing locality and NUMA aware MPI runtime for nested virtualization based HPC cloud with SR-IOV enabled InfiniBand

VEE 2017 - Proceedings of the 2017 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

Volumn , Issue , 2017, Pages 187-200

  Zhang, Jie ^a   Lu, Xiaoyi ^a   Panda, Dhabaleswar K ^a  

^a Ohio State University   (United States)

Author keywords

Cloud computing; CMA; Container; Hypervisor; IVShmem; MPI; Nested virtualization; SR IOV

Indexed keywords

CLOUD COMPUTING; CONTAINERS; VIRTUAL REALITY;

COLLECTIVE OPERATIONS; COMPUTING ENVIRONMENTS; HIGH PERFORMANCE COMPUTING (HPC); HYPERVISOR; IVSHMEM; NESTED VIRTUALIZATION; PERFORMANCE BOTTLENECKS; RUNNING APPLICATIONS;

VIRTUALIZATION;

EID: 85018793734     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/3050748.3050765     Document Type: Conference Paper

References (36)

1. Amazon EC2. http://aws.amazon.com/ec2/.
2. M. Ben-Yehuda, M. D. Day, Z. Dubitzky, M. Factor, N. Har'El, A. Gordon, A. Liguori, O. Wasserman, and B.-A. Yassour. The turtles project: Design and implementation of nested virtualization. In OSDI, volume 10, pages 423-436, 2010.
3. Chameleon Cloud. https://www.chameleoncloud.org/.
4. Cross Memory Attach (CMA). http://kernelnewbies.org/Linuxi 3.2.
5. Docker. https://www.docker.com/.
6. Y. Dong, X. Yang, J. Li, G. Liao, K. Tian, and H. Guan. High Performance Network Virtualization with SR-IOV. Journal of Parallel and Distributed Computing, 2012.
7. W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. An Updated Performance Comparison of Virtual Machines and Linux Containers. Technical Report RC25482(AUS1407-001), 2014.
8. Google Compute Engine (GCE). https://cloud.google.com/compute/.
9. W. Huang, J. Liu, B. Abali, and D. K. Panda. A Case for High Performance Computing with Virtual Machines. In Proceedings of the 20th Annual International Conference on Supercomputing, ICS'06, New York, NY, USA, 2006.
10. W. Huang, M. J. Koop, Q. Gao, and D. K. Panda. Virtual Machine Aware Communication Libraries for High Performance Computing. In Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC), Reno, USA, 2007.
11. W. Huang, J. Liu, M. Koop, B. Abali, and D. Panda. Nomad: Migrating OS-bypass Networks in Virtual Machines. In Proceedings of the 3rd International Conference on Virtual Execution Environments, VEE'07, New York, NY, USA, 2007.
12. Z. Huang, R. Ma, J. Li, Z. Chang, and H. Guan. Adaptive and Scalable Optimizations for High Performance SR-IOV. In Proceeding of 2012 IEEE International Conference Cluster Computing (CLUSTER), pages 459-467. IEEE, 2012.
13. J. Zhang, X. Lu, J. Jose, M. Li, R. Shi, D. K. Panda. High Performance MPI Library over SR-IOV Enabled InfiniBand Clusters. In Proceedings of International Conference on High Performance Computing (HiPC), Goa, India, December 17-20 2014.
14. J. Zhang, X. Lu, J. Jose, R. Shi, D. K. Panda. Can Inter-VM Shmem Benefit MPI Applications on SR-IOV based Virtualized InfiniBand Clusters? In Proceedings of 20th International Conference Euro-Par 2014 Parallel Processing, Porto, Portugal, August 25-29 2014.
15. Kernel-based Virtual Machine (KVM). http://www.linuxkvm.org/page/Main Page.
16. K. Kim, C. Kim, S.-I. Jung, H.-S. Shin, and J.-S. Kim. Inter-domain Socket Communications Supporting High Performance and Full Binary Compatibility on Xen. In Proceedings of the 4th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE), Seattle, USA, 2008.
17. Linux Containers. https://linuxcontainers.org.
18. Linux VServer. http://linux-vserver.org.
19. J. Liu. Evaluating Standard-Based Self-Virtualizing Devices: A Performance Study on 10 GbE NICs with SR-IOV Support. In Proceeding of 2010 IEEE International Symposium Parallel & Distributed Processing (IPDPS), pages 1-12. IEEE, 2010.
20. J. Liu, W. Huang, B. Abali, and D. K. Panda. High Performance VMM-bypass I/O in Virtual Machines. In Proceedings of the Annual Conference on USENIX'06 Annual Technical Conference, ATC'06, Berkeley, CA, USA, 2006.
21. A. C. Macdonell. Shared-Memory Optimizations for Virtual Machines. PhD Thesis. University of Alberta, Edmonton, Alberta, Fall 2011.
22. Microsft. Nested virtualization. https://msdn. microsoft.com/enus/virtualization/hyperv on windows/user guide/nesting.
23. MVAPICH2-Virt. http://mvapich.cse.ohio-state.edu/.
24. Oracle. Nested virtualization: Achieving up to 2x better aws performance! https://www.ravellosystems.com/blog/nestedvirtualizationachieving-up-to-2x-better-aws-performance/.
25. OSU Micro-benchmarks. http://mvapich.cse.ohio-state.edu/benchmarks/.
26. Photon OS. https://vmware.github.io/photon/.
27. S. Soltesz, H. Pötzl, M. E. Fiuczynski, A. Bavier, and L. Peterson. Container-based Operating System Virtualization: A Scalable, High-performance Alternative to Hypervisors. In Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems (EuroSys'07), Lisbon, Portugal, 2007.
28. VMware ESX/ESXi. https://www.vmware.com/products/esxiandesx/overview.
29. VMware vCloud Air. http://vcloud.vmware.com/.
30. J. Wang, K.-L. Wright, and K. Gopalan. XenLoop: A Transparent High Performance Inter-vm Network Loopback. In Proceedings of the 17th International Symposium on High Performance Distributed Computing (HPDC), Boston, USA, 2008.
31. M. Xavier, M. Neves, F. Rossi, T. Ferreto, T. Lange, and C. De Rose. Performance Evaluation of Container-Based Virtualization for High Performance Computing Environments. In Parallel, Distributed and Network-Based Processing (PDP), 2013 21st Euromicro International Conference on, pages 233-240, Belfast, Northern Ireland, Feb. 2013. doi: 10.1109/PDP.2013.41.
32. Xen. http://www.xen.org/.
33. J. Zhang, X. Lu, and D. K. Panda. High Performance MPI Library for Container-Based HPC Cloud on InfiniBand Clusters. In 2016 45th International Conference on Parallel Processing (ICPP), Aug. 2016.
34. J. Zhang, X. Lu, and D. K. Panda. Performance Characterization of Hypervisor- and Container-Based Virtualization for HPC on SR-IOV Enabled InfiniBand Clusters. In 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), May 2016.
35. X. Zhang, S. McIntosh, P. Rohatgi, and J. L. Griffin. XenSocket: A High-throughput Interdomain Transport for Virtual Machines. In Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware (Middleware), Newport Beach, USA, 2007.
36. Y. Zhou, B. Subramaniam, K. Keahey, and J. Lange. Comparison of Virtualization and Containerization Techniques for High Performance Computing. In Proceedings of the 2015 ACM/IEEE conference on Supercomputing, Austin, USA, Nov. 2015.