SR-IOV support for virtualization on InfiniBand clusters: Early experience

Proceedings - 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2013

Volumn , Issue , 2013, Pages 385-392

  Jose, Jithin ^a   Li, Mingzhe ^a   Lu, Xiaoyi ^a   Kandalla, Krishna Chaitanya ^a   Arnold, Mark Daniel ^a   Panda, Dhabaleswar K ^a  

^a The Ohio State University   (United States)

Author keywords

HPC; InfiniBandClusters; SR IOV; Virtualization

Indexed keywords

HIGH PERFORMANCE COMPUTING SYSTEMS; HPC; INFINIBANDCLUSTERS; PARTITIONED GLOBAL ADDRESS SPACE; PERFORMANCE CHARACTERISTICS; POINT-TO-POINT COMMUNICATION; SR-IOV; VIRTUALIZATIONS;

CLOUD COMPUTING; COMMERCE; COMPLEX NETWORKS; COMPUTER ARCHITECTURE; COMPUTER SYSTEMS; GRID COMPUTING; HARDWARE; QUEUEING NETWORKS; VIRTUAL REALITY;

BENCHMARKING;

EID: 84881295404     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGrid.2013.76     Document Type: Conference Paper

References (36)

1. M. Rosenblum and T. Garfinkel, "Virtual Machine Monitors: Current Technology and Future Trends, " Computer, vol. 38, no. 5, pp. 39-47, 2005.
2. K. Adams and O. Agesen, "A Comparison of Software and Hardware Techniques for x86 Virtualization, " in ACM SIGOPS Operating Systems Review, vol. 40, no. 5. ACM, 2006, pp. 2-13.
3. AMD, "AMD Secure Virtual Machine Architecture Reference Manual, Rev. 3.01, May 2005.".
4. R. Uhlig, G. Neiger, D. Rodgers, A. Santoni, F. Martins, A. Anderson, S. Bennett, A. Kagi, F. Leung, and L. Smith, "Intel Virtualization Technology, " Computer, vol. 38, no. 5, pp. 48-56, 2005.
5. MPI Forum, "MPI: A Message Passing Interface, " in Proceedings of Supercomputing, 1993.
6. "Partitioned Global Address Space, " http://www.pgas.org/.
7. Infiniband Trade Association, http://www.infinibandta.org.
8. Top500 Supercomputing System, http://www.top500.org.
9. "IP over InfiniBand Working Group, " http://www.ietf.org/html. charters/ipoib-charter.html.
10. J. Sugerman, G. Venkitachalam, and B. Lim, "Virtualizing I/O Devices on VMware Workstations hosted Virtual Machine Monitor, " in USENIX Annual Technical Conference, 2001.
11. Kernel Virtual Machine, http://kvm.qumranet.com/.
12. J. Santos, Y. Turner, G. Janakiraman, and I. Pratt, "Bridging the Gap Between Software and Hardware Techniques for I/O Virtualization, " in Proceedings of the USENIX08 Annual Technical Conference, 2008.
13. Single Root I/O Virtualization, http://www.pcisig.com/specifications/iov/ singleroot/.
14. Multi-Root I/O Virtualization, http://www.pcisig.com/specifications/iov/ multi-root/.
15. J. Liu, "Evaluating Standard-Based Self-Virtualizing Devices: A Performance Study on 10 GbE NICs with SR-IOV Support, " in Parallel & Distributed Processing (IPDPS), 2010 IEEE International Symposium on. IEEE, 2010, pp. 1-12.
16. Y. Dong, X. Yang, J. Li, G. Liao, K. Tian, and H. Guan, "High Performance Network Virtualization with SR-IOV, " Journal of Parallel and Distributed Computing, 2012.
17. J. Suzuki, Y. Hidaka, J. Higuchi, T. Baba, N. Kami, and T. Yoshikawa, "Multi-Root Share of Single-Root I/O Virtualization (SR-IOV) Compliant PCI Express Device, " in High Performance Interconnects (HOTI), 2010 IEEE 18th Annual Symposium on. IEEE, 2010, pp. 25-31.
18. Z. Huang, R. Ma, J. Li, Z. Chang, and H. Guan, "Adaptive and Scalable Optimizations for High Performance SR-IOV, " in Cluster Computing (CLUSTER), 2012 IEEE International Conference on. IEEE, 2012, pp. 459-467.
19. MVAPICH2/MVAPICH2-X: MPI/PGAS over InfiniBand, 10GigE/iWARP and RoCE, http://mvapich.cse.ohio-state.edu/.
20. M. Koop, J. Sridhar, and D. Panda, "TupleQ: Fully- Asynchronous and Zero-Copy MPI over InfiniBand, " in Parallel & Distributed Processing, 2009. IPDPS 2009. IEEE International Symposium on. IEEE, 2009, pp. 1-8.
21. M. J. Koop, J. K. Sridhar, and D. K. Panda, "Scalable MPI Design over InfiniBand using eXtended Reliable Connection.".
22. M. Koop, T. Jones, and D. Panda, "MVAPICH-Aptus: Scalable High-Performance Multi-Transport MPI over Infini- Band, " in Parallel and Distributed Processing, 2008. IPDPS 2008. IEEE International Symposium on. IEEE, 2008, pp. 1-12.
23. S. Sur, M. Koop, and D. Panda, "High-Performance and Scalable MPI over InfiniBand with Reduced Memory Usage: an In-Depth Performance Analysis, " in Proceedings of the 2006 ACM/IEEE conference on Supercomputing. ACM, 2006, p. 105.
24. J. Jose, K. Kandalla, M. Luo, and D. Panda, "Supporting Hybrid MPI and OpenSHMEM over InfiniBand: Design and Performance Evaluation, " in Parallel Processing (ICPP), 2012 41st International Conference on. IEEE, 2012, pp. 219-228.
25. M. Koop, M. Luo, and D. Panda, "Reducing Network Contention with Mixed Workloads on Modern Multicore Clusters, " in Cluster Computing and Workshops, 2009. CLUSTER' 09. IEEE International Conference on. IEEE, 2009, pp. 1-10.
26. MLNX OFED (OpenFabrics Enterprise Distribution), http://www.mellanox.com/ related-docs/prodsoftware/PBOFED.pdf.
27. NetPerf, http://www.netperf.org/netperf/.
28. OSU Micro-benchmarks, http://mvapich.cse.ohio-state.edu/benchmarks/.
29. J. Liu, W. Huang, B. Abali, and D. Panda, "High Performance VMM-bypass I/O in Virtual Machines, " in Proceedings of the annual conference on USENIX, vol. 6, 2006, pp. 3-3.
30. The Graph500, http://www.graph500.org.
31. A. Menon, J. R. Santos, Y. Turner, G. J. Janakiraman, and W. Zwaenepoel, "Diagnosing performance overheads in the xen virtual machine environment, " in Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments, ser. VEE '05. New York, NY, USA: ACM, 2005, pp. 13-23.
32. P. Apparao, S. Makineni, and D. Newell, "Characterization of network processing overheads in xen, " in Proceedings of the 2nd International Workshop on Virtualization Technology in Distributed Computing, ser. VTDC '06. Washington, DC, USA: IEEE Computer Society, 2006.
33. W. Huang, M. J. Koop, Q. Gao, and D. K. Panda, "Virtual Machine Aware Communication Libraries for High Performance Computing, " in Proceedings of the 2007 ACM/IEEE conference on Supercomputing, ser. SC '07, New York, NY, USA, 2007.
34. W. Huang, J. Liu, B. Abali, and D. K. Panda, "A Case for High Performance Computing with Virtual Machines, " in Proceedings of the 20th annual international conference on Supercomputing, ser. ICS '06, New York, NY, USA, 2006.
35. J. Liu, W. Huang, B. Abali, and D. K. Panda, "High Performance VMM-bypass I/O in Virtual Machines, " in Proceedings of the annual conference on USENIX '06 Annual Technical Conference, ser. ATEC '06, Berkeley, CA, USA, 2006.
36. W. Huang, J. Liu, M. Koop, B. Abali, and D. Panda, "Nomad: Migrating OS-bypass Networks in Virtual Machines, " in Proceedings of the 3rd international conference on Virtual execution environments, ser. VEE '07, New York, NY, USA, 2007.