Analysis of Virtualization
Technologies for High Performance
Computing Environments
Andrew J. Younge, Robert Henschel, James
T. Brown, Gregor von Laszewski, Judy Qiu,
Geoffrey C. Fox
Indiana University Bloomington
https://portal.futuregrid.org
1
Outline
•
•
•
•
•
Introduction
Related work
Feature comparison
FutureGrid experimental setup
Performance comparison
– HPCC
– SPEC
• Discussion
https://portal.futuregrid.org
2
Introduction
• What is Virtualization?
– A method of partitioning a physical computer into multiple
“virtual” computers, each acting independently as if they were
running directly on hardware.
• What is a Hypervisor?
– A technique used to run multiple operating systems
simultaneously on a single resource.
– Also called a Virtual Machine Monitor (VMM).
• What is a Virtual Machine?
– A software implementation of a machine that executes as if it
was running on a physical resource directly.
• Why does it matter?
– Cloud Computing!!!
https://portal.futuregrid.org
3
Motivation
• Most “Cloud” deployments rely on
virtualization.
– Amazon EC2, GoGrid, Azure, Rackspace Cloud …
– Nimbus, Eucalyptus, OpenNebula, OpenStack …
• Number of Virtualization tools or Hypervisors
available today.
– Xen, KVM, VMWare, Virtualbox, Hyper-V …
• Need to compare these hypervisors for use
within the scientific computing community.
https://portal.futuregrid.org
4
Current Hypervisors
https://portal.futuregrid.org
5
Hypervisors
• Evaluate Xen, KVM, and VirtualBox hypervisors
against native hardware
– Common, well documented
– Open source, open architecture
– Relatively mature & stable
• Cannot benchmark VMWare hypervisors due
to proprietary licensing issues.
https://portal.futuregrid.org
6
Related Research
• Some work has already been done to evaluate
performance…
•
•
•
•
•
Karger, P. & Safford, D. I/O for virtual machine monitors: Security and performance issues.
Security & Privacy, IEEE, IEEE, 2008, 6, 16-23
Koh, Y.; Knauerhase, R.; Brett, P.; Bowman, M.; Wen, Z. & Pu, C. An analysis of performance
interference effects in virtual environments. Performance Analysis of Systems & Software,
2007. ISPASS 2007. IEEE International Symposium on, 2007, 200209.
K. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H. Wasserman, and N.
Wright, “Performance Analysis of High Performance Computing Applications on the Amazon
Web Services Cloud,” in 2nd IEEE International Conference on Cloud Computing Technology
and Science. IEEE, 2010, pp. 159–168.
P. Barham, B. Dragovic, K. Fraser, S. Hand, T. L. Harris, A. Ho, R. Neugebauer, I. Pratt, and A.
Warfield, “Xen and the art of virtualization,” in Proceedings of the 19th ACM Symposium on
Operating Systems Principles, New York, U. S. A., Oct. 2003, pp. 164–177.
Adams, K. & Agesen, O. A comparison of software and hardware techniques for x86
virtualization. Proceedings of the 12th international conference on Architectural support for
programming languages and operating systems, 2006, 2-13.
https://portal.futuregrid.org
7
Features
Xen
KVM
VirtualBox
VMWare
Paravirtualization
Yes
No
No
No
Full Virtualization
Yes
Yes
Yes
Yes
Host CPU
X86, X86_64, IA64
X86, X86_64, IA64,
PPC
X86, X86_64
X86, X86_64
Guest CPU
X86, X86_64, IA64
X86, X86_64, IA64,
PPC
X86, X86_64
X86, X86_64
Host OS
Linux, Unix
Linux
Windows, Linux, Unix
Proprietary Unix
Guest OS
Linux, Windows, Unix
Linux, Windows, Unix
Linux, Windows, Unix
Linux, Windows, Unix
VT-x / AMD-v
Opt
Req
Opt
Opt
Supported Cores
128
16*
32
8
Supported Memory
4TB
4TB
16GB
64GB
Xen-GL
VMGL
Open-GL
Open-GL, DirectX
GPL
GPL
GPL/Proprietary
Proprietary
3D Acceleration
Licensing
https://portal.futuregrid.org
8
Usability
• KVM and VirtualBox trivial to install & deploy.
– Xen requires special kernel, leading to more
complications.
– VMWare ESX runs as a standalone hypervisor.
• All are supported under Libvirt API.
– Used by many IaaS frameworks.
• Xen & Virtualbox have nice CLI, VMWare has
an advanced web based GUI.
https://portal.futuregrid.org
9
Performance Analysis
• In order to assess various performance
metrics, benchmarks are needed.
– Provide a fair, apples-to-apples comparison
between each hypervisor.
– Comparisons can be made across other
benchmark submissions on different machines.
– Reproducible and verifiable results.
– Open standards, no special optimizations or tricks
available (hopefully).
https://portal.futuregrid.org
10
FutureGrid
FutureGrid is an experimental grid and cloud test-bed with
a wide variety of computing services available to users.
Germany
Interet 2
TeraGrid
NID
10GB/s
Router
7
10GB/s
10GB/s
10GB/s
10GB/s
4
11
6
France
5
IU:
1GB/s
7
11 TF IBM 1024 cores
6 TF Cray 672 cores
TACC: 12 TF Dell 1152 cores
UCSD: 7 TF IBM 672 cores
12
2
UC:
7 TF IBM 672 cores
UF:
3 TF IBM 256 cores
• HW Resources at: Indiana University, San Diego Supercomputer Center, University of Chicago /
Argonne National Lab, Texas Applied Computing Center, University of Florida, & Purdue
• Software Partners: University of Southern California, University of Tennessee Knoxville, University of
Virginia, Technische Universtität Dresden
https://portal.futuregrid.org
Testing Environment
• All tests conducted on the • India: 256 1U compute nodes
IU India cluster as part of
– 2 Intel Xeon 5570 Quad core
FutureGrid.
CPUs at 2.93Ghz
– 24GB DDR2 Ram
• Identical nodes used for
each hypervisor, as well
– 500GB 10k HDDs
as a bare-metal (native)
– InfiniBand DDR 20Gbs
machine for the control
group.
• Each host OS runs RedHat
Enterprise Linux 5.5.
12
Testing Environment (cont)
• Guest Operating System:
–
–
–
–
–
–
–
RedHat Enterprise Linux 5.5 x86_64
Kernel 2.6.18-194.8.1.el5
Minimal server installation (Base, Core)
Default settings and services
Full virtualization using Intel VT-x
Each VM gets 16GB of memory
Virtualization Add- ons:
•
•
•
•
Xen: version 3.1.0 with 2.6.18-194.8.1.el5xen kernel
KVM: version 83
VirtualBox: version 3.2.10_66523_rhel5
Bare-Metal OS = Host OS
https://portal.futuregrid.org
13
Benchmark Setup
HPCC
• Industry standard HPC
benchmark suite from
University of Tennessee.
• Sponsored by NSF, DOE,
DARPA.
• Includes Linpack, FFT
benchmarks, and more.
• Targeted for CPU and
Memory analysis.
SPEC OMP
• From the Standard
Performance Evaluation
Corporation.
• Suite of applications
aggrigated together
• Focuses on well-rounded
OpenMP benchmarks.
• Full system performance
analysis for OMP.
https://portal.futuregrid.org
14
https://portal.futuregrid.org
15
https://portal.futuregrid.org
16
https://portal.futuregrid.org
17
https://portal.futuregrid.org
18
https://portal.futuregrid.org
19
Performance Roundup
• Hypervisor performance in Linpack reaches roughly 70% of native
performance during our tests, with Xen showing a high degree of
variation.
• FFT benchmarks seem to be less influenced by hypervisors, with
KVM and VirtualBox performing at native speeds yet Xen incurs a
50% performance hit with MPIFFT.
• With PingPong latency KVM and VirtualBox perform close to native
speeds while Xen has large latencies under maximum conditions.
• Bandwidth tests vary largely, with the VirtualBox hypervisor
performing the best, however having large performance variations.
• SPEC benchmarks show KVM at near-native speed, with Xen and
VirtualBox close behind.
https://portal.futuregrid.org
20
Conclusion
• Big Question: Is Cloud Computing viable for
scientific High Performance Computing?
– Our answer is “Yes” (for the most part).
• Features: All hypervisors are similar.
• Performance: KVM is fastest across most
benchmarks, VirtualBox close.
• Overall, we have found KVM to be the best
hypervisor choice for HPC.
– Currently moving to KVM for all of FutureGrid.
https://portal.futuregrid.org
21
THANK YOU!
https://portal.futuregrid.org
22
Virtual Machines
$ : 7/#; ) 7: - &<) #
!" #
8229#
!" #
8229#
8229#
8229#
1 2) &%345##
*67/) , #
1 2) &%345##
*67/) , #
*+, - .%/) ' #
0%&' ( %&) #
*+, - .%/) ' #
0%&' ( %&) #
!" " #
$ %&' ( %&) #
http://futuregrid.org
23
Feature Roundup
• All hypervisors evaluated have acceptable
level of features for x86 virtualization.
• Xen provides best expandability, supporting
up to 128 CPUs and 4TB of RAM.
– Can remove CPU limit for KVM.
– VirtualBox needs to add support for >16GB RAM.
• All have API plugins to allow for simplified IaaS
usage.
Need to learn more from a performance comparison….
https://portal.futuregrid.org
24
Benchmarks
• SPEC Benchmarks
– specCPU 2006 – CPU bound benchmark
– specMPI 2007 – MPI: cpu, memory, network
– specOMP 2001 – Open MP: cpu, memory, IPC
– specVIRT 2010 – CPU, memory, disk I/O, network
• HPCC Benchmarks
– HPL (Linpack), DGEMM, STREAM, PTRANS,
RandomAccess, FFT, etc
http://futuregrid.org
25
Descargar

Slide 1