Actas JP2011 - Universidad de La Laguna

Actas XXII Jornadas de Paralelismo (JP2011) , La Laguna, Tenerife, 7-9 septiembre 2011Fig. 1.rCUDA architecture on a VMM environment.When used with VMs, the communication protocolin rCUDA will make use of the virtual networkdevice to communicate the front-end and back-endmiddleware. Therefore, the network has to be configuredin a way that both the VM and the host OScan address IP packets to each other. Fig. 1 showsan rCUDA architecture diagram modified to reflectits usage in VM environments. We were able to successfullytest the current implementation of rCUDAin KVM, VB-OSE, and VMware Server virtualizationsolutions. However, we were unable to run itin a recent release of the Xen Hypervisor (3.4.3), aswe could not gain access to a recent NVIDIA GPUdriver that worked properly under the modified kernelfor the administrative domain, with the ultimatereason being that this driver is not designed to supportthe Xen environment.With rCUDA, multiple VMs running in the samephysical computer can make concurrent use of allCUDA-compatible devices installed on the computer(as long as there is enough memory on the devices tobe allocated by the different applications). Furthermore,although not addressed in this paper, rCUDAalso allows the usage of a GPU located in a differentphysical computer.In the following section we provide an in-depthanalysis of the use of rCUDA to enable GPGPUcapabilities within VMs. We believe our proposalis the first work describing a VMM-independentproduction-ready CUDA solution for VMs.V. Experimental EvaluationIn this section we conduct a collection of experimentsin order to evaluate the performance of therCUDA framework on a VMM environment. Thetarget system consists of two Quad-core Intel XeonE5410 processors running at 2.33 GHz with 8 GB ofmain memory. An OpenSuse Linux distribution withkernel version 2.6.31 is run at both host and guestsides. The GPGPU capabilities are provided by twoNVIDIA GeForce 9800 GX2 cards featuring a totalof 4 NVIDIA G92 GPUs; the driver version is 190.53.We selected two Open Source VMMs for theperformance analysis: KVM (userspace qemu-kvmv0.12.3) and VB-OSE 3.1.6, with their VMs configuredto make use of para-virtualized network devices.In addition, for load isolation purposes, each VM wasconfigured to make use of only one processor core.All benchmarks employed in our evaluation arepart of the CUDA SDK. From the 67 benchmarksin the suite, we selected 10 representativeSDK benchmarks of varying computationFig. 2.Native vs. KVM and VB-OSE.loads and data sizes, which use different CU-DA features: alignedTypes (AT), asyncAPI (AA),bicubicTexture (BT), BlackScholes (BS), box-Filter (BF), clock (CLK), convolutionSeparable(CS), fastWalshTransform (FWT), image-Denoising (ID), and matrixMul (MM). A descriptionof each benchmark can be found in the documentationof the SDK package [15]. The benchmarkswere executed with the default options, otherthan setting the target device. In addition, benchmarksrequiring OpenGL capabilities for their defaultexecutions (BT, BF, and ID) were executedwith the -qatest argument, in order to perform a“quick auto test”, which does not make use of thegraphics-oriented API. To make the original benchmarkcode compatible with rCUDA, which does notsupport the C for CUDA extensions, the pieces ofcode using these extensions were rewritten using theplain C API (only a 7% of the total effective sourcelines of code required being modified).The execution times reported in the next experimentsare the minimum from 5 executions, in orderto avoid eventual network and CPU noise. They reflectthe elapsed time experienced by the users, fromthe start of the execution of the application till theend of it. The experiments are presented in this sectionin two groups. First, those concerning one VMare presented. Later, we introduce experiments involvingseveral VMs being concurrently executed.A. Single Virtual MachineWe first analyze the performance of the CUDASDK benchmarks running in a VM using rCUDA,and compare their execution times with those of anative environment —i.e., using the regular CUDARuntime library in a non-virtualized environment.The results of this experiment are reported in Fig. 2.It would also be interesting including data for a versionof the benchmarks that makes only use of theCPU. However, as it is difficult to find optimized algorithmsfor CPUs performing the same operationsas all of our benchmarks, and those included in theSDK package are often naive versions, we cannotpresent such a comparison. Nevertheless, it is notstrictly required for understanding the experimentspresented and, additionally, the convenience of usingvirtualized remote GPUs instead of the local CPUwas previously discussed [11].JP2011-343