Slides (PDF, 279,8 KB)

Slides (PDF, 279,8 KB)

1 / 23Xen and the Art ofVirtualizationPaul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris,Alex Ho, Rolf Neugebauer, Ian Pratt, Andrew WarfieldDmitry Nedospasov01.06.10

2 / 23Goals of the ProjectA VMM which:1. has high performance,2. application binary compatibility,3. for which OSes can easily be ported,4. is capable of hosting 100(’s) VMs

3 / 23VMM Requirements1. VMs must be isolated from one another2. Support a variety of OSes3. Performance overhead should be minimized

Conventional ways to share resources◮ Simply use the OS protections⇒ doesn’t protect processes from impact of others◮ Computational Grids⇒ not when resources are oversubscribed or users areuncooperative◮ Full Virtualization⇒ VMs cannot directly access hardware⇒ Privileged instructions must be trapped◮ Retrofit support for protection (multiplexing)⇒ Xen does just that at an OS Level 11 Higher resource overhead4 / 23

5 / 23Design Principles1. Support for unmodified application Binaries2. Supporting full multi-application OSes3. Paravirtualization necessary for highperformance4. Hiding resource virtualization riskscorrectness and performance

6 / 23Design Aspectes◮ Guest OSes have to be modified to run at a lowerpriv. level, i.e. ring 1 instead of 0 on x86.◮ Privileged instructions are paravirtualized by requiringthem to be validated within Xen.◮ Exception handlers generally stay the same, exceptfor the Page Fault handler⇒ CR2 not accessible so we write to the stack frame◮ Only page faults and syscalls occur often enough toaffect performance⇒ a ‘fast’ exception handler is made available⇒ only pagefaults must be delivered via Xen◮ I/O via a clean set of device abstractions

7 / 23Porting EffortOS Subsection# lines of codeLinux XPArchitecture-independent 78 1299Virtual network driver 484 –Virtual block-device driver 1070 –Xen-specific (non-driver) 1363 3321Total 2995 4620Portion of total x86 code base 1.36% 0.04%

8 / 23Memory Management◮ x86 lacks a software-managed TLB - missesare costly! (Entering Xen is costly)⇒ Xen exists on top 64MB of every address space⇒ Guests allocate, manage HW PTs, Xen validates them◮ Guests can only map memory they own◮ Guests can batch update requests⇒ Further reduces overhead of entering Xen◮ Guests cannot install fully-privilegedsegments

9 / 23Control and ManagementThe initial domain (dom0) is created atboot-time and it,◮ performs the privileged HW instructions◮ Creates and terminates guest domains(domU)The control interface also provides interfacesfor:◮ creating Virtual Network Interfaces (VIFs)◮ creating Virtual Block Devices (VBDs)

10 / 23Control and Management cont. . .ControlPlaneSoftwareUserSoftwareUserSoftwareUserSoftwareGuestOS(XenoLinux)GuestOS(XenoLinux)GuestOS(XenoBSD)GuestOS(XenoXP)Xeno-AwareDevice DriversXeno-AwareDevice DriversXeno-AwareDevice DriversXeno-AwareDevice DriversDomain0controlinterfacevirtualx86 CPUvirtualphy memvirtualnetworkvirtualblockdevXENH/W (SMP x86, phy mem, enet, SCSI/IDE)

11 / 23Hypercalls and EventsdomX → Xen◮ synchronous◮ done via hypercallsXen → domX◮ asynchronous◮ lightweight notifications akin to Unix Signals

I/O RingsI/O virtualization (3)2 Zero-CopySystem Architecture Group,University of Karlsruhe212 / 2316

13 / 23CPU Scheduling◮ Xen uses Borrowed Virtual Time (BVT)⇒ both work-conserving and low-latency wake-update◮ Minimizes effect on OS subsystems thatmust run in a ‘timely’ fashion (i.e. TCP)◮ Temporarily violates ‘ideal’ fair sharing tofavor recently woken doms

14 / 23Time and TimersGuests are provided the notion of:1. real time⇒ nanoseconds passed since machine boot2. virtual time⇒ only advances while it’s executing (useful in scheduler)3. wall-clock time⇒ offset to be added to the current real time

15 / 23Virtual address translation1. VMware provides guests with a Virtual PTwhich it traps and validates⇒ requires explicit propagation of updates to ‘accessed’ and‘dirty’ bits2. Xen is only involved in PT updates⇒ Xen registers PTs directly with the MMU (ro for Guests)⇒ PT updates propagated via a hypercall (which is validated)

16 / 23Phys Mem, Network, DiskPhysical Memory:◮ Xen supports memory ballooning via a driver (easierto implement this way)◮ Xen doesn’t garuantee contiguous memoryNetwork:◮ Virtual FW Router (VFR) to which VIFs are attached◮ 2 I/O Rings (TX and RX), packets sent via bufferdescriptor (zero-copy)Disk:◮ Only dom0 has access to physical disks◮ domU’s VBD implemented via I/O ring (zero-copy)

Evaluation Foreword◮ ESX EULA doesn’t allow benchmarking 3Tested Platforms were:◮ Xen - Xenolinux (2.4.21) in domU◮ Linux (Native Linux)◮ VMWare Workstation◮ User Mode Linux (UML)3 Fail?17 / 23

Evalutation: Relative Performance18 / 23

19 / 23Evaluation: Microbenchmarksnull null opensict sig sig fork exec shConfig call I/O stat closeTCP inst hndl proc proc procL-SMP 0.53 0.81 2.10 3.51 23.2 0.83 2.94 143 601 4k2L-UP 0.45 0.50 1.28 1.92 5.70 0.68 2.49 110 530 4k0Xen 0.46 0.50 1.22 1.88 5.69 0.69 1.75 198 768 4k8VMW 0.73 0.83 1.88 2.99 11.1 1.02 4.63 874 2k3 10kUML 24.7 25.1 36.1 62.8 39.9 26.0 46.0 21k 33k 58klmbench: Processes - times in µs

20 / 23Evaluation: NetworkingTCP MTU 1500 TCP MTU 500TX RX TX RXLinux 897 897 602 544Xen 897 (-0%) 897 (-0%) 516 (-14%) 467 (-14%)VMW 291 (-68%) 615 (-31%) 101 (-83%) 137 (-75%)UML 165 (-82%) 203 (-77%) 61.1(-90%) 91.4(-83%)ttcp: Bandwidth in Mb/s

Evalutation: ScalabilityNormalised Throughput2. (50ms time slice)1.0XenoLinux (5ms time slice)0 10 20 30 40 50 60 70 80 90 100 110 120 130Concurrent Processes/DomainsFigure 6: Normalized aggregate performance of a subset ofSPEC CINT2000 running concurrently on 1-128 domains 21 / 23wAoplotomRwTgseEL

22 / 23Conclusion◮ Xen provides an execellent platform fordeploying network-centric services◮ Xen allows us to host and deploy servicesfor a short period of time◮ Performance is in the ballpark of a baselineLinux System

Questions? Pizza!23 / 23