Attacking Hypervisors via Firmware and Hardware

Attacking Hypervisorsvia Firmware and HardwareMikhail Gorobets, Oleksandr Bazhaniuk, Alex Matrosov,Andrew Furtak, Yuriy BulyginAdvanced Threat Research

AgendaHypervisor based isolationFirmware rootkit vs hypervisorAttacking hypervisor emulation of hardware devicesAttacking hypervisors through system firmwareTools and mitigationsConclusions

Image sourceHypervisor Based Isolation

Hypervisor Based IsolationVirtual MachineVirtual MachineAppAppAppAppOperating SystemOperating SystemPrivilegeVMM / HypervisorSystem Firmware(BIOS, U/EFI firmware, SMI handlers, Coreboot…)MemoryHardwareCPUGraphicsI/ONetwork

Hypervisor Based IsolationVirtual MachineVirtual MachineAppAppAppAttackOperating SystemOperating SystemPrivilegeVMM / HypervisorSystem Firmware(BIOS, U/EFI firmware, SMI handlers, Coreboot…)MemoryHardwareCPUGraphicsI/ONetwork

Hypervisor ProtectionsSoftware IsolationCPU / SoC: traps to hypervisor (VM Exits),MSR & I/O permissions bitmaps, rings (PV)…Memory / MMIO: hardware page tables (e.g.EPT, NPT), software shadow page tablesDevices IsolationCPU / SoC: interrupt remappingMemory / MMIO: IOMMU, No-DMA ranges

CPU Virtualization (simplified)VM Guest OSInstructions,exceptions,interrupts… Access to CPUMSRs(e.g. DEBUGCTL)Access toI/O portsAccess to (e.g. 0xB2)memory(EPT violations)Hypervisor Traps (VM Exits)VMM HostVM Exit HandlerVM ControlStructure (VMCS)MSR BitmapsExtended PageTablesI/O Bitmaps

Protecting Memory with HW Assisted PagingVM Guest OSVMM HostProcess VirtualMemoryCR3Guest PhysicalMemoryGPA0VMCSEPTPHost PhysicalMemoryHPA0VA0Guest Page TablesGPA1EPTHPA1VA1GPA2GPA0 HPA3HPA2VA2GPA3GPA2 HPA5HPA3VA3VA4…GPA4GPA5GPA6GPA4 HPA4(1:1 mapping)GPA6 blockHPA4HPA5HPA6……

Hypervisor ProtectionsSystem Firmware Isolation

Firmware Rootkitvs HypervisorImage source

What is firmware rootkit?Virtual MachineVirtual MachineAppAppAppAppOperating SystemOperating SystemPrivilegeVMM / HypervisorSystem FirmwareRootkit(e.g. DXE driver)MemoryHardwareCPUGraphicsI/ONetwork

Firmware rootkit can open a backdoor foran attacker VM to access all other VMsVirtual MachineApp AppOperating SystemAttacker VMApp AppOperating System3. Now using thisbackdoor, attacker VMcan access all ofmemory of victim VMsVMM / HypervisorSystem FirmwareBackdoorRootkit2. During each bootrootkit installs abackdoor for anattacker controlled VM1. At some pointsystem firmware gotinfected with a rootkitstaying persistent

“Backdoor” for attacker’s VM1. Firmware rootkitsearches & modifies VM’sVMCS(B), VMM page tables2. Rootkit added pagetable entries to attackerVM which expose entirephysical memoryNow attacker VM has fullaccess to physical memoryof VMM and other VMs

So how would one install a rootkit inthe firmware?

Using hardware SPI flash programmer…

USB & exploiting weak firmware protections...

Software access and exploiting somevulnerability in firmware …From privileged guest (e.g. Dom0). Requiresprivesc from normal guest (e.g. DomU) or remoteFrom the host OS before/in parallel to VMMFrom normal guest if firmware is exposed to theguest by VMMFor example, if firmware is not adequatelywrite protected in system flash memory

Image sourceDEMORootkit in System Firmware ExposesSecrets from Virtual Machines

Installing rootkit in firmware from root partition

Attacker VM exposes secrets of other VMsthrough a backdoor opened by the rootkit

We flashed rootkited part of firmware imagefrom within a root partition to install the rootkitThe system doesn’t properly protectfirmware in SPI flash memory so we couldbypass write-protectionFinally more systems protect firmware on theflash memorycommon.bios_wpCHIPSEC module to test write-protectionMalware can exploit vulnerabilities in firmwareto install a rootkit on such systemsAttacking and Defending BIOS in 2015

VMM “forensics”With the help of a rootkit in firmware any VM guestcan extract all information about hypervisor and otherVMs … and just from memory• VMCS structures, MSR and I/O bitmaps for each VM guest• EPT for each VM guest• Regular page tables for hypervisor and each VM guest• IOMMU pages tables for each IOMMU device• Full hypervisor memory map, VM exit handler…• Real hardware configuration (registers for real PCIe devices,MMIO contents…)

VMCS, MSR and I/O bitmaps..

VMM Hardware Page Tables…

Image sourceAttacking Hypervisor Emulation ofHardware Devices

XSA-75SYSRET…XSA-108…MS13-092XSA-122…CloudburstCVE-2014-0983…VENOMXSA-138…Hardware Emulation Attack VectorsVM Guest OSInstructions(CPUID…)Access toCPU MSRsHypercallAPIAccess todevice MMIO,CMD buffers…Access todevice I/OportsVMMHostHypervisorINSTREmulationCPU MSREmulationHypercallImplDeviceMMIO/BuffersEmulationDevice I/OEmulation

Did you know that VMMs emulate virtualdevices of other VMMs?So Cloudburst was fixed in VMWare but … QEMU andVirtualBox also emulate VMWare virtual SVGA deviceVirtual MachineAppAppOperating SystemVirtual sVGA DeviceSVGA_CMD_RECT_FILL…Host / HypervisorFrame buffersVGA commandsFIFO buffer

Guest to Host Memory CorruptionQEMU / KVMCVE-2014-36893 vulnerabilities in the vmware-vga driver in QEMU allows local guestto write to QEMU memory and gain host/hypervisor privileges viaunspecified parameters related to rectangle handlingOracle VirtualBox (Jan 2015 Critical Patch Update)CVE-2014-6588Memory corruption in VMSVGAGMRTRANSFERCVE-2014-6589, CVE-2014-6590Memory corruptions in VMSVGAFIFOLOOPCVE-2015-0427Integer overflow memory corruption in VMSVGAFIFOGETCMDBUFFER

Crashing Host or Guest from Ring3 …CVE-2015-0377Writing arbitrary data to upper 32 bits of IA32_APIC_BASE MSRcauses VMM and host OS to crash on Oracle VirtualBox 3.2,4.0.x-4.2.x# chipsec_util.py msr 0x1B 0xFEE00900 0xDEADBEEFCVE-2015-0418, CVE-2014-3646VirtualBox and KVM guest crash when executing INVEPT/INVVPIDinstructions in Ring3VirtualBoxINVEPT : VM crashINVVPID : VM crashVMCALL : #UD faultVMLAUNCH : #UD faultVMRESUME : #UD faultKVMINVEPT : VM crashINVVPID : VM crashVMCALL : No ExceptionVMLAUNCH : #UD faultVMRESUME : #UD fault

Attacking Hypervisors throughSystem Firmware(with OS kernel access)Image source

Pointer Vulnerabilities in SMI HandlersPhys MemoryRAX (code)RBX (pointer)RCX (function)RDXRSIRDISMISMI Handlers inSMRAMFake structure inside SMRAMOS MemoryExploit tricks SMI handler to write to an address inside SMRAMAttacking and Defending BIOS in 2015

SMI PointerExploiting firmware SMI handler to attack VMMVirtual Machine(child partition)AppAppOperating SystemRoot partitionApp AttackOperating SystemVMM allows VM toinvoke SMI handlers(grants access to SWSMI I/O port 0xB2)MemoryHypervisorSMI HandlersSystem FirmwareI/OHardwareCPUNetworkGraphicsCompromised VMinjects SMM payloadthrough the inputpointer vulnerability inSMI handlerSMM firmwarepayload modifieshypervisor code orVMCS/EPT to installa backdoor

DEMOAttacking Hypervisor viaPoisonous Pointers in Firmware SMI handlers

Root cause? Port B2h is open to VM in I/O bitmap

So that’s a firmware issue! Firmware hasto validate pointersPhys MemoryRAX (code)RBX (pointer)RCX (function)RDXRSIRDISMISMI Handlers inSMRAMHypervisor Memory(Protected by EPT)Firmware SMI handler validates input pointers to ensure theyare outside of SMRAM preventing overwrite of SMI code/data

Point SMI handler to overwrite VMM page!Phys MemoryRAX (code)RBX (pointer)SMISMI Handlers inSMRAMRCX (function)RDXRSIRDIHypervisor Memory(Protected by EPT)VMM Protected PageVMMProtectionsare OFF• VT state and EPT protections are OFF in SMM (without STM)• SMI handler writes to a protected page via supplied pointer

Attacking VMM by proxying through SMI handlerVirtual Machine(child partition)AppAppOperating SystemRoot partitionApp AttackOperating SystemVM with direct access toSMIs invokes SMIhandler and supplies apointer to some VMMpageVMM / HypervisorMemorySMI HandlersSystem FirmwareHardwareCPUGraphicsSMI handler writes tothe supplied pointeroverwriting contents ofprotected VMM pageI/ONetwork

Sometimes attacker doesn’t need avulnerability in firmware…When VMM grants VM direct access tofirmware or hardware interfacesVM exploit doesn’t always need to exploitfirmware first through these interfacesIt may use firmware or hardware as aconfused deputy and attack VMM throughsome function on behalf of firmwareRead excellent paper Hardware InvolvedSoftware Attacks by Jeff Forristal

Do Hypervisors Dreamof Electric Sheep?Vulnerability used in this section is VU#976132 a.k.a. S3 ResumeBoot Script Vulnerability independently discovered by ATR of IntelSecurity, Rafal Wojtczuk of Bromium and LegbaCoreIt’s also used in Thunderstrike 2 by LegbaCore & Trammell Hudson

NORMAL BOOTS3 RESUMEWaking the system from S3 “sleep” stateVirtual MachineApps / OSBDSVMM / HypervisorU/EFI System FirmwareDXEUEFI core& driversS3 BootScript TableRestoreshardware configScript EnginePlatform InitPlatform Init

What is S3 boot script table?A table of opcodes in physical memory which restoresplatform configurationS3_BOOTSCRIPT_MEM_WRITE opcode writes some value tospecified memory location on behalf of firmwareS3_BOOTSCRIPT_DISPATCH/2S3_BOOTSCRIPT_PCI_CONFIG_WRITES3_BOOTSCRIPT_IO_WRITE…

NORMAL BOOTMODIFYS3 RESUMEXen exposes S3 boot script table to Dom0Privileged PV guest (Dom0)ExploitVM modifies S3 bootscript table in memoryUpon resume, firmwareexecutes rogue S3 scriptXen HypervisorU/EFI System FirmwareBDSDXEUEFI core& driversS3 BootScript TableRestoreshardware configScript EnginePlatform PEI0xDBAA4000Platform PEI

Xen attack via S3 boot scriptFound S3 boot scripttable in memoryaccessible to Dom0Changing the bootscript to access Xenhypervisor pages

Dumping Dom0VMCS from memoryprotected by EPT

Image sourceDEMOAttacking Xenin its sleep

Déjà vu?Xen 0wning Trilogy (Part 3) by Invisible Things Lab

So these firmware vulnerabilities areexploitable from privileged guest (e.g. rootpartition, Dom0 ..)What about use cases where guests mustbe strongly isolated from the rootpartition?

Image sciencenews.orgTools and Mitigations

First things first - fix that firmware!Firmware can be tested for vulnerabilities!common.uefi.s3bootscript(tests S3 boot script protections)tools.smm.smm_ptr(tests for SMI pointer issues)Protect the firmware in system flash memorycommon.bios_wpcommon.spi_lock...(tests firmware protections in system flash memory)

Testing hypervisors…Simple hardware emulation fuzzingmodules for open source CHIPSECtools.vmm.*_fuzzI/O, MSR, PCIe device, MMIO overlap, more soon …Tools to explore VMM hardware configchipsec_util iommu (IOMMU)chipsec_util vm (CPU VM extensions)

Dealing with system firmware attacks..A number of interfaces through whichfirmware can be attacked or relay attack ontoVMM• UEFI variables, SMI handlers, S3 boot script, SPI flashMMIO, FW update..• FW doesn’t know memory VMM needs to protectVMM need to be careful with which of these itexposes to VMs including to administrative(privileged) guests• Some need not be exposed (e.g. S3 boot script), somemay be emulated and monitored

Conclusions• Compromised firmware is bad news for VMM.Test your system’s firmware for security issues• Windows 10 enables path for firmwaredeployment via Windows Update• Secure privileged/administrative guests; attacksfrom such guests are important• Vulnerabilities in device and CPU emulation arevery common. Fuzz all HW interfaces• Firmware interfaces/features may affecthypervisor security if exposed to VMs. Both needto be designed to be aware of each other

References1. CHIPSEC: https://github.com/chipsec/chipsec2. Intel’s ATR Security of System Firmware3. Attacking and Defending BIOS in 2015 by Intel ATR4. Hardware Involved Software Attacks by Jeff Forristal5. Xen 0wning Trilogy by Invisible Things Lab6. http://www.legbacore.com/Research.html7. Low level PC attack papers by Xeno Kovah

Thank You!