One - The Linux Kernel Archives

Improving Kernel Performance by Unmapping the
Page Cache
James Bottomley
SteelEye Technology, Inc.
James.Bottomley@SteelEye.com
Abstract
The current DMA API is written on the founding
assumption that the coherency is being
done between the device and kernel virtual addresses.
We have a different API for coherency
between the kernel and userspace. The upshot
is that every Process I/O must be flushed twice:
Once to make the user coherent with the kernel
and once to make the kernel coherent with the
device. Additionally, having to map all pages
for I/O places considerable resource pressure
on x86 (where any highmem page must be separately
mapped).
We present a different paradigm: Assume that
by and large, read/write data is only required
by a single entity (the major consumers of large
multiply shared mappings are libraries, which
are read only) and optimise the I/O path for this
case. This means that any other shared consumers
of the data (including the kernel) must
separately map it themselves. The DMA API
would be changed to perform coherence to the
preferred address space (which could be the
kernel). This is a slight paradigm shift, because
now devices that need to peek at the data may
have to map it first. Further, to free up more
space for this mapping, we would break the assumption
that any page in ZONE_NORMAL
is automatically mapped into kernel space.
The benefits are that I/O goes straight from
the device into the user space (for processors
that have virtually indexed caches) and the kernel
has quite a large unmapped area for use in
kmapping highmem pages (for x86).
1 Introduction
In the Linux kernel 1 there are two addressing
spaces: memory physical which is the location
in the actual memory subsystem and CPU virtual,
which is an address the CPU’s Memory
Management Unit (MMU) translates to a memory
physical address internally. The Linux kernel
operates completely in CPU virtual space,
keeping separate virtual spaces for the kernel
and each of the current user processes. However,
the kernel also has to manage the mappings
between physical and virtual spaces, and
to do that it keeps track of where the physical
pages of memory currently are.
In the Linux kernel, memory is split into zones
in memory physical space:
• ZONE_DMA: A historical region where
ISA DMAable memory is allocated from.
On x86 this is all memory under 16MB.
• ZONE_NORMAL: This is where normally
allocated kernel memory goes. Where
1 This is not quite true, there are kernels for processors
without memory management units, but these are
very specialised and won’t be considered further