Lustre 1.6 Operations Manual

What is the Lustre data path?
On the OST, data is read directly from the disk into pre-allocated network I/O
buffers, in chunks up to 1 MB in size. This data is sent (zero-copy where possible) to
the clients, where it is put (again, zero-copy where possible) into the file's data
mapping. The clients maintain local writeback and readahead caches for Lustre.
On the OST, the filesystem metadata such as inodes, bitmaps and file allocation
information is cached in RAM (up to the maximum amount that the kernel allows).
No user data is currently cached on the OST.
In cases where only few files are read by many clients, it makes sense to use a RAID
device with a lot of local RAM cache so that the multiple read requests can skip the
disk access.
The networking code bundles up page requests into a maximum of 1 MB in a single
RPC to minimize overhead. In each client OSC, this is controlled by the
/proc/fs/lustre/osc/*/max_pages_per_rpc field. The size of the writeback
cache can be tuned via /proc/fs/lustre/osc/*/max_dirty_mb. The size of the
readahead can be tuned via /proc/fs/lustre/llite/max_read_ahead_mb.
Total client side cache usage can be limited via
/proc/fs/lustre/llite/max_cached_mb.
Questions about using Lustre quotas
This section covers various aspects of using Lustre quotas.
When I enable quotas with lfs quotaon, will it automatically set default quotas for
all users or do I have to set them for each user/group individually?
In that case, the default limit will be 0, which means no limit.
What happens if a user/group has already more files/disk usage than his quotas
allows?
Given that it will be 0 initially, no users will be over quotas. To preempt the next
question, if a user has a limit set that is less than his existing usage, he will simply
start to get -EDQUOT errors on subsequent attempts to write data.
We only want group quotas, do we have to enable user quotas as well?
We do not know of any particular failure if only group quotas are enabled, but the
more your use cases match our testing then the better off you will be.
For user quotas, even if you do not want to enforce limits, you can enable quotas
but not set any limits. Doing this makes future operation of enabling limits on
users easier (when/if you decide to) as usage will already be tracked and
accounted for (saving you the need to do that initial accounting). It also provides
you with a means to quickly assess how much space is being consumed on a userby-user
basis.
D-36 Lustre 1.6 Operations Manual • September 2008