Lustre 1.6 Operations Manual

22.3.19 Drawbacks in Doing Multi-client O_APPEND
Writes
It is possible to do multi-client O_APPEND writes to a single file, but there are few
drawbacks that may make this a sub-optimal solution. These drawbacks are:
■
■
■
Each client needs to take an EOF lock on all the OSTs, as it is difficult to know
which OST holds the end of the file until you check all the OSTs. As all the clients
are using the same O_APPEND, there is significant locking overhead.
The second client cannot get all locks until the end of the writing of the first
client, as the taking serializes all writes from the clients.
To avoid deadlocks, the taking of these locks occurs in a known, consistent order.
As a client cannot know which OST holds the next piece of the file until the client
has locks on all OSTS, there is a need of these locks in case of a striped file.
22.3.20 Slowdown Occurs During Lustre Startup
When Lustre starts, the Lustre filesystem needs to read in data from the disk. For the
very first mdsrate run after the reboot, the MDS needs to wait on all the OSTs for
object precreation. This causes a slowdown to occur when Lustre starts up.
After the filesystem has been running for some time, it contains more data in cache
and hence, the variability caused by reading critical metadata from disk is mostly
eliminated. The filesystem now reads data from the cache.
22.3.21 Log Message ‘Out of Memory’ on OST
When planning the hardware for an OSS node, consider the memory usage of
several components in the Lustre system. If insufficient memory is available, an ‘out
of memory’ message can be logged.
During normal operation, several conditions indicate insufficient RAM on a server
node:
■ kernel "Out of memory" and/or "oom-killer" messages
■
■
Lustre "kmalloc of 'mmm' (NNNN bytes) failed..." messages
Lustre or kernel stack traces showing processes stuck in "try_to_free_pages"
For information on determining the MDS’s memory and OSS memory requirements,
see Memory Requirements.
Chapter 22 Lustre Troubleshooting Tips 22-17