Lustre 1.6 Operations Manual

22.3 Lustre Performance Tips
This section describes various tips to improve Lustre performance.
22.3.1 Setting SCSI I/O Sizes
Some SCSI drivers default to a maximum I/O size that is too small for good Lustre
performance. we have fixed quite a few drivers, but you may still find that some
drivers give unsatisfactory performance with Lustre. As the default value is hardcoded,
you need to recompile the drivers to change their default. On the other hand,
some drivers may have a wrong default set.
If you suspect bad I/O performance and an analysis of Lustre statistics indicates that
I/O is not 1 MB, check /sys/block//queue/max_sectors_kb. If it is
less than 1024, set it to 1024 to improve the performance. If changing this setting
does not change the I/O size as reported by Lustre, you may want to examine the
SCSI driver code.
22.3.2 Write Performance Better Than Read Performance
Typically, the performance of write operations on a Lustre cluster is better than read
operations. When doing writes, all clients are sending write RPCs asynchronously.
The RPCs are allocated, and written to disk in the order they arrive. In many cases,
this allows the back-end storage to aggregate writes efficiently.
In the case of read operations, the reads from clients may come in a different order
and need a lot of seeking to get read from the disk. This noticeably hampers the read
throughput.
Currently, there is no readahead on the OSTs themselves, though the clients do
readahead. If there are lots of clients doing reads it would not be possible to do any
readahead in any case because of memory consumption (consider that even a single
RPC (1 MB) readahead for 1000 clients would consume 1 GB of RAM).
For filesystems that use socklnd (TCP, Ethernet) as interconnect, there is also
additional CPU overhead because the client cannot receive data without copying it
from the network buffers. In the write case, the client CAN send data without the
additional data copy. This means that the client is more likely to become CPU-bound
during reads than writes.
22-4 Lustre 1.6 Operations Manual • September 2008