Lustre 1.6 Operations Manual

21.5.3 Setting Write-Back Cache
Performance is noticeably improved by running Lustre with write-back cache turned
on. However, there is a risk that when the DDN controller crashes you need to run
e2fsck. Still, it takes less time than the performance hit from running with the writeback
cache turned off.
For increased data security and in failover configurations, you may prefer to run
with write-back cache off. However, you might experience performance problems
with the small writes during journal flush. In this mode, it is highly beneficial to
increase the number of OST service threads options ost ost_num_threads=512
in /etc/modprobe.conf. The OST should have enough RAM (about 1.5 MB
/thread is preallocated for I/O buffers). Having more I/O threads allows you to
have more I/O requests in flight, waiting for the disk to complete the synchronous
write.
You have to decide whether performance is more important than the slight risk of
data loss and downtime in case of a hardware/software problem on the DDN.
Note – There is no risk from an OSS/MDS node crashing, only if the DDN itself
fails.
21.5.4 Setting maxcmds
For S2A DDN 8500 array, changing maxcmds to 4 (from the default 2) improved
write performance by as much as 30% in a particular case. This only works with
SATA-based disks and when only one controller of the pair is actually accessing the
shared LUNs.
However, this setting comes with a warning. DDN support does not recommend
changing this setting from the default. By increasing the value to 5, the same setup
experienced some serious problems.
The CLI command for the DDN client is provided below (default value is 2).
$ diskmaxcmds=3
For S2A DDN 9500/9550 hardware, you can safely change the default from 6 to 16.
Although the maximum value is 32, values higher than 16 are not currently
recommended by DDN support.
21-8 Lustre 1.6 Operations Manual • September 2008