Lustre 1.6 Operations Manual

25.1.2 Disadvantages of Striping
There are two disadvantages to striping which should deter you from choosing a
default policy that stripes over all OSTs unless you really need it: increased overhead
and increased risk.
25.1.2.1 Increased Overhead
Increased overhead comes in the form of extra network operations during common
operations such as stat and unlink, and more locks. Even when these operations are
performed in parallel, there is a big difference between doing 1 network operation
and 100 operations.
Increased overhead also comes in the form of server contention. Consider a cluster
with 100 clients and 100 OSSs, each with one OST. If each file has exactly one object
and the load is distributed evenly, there is no contention and the disks on each
server can manage sequential I/O. If each file has 100 objects, then the clients all
compete with one another for the attention of the servers, and the disks on each
node seek in 100 different directions. In this case, there is needless contention.
25.1.2.2 Increased Risk
Increased risk is evident when you consider the example of striping each file across
all servers. In this case, if any one OSS catches on-fire, a small part of every file is
lost. By comparison, if each file has exactly one stripe, you lose fewer files, but you
lose them in their entirety. Most users would rather lose some of their files entirely
than all of their files partially.
25.1.3 Stripe Size
Choosing a stripe size is a small balancing act, but there are reasonable defaults. The
stripe size must be a multiple of the page size. For safety, Lustre’s tools enforce a
multiple of 64 KB (the maximum page size on ia64 and PPC64 nodes), so users on
platforms with smaller pages do not accidentally create files which might cause
problems for ia64 clients.
Although you can create files with a stripe size of 64 KB, this is a poor choice.
Practically, the smallest recommended stripe size is 512 KB because Lustre sends 1
MB chunks over the network. This is a good amount of data to transfer at one time.
Choosing a smaller stripe size may hinder the batching.
Chapter 25 Striping and I/O Options 25-3