Lustre 1.6 Operations Manual

27.1 Expanding the Filesystem by Adding
OSTs
To add OSTs to existing Lustre filesystems:
1. Add a new OST by passing on the following commands, run:
$ mkfs.lustre --fsname=spfs --ost --mgsnode=mds16@tcp0 /dev/sda
$ mkdir -p /mnt/test/ost0
$ mount -t lustre /dev/sda /mnt/test/ost0
2. Migrate the data (possibly).
The filesystem is quite unbalanced when new empty OSTs are added. New file
creations are automatically balanced. If this is a scratch filesystem or files are
pruned at a regular interval, then no further work may be needed. Files existing
prior to the expansion can be rebalanced with an in-place copy, which can be
done with a simple script.
The basic method is to copy existing files to a temporary file, then move the temp
file over the old one. This should not be attempted with files which are currently
being written to by users or applications. This operation redistributes the stripes
over the entire set of OSTs. For a sample data migration script, see A Simple Data
Migration Script.
A very clever migration script would do the following:
■
■
■
■
Examine the current distribution of data.
Calculate how much data should move from each full OST to the empty ones.
Search for files on a given full OST (using lfs getstripe).
Force the new destination OST (using lfs setstripe).
■ Copy only enough files to address the imbalance.
If a Lustre administrator wants to explore this approach further, per-OST diskusage
statistics can be found under /proc/fs/lustre/osc/*/rpc_stats
27-2 Lustre 1.6 Operations Manual • September 2008