Lustre 1.6 Operations Manual

22.3.4 OSTs Become Read-Only
If the SCSI devices are inaccessible to Lustre at the block device level, then ext3
remounts the device
read-only to prevent filesystem corruption. This is a normal behavior. The status in
/proc/fs/lustre/healthcheck also shows "not healthy" on the affected nodes.
To recover from this problem, you must restart Lustre services using these
filesystems. There is no other way to know that the I/O made to disk, and the state
of the cache may be inconsistent with what is on disk.
22.3.5 Identifying a Missing OST
If an OST is missing for any reason, you may need to know what files are affected.
Although an OST is missing, the files system should be operational. From any
mounted client node, generate a list of files that reside on the affected OST. It is
advisable to mark the missing OST as ’unavailable’ so clients and the MDS do not
time out trying to contact it.
1. On MDS and client nodes, run:
# lctl dl
2. Deactivate the OST, run:
# lctl --device deactivate
Note that the device number will be different for the MDS and clients.
Note – If the OST later becomes available it needs to be reactivated, run:
# lctl --device activate
3. Determine all the files that are striped over the missing OST, run:
# lfs find -R -o {OST_UUID} /mountpoint
This returns a simple list of filenames from the affected filesystem.
4. If necessary, you can read the valid parts of a striped file, run:
# dd if=filename of=new_filename bs=4k conv=sync,noerror
22-6 Lustre 1.6 Operations Manual • September 2008