23.07.2014 Views

Lustre 1.6 Operations Manual

Lustre 1.6 Operations Manual

Lustre 1.6 Operations Manual

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

22.3.3 OST Object is Missing or Damaged<br />

If the OSS fails to find an object or finds a damaged object, this message appears:<br />

OST object missing or damaged (OST "ost1", object 98148, error -2)<br />

If the reported error is -2 (-ENOENT, or "No such file or directory"), then the object<br />

is missing. This can occur either because the MDS and OST are out of sync, or<br />

because an OST object was corrupted and deleted.<br />

If you have recovered the filesystem from a disk failure by using e2fsck, then<br />

unrecoverable objects may have been deleted or moved to /lost+found on the raw<br />

OST partition. Because files on the MDS still reference these objects, attempts to<br />

access them produce this error.<br />

If you have recovered a backup of the raw MDS or OST partition, then the restored<br />

partition is very likely to be out of sync with the rest of your cluster. No matter<br />

which server partition you restored from backup, files on the MDS may reference<br />

objects which no longer exist (or did not exist when the backup was taken);<br />

accessing those files produces this error.<br />

If neither of those descriptions is applicable to your situation, then it is possible that<br />

you have discovered a programming error that allowed the servers to get out of<br />

sync. Please report this condition to the <strong>Lustre</strong> group, and we will investigate.<br />

If the reported error is anything else (such as -5, "I/O error"), it likely indicates a<br />

storage failure. The low-level filesystem returns this error if it is unable to read from<br />

the storage device.<br />

Suggested Action<br />

If the reported error is -2, you can consider checking in /lost+found on your raw<br />

OST device, to see if the missing object is there. However, it is likely that this object<br />

is lost forever, and that the file that references the object is now partially or<br />

completely lost. Restore this file from backup, or salvage what you can and delete it.<br />

If the reported error is anything else, then you should immediately inspect this<br />

server for storage problems.<br />

Chapter 22 <strong>Lustre</strong> Troubleshooting Tips 22-5

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!