Actas JP2011 - Universidad de La Laguna

Actas XXII Jornadas de Paralelismo (JP2011) , La Laguna, Tenerife, 7-9 septiembre 20113HashCRUSHFig. 1.Directory object migration.Fig. 2. Access to a directory containing a symbolic link./usr/new is a soft link to /usr/old.missions and its parent’s path permissions). UnlikeLH, only directories have dual–entry ACLs in FPFS.A file’s permissions are derived from its ACL, andits directory’s dual–entry ACL.When checking permissions, the OSD+ containingthe target directory object searches in the metadatalog for invalidations along the requested objectpath. If they exist, the parent directory is accessedto get its dual–entry ACL. Once permissions are updated,the requested ACL is calculated. Since parent’spermissions might also be invalid, this processis repeated recursively until the changed directory isreached, or an updated directory is reached. Permissionsare updated in a lazy fashion, minimizing thepart of the hierarchy traversed.D. LinksPlacing directories by hashing their pathnamespresents the problem of locating the correct OSD+sfor paths that include symbolic links. Any access toa subtree of the linked directory hierarchy will fail, asit happens with a renamed directory whose objectshave not been migrated.LH [6] proposes the creation of shortcuts to dealwith files whose pathnames contains symbolic links.A shortcut to one of these files is created the firsttime the file is accessed by traversing the directoryhierarchy. Any subsequent access to the same filewill use the shortcut. But this approach presentstwo problems: shortcuts take up space and, moreimportantly, when the access to a file fails, there isno way to know if the failure is due to a missingfile or the existence of symbolic links in the name.This ambiguity always produces the traversal of thedirectory hierarchy up to the root directory whenaccessing to any missing file.Our proposal for symbolic links does not suffer themissing file problem of LH. In FPFS, a symbolic linkis treated as a directory rename. The differences arethat: any access to a directory containing a symboliclink never produces the migration of the directory,and a client accessing one of these directories receivesthe resolved path to contact with the original OSD+(see Fig. 2).E. AtomicityAn important aspect is that all the metadata operationsmust be atomic to provide a coherent view.When a metadata operation is performed by a singleOSD+ (e.g., create, unlink, etc.), the backend filesystem itself guarantees its atomicity, POSIX semantics,and atomicity when clients access in parallel.However, there are operations, such as rename,mkdir or rmdir, which usually involve two OSD+s.Now, atomicity is guaranteed by means of the backendfile system, and a three–phase commit protocol(3PC) [13] where one node acts as the coordinatordirecting the remaining nodes or participants.IV. The Metadata Cluster: ImplementationA prototype of the metadata cluster has been builton a Linux environment. Each OSD+ is a user–space multithreaded process, running on a mainstreamcomputer, which uses a conventional file systemas storage backend. The Linux syscall interfaceis used to access the backend file system, whichmust be POSIX–compliant and support extended attributes.Our FPFS implementation not only exploitsthe underlying file system’s features (atomicity,errors checking, etc.), but also exports them tothe parallel file system.For every new established connection from anFPFS client or another OSD+, one thread islaunched. It lasts as long as the communicationchannel remains open; hence, performance is improveddue to the absence of connection establishmentand termination handshakes, associated witheach message. In the current implementation, connectionsuse TCP/IP and UDP/IP protocols.A. Directory ObjectsInternally, a directory object is implemented as aregular directory whose path is its directory pathnamein the parallel file system. Thus, the directoryhierarchy is imported within each OSD+ by replicatinga partial namespace of the global hierarchy.To preserve the hierarchy, directory objects maintainan entry for every file and subdirectory theycontain. Hence, several types of directories are differentiatedthrough extended attributes: a first type toimplement directory objects; a second one to maintainthe hierarchy (e.g., the subdirectories); a thirdto internally construct the paths of the directoriesobjects; and finally, temporal directories to keep renamedmetadata which has not been migrated yet.JP2011-553