Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical file system. A request for a unique directory in the hierarchical file system is received. The sub-file system of the hierarchical file system containing the unique directory is provided while ensuring that each directory resides in only one sub-file system. The system also maintains, in a top-file system, a corresponding directory and a symbolic pointer which points from the corresponding directory to a sub-file system where a given directory resides. Creation of new directories in sub-file systems also includes creation of corresponding directories in the top-file system and symbolic pointers from corresponding directory to new directory.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical set of file systems and maintaining a directory while providing the directory is capable of residing in more than one sub-file system within the hierarchical file system. If available space in a first sub-file system is below a threshold, the system identifies a second sub-file system with available space above the threshold, wherein the first sub-file system contains a first portion of the directory. If a second portion of the directory exists in the second sub-file system, the system creates or expands a file in the second portion of the directory, otherwise the system first creates the second portion of the directory in the second sub-file system and then creates a link from the first portion of the directory to the second portion of the directory.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical set of file systems and maintaining a corresponding directory in a top-file system and a number of symbolic pointers in the corresponding directory which each point from the corresponding directory to a sub-file system where any portion of the tracked directory resides. If available space in a first sub-file system is below a threshold, the system identifies a second sub-file system with available space above the threshold, wherein the first sub-file system contains a first portion of the directory. If a second portion of the directory does not exist in the second sub-file system, the system creates the second portion of the directory in the second sub-file system and creates a link from the first portion of the directory to the second portion of the directory.

Abstract: One embodiment provides a method for re-balancing data and metadata across multiple sub-file systems of a file system. The method includes determining sub-file systems including an amount of data that exceeds a threshold. At least one cell in the sub-file systems is identified as a candidate for re-balancing. A re-balance process is performed on the at least one cell that includes performing a flush operation to flush dirty data from file system buffers, copying an inode table for an independent set of files in the at least one cell to a destination sub-file system, notifying an allocation manager for the destination sub-file system of a new storage pool to manage, and performing an un-quiesce operation on I/O operations to each element in the at least one cell.

Abstract: One embodiment provides a method for file system namespace rebuilding. The method includes creating attribute data structures for a top-file system and sub-file system hierarchy system. The attribute data structures including hierarchy relationship information. The attribute data structures are stored in the sub-file systems. The top-file system namespace is rebuilt by extracting the hierarchy relationship information from an extended attribute of the attribute data structures in each stub of each sub-file system to build a table. The top-file system hierarchy is built one level at a time starting with the root directory having a parent of NULL.

Abstract: One embodiment provides a method for storage management in an aggregated file system. The method includes tracking inode numbers for sub-file systems of the aggregated file system. A request for a unique range of inode numbers in the aggregated file system is received. The unique range of inode numbers is provided to one or more sub-file systems of the aggregated file system while providing that each sub-file system of the aggregated file system has a required range of inode numbers and that no two sub-file systems have overlapping inode numbers.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical file system. A request for creation of an additional directory in the hierarchical file system is received. In response to the request, the additional directory is created in a sub-file system of the hierarchical file system while providing that each directory resides in only one sub-file system. If the additional directory is a child directory, the additional directory is created in the sub-file system containing the parent directory of the child directory.

Abstract: A method includes integrating a file system recovery log layer in a file system. The file system buffers data in a cyclical manner, and transforms all incoming random requests into a series of synchronous sequential updates. The method determines whether to flush a received write transaction to a recovery log that is stored in the file system recovery log layer. If it is determined to flush the received write transaction to the recovery log and the received write transaction is a first write transaction for writing data associated with the received write transaction to a file system block. The data associated with the received write transaction is appended in the recovery log and byte-ranges remaining in the file system block are recorded.

Abstract: Embodiments of the invention relate to small write performance enhancements for parallel file systems. One embodiment includes flushing a received write transaction from a first memory device to a second memory device on a node in a file system based on one or more byte-ranges within a file system block. It is determined to flush the received write transaction to a recovery log that is stored in a non-volatile storage layer of the file system. If it is determined to flush the received write transaction to the recovery log: appending data associated with the received write transaction in the recovery log of the file system, replicating the data associated with the received write transaction in another non-volatile storage layer of another node if required, and marking the one or more byte-ranges as committed.

Abstract: A method includes flushing a write transaction based on byte-ranges within a file system block. It is determined if the write transaction is a first write transaction to the file system block and whether to flush the received write transaction to a recovery log. If it is determined to flush the write transaction to the recovery log and the received write transaction is the first write transaction: the data associated with the write transaction is appended in the recovery log and byte-ranges remaining in the file system block are recorded, and also recording an associated particular bit value or particular bit pattern that the remaining byte-ranges must be set to before the remaining byte-ranges are accessed or when a recovery sequence for recovering data in the recovery log executes due to failure. The write transaction is replicated. The entire file system block byte-range is marked as committed.

Abstract: A system facilitates access to data in a network and includes a cache that stores instructions. A processor executes the instructions including: caching processing configured to integrate caching into a local cluster file system, and cache local file data in the cache based on fetching file data on demand from a remote cluster file system. The cache is visible to file system clients as a Portable Operating System Interface (POSIX) compliant file system. Applications execute on a multi-node cache cluster using POSIX semantics via a POSIX compliant file system interface. Data cache is locally and remotely consistent for updates.

Abstract: Embodiments of the invention relate to small write performance enhancements for parallel file systems. One embodiment includes flushing a received write transaction from a first memory device to a second memory device on a node in a file system based on one or more byte-ranges within a file system block. It is determined to flush the received write transaction to a recovery log that is stored in a non-volatile storage layer of the file system. If it is determined to flush the received write transaction to the recovery log: appending data associated with the received write transaction in the recovery log of the file system, replicating the data associated with the received write transaction in another non-volatile storage layer of another node if required, and marking the one or more byte-ranges as committed.

Abstract: A method for managing storage allocation includes adaptively determining, by a storage device processor, a region width across disk spaces for a group of storage devices that is inversely proportional to a number of nodes sharing a particular storage device. An adaptive storage device allocation region of the particular storage device is created based on the determined region width across the disk spaces for the group of storage devices.

Abstract: A system facilitates access to data in a network. One implementations includes a caching layer function that is configured to integrate into a local cluster file system, to cache local file data in a cache based on fetching file data on demand from a remote cluster file system, and to operate on a multi-node cache cluster. The cache is visible to file system clients as a Portable Operating System Interface (POSIX) compliant file system, applications execute on the multi-node cache cluster using POSIX semantics via a POSIX compliant file system interface, and data cache is locally consistent for updates made at the multi-node cache cluster using distributed locking for the data cache.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression for efficient random access to the data. A first partition of a first data block of a compression group is compressed. The first compressed partition is stored in a first compression entity. An in-memory table is maintained, which includes updating the in-memory table with data associated with an address of the stored compressed first partition. At such time as it is determined that the first compression entity is full, the in-memory table is compressed and written to the first compression entity. Accordingly, the in-memory table, which stores partition compression data, is store with the compression entity.

Abstract: A technique for restoring file systems by applying full and incremental backups together while streaming them from multiple devices provides a faster system restore from a full backup where incremental backups must also be applied. A first storage device streams the full backup while at least one second storage device streams at least one incremental backup. As files are detected in the incremental backup, they are written instead of the corresponding file in the full backup. Incremental backups can be pre-merged to reduce their number to one less than a number of storage devices available to stream the backups, so that the full backup can be streamed along with the pre-merged incremental backup(s) to restore the file system.

Abstract: Embodiments of the invention relate to supporting coordinated access to a clustered file system's shared disk storage subsystem by using dynamic creation of file access layout for different workloads and access patterns. Active file servers in the cluster have equivalent access to shared data of said file system. In one embodiment, a particular file server in the cluster is configured to coordinate the file system's layout. The particular file server, in response to a particular client's request for a layout of a file stored by said file system in said subsystem determines a file access layout for a parallel file access protocol to use when servicing an I/O request by said client, marks said layout as active for the client, and sends the layout to the client. The file access layout is determined based on one or more of current file system resources, a specific workload type and a specific byte-range of the file included in the client request, and properties of the file.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A first compression is performed on a first set of data to create first compressed data. The first compressed data is stored in one or more blocks of a first compression group. A size of free space of a last block of the first compression group is discovered and calculated. A second compression is performed on a second set of data to create second compressed data. At least a portion of the second compressed data is supplied to the first compression group for padding into the last block in response to determining that the size of the free space is sufficient. An unpadded portion of the second compressed data is stored in one or more blocks of a second compression group.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A compression is performed on a first set of data to create a first set of compressed data partitions in a compression group. A partition table is constructed, and partition entries for the first data block are added to the table in conjunction with the first set. A current size of the compression group is assessed. In response to a compression ratio being greater than a target compression ratio and internal fragmentation of the compression group being smaller than a threshold, the compression group is dynamically completed. The dynamic completion decides a size for the compression group. The partition table is added to the compression group by assigning space within the first compression group for the table. The compression group is written to persistent storage.

Abstract: A method for early compression related processing in a file system with offline compression. The method includes receiving a data file in a buffer. A processor detects that at least a portion of a data block of the data file resides in the buffer. A compressibility indication of the data block is determined based on performing at least one compressibility analysis operation on the data block. The compressibility indication of the data block is stored. A background compression task is performed on the data block based on: determining a compression decision for the data block based on the compressibility indication, and compressing the data block based on the compression decision.