Abstract: A method includes integrating a file system recovery log layer in a file system. The file system buffers data in a cyclical manner, and transforms all incoming random requests into a series of synchronous sequential updates. The method determines a length value of a write transaction byte-range for a received write transaction. Upon the length value exceeding a threshold value, data for the write transaction byte-range is written to permanent storage in the file system. Upon the length value being less than the threshold value, the data for the write transaction byte-range is written to a recovery log that is stored in the file system recovery log layer. The threshold value is changed dynamically while the file system is mounted.

Abstract: One embodiment provides a method for storage management in a hierarchical file system that includes creating a structure for the hierarchical file system including a top-file system and sub-file systems that each include files and directories. Directories for the sub-file systems of the hierarchical file system are tracked. A request for creation of a unique directory in the hierarchical file system is received. Upon a request by a particular sub-file system for more inodes without any inodes available, inode numbers are revoked from at least one other sub-file system having unused inodes.

Abstract: An identification system that may be used in heterogeneous computing environments provides a fail-free path to providing identifiers from a single canonical namespace. Objects or gateways requiring an identifier for access are accessed using an identifier for the canonical namespace. If an entity requests access using an identifier from another namespace, an external database is consulted to determine if a mapping exists for the identifier to another identifier the canonical namespace. If no mapping exists, or the external database is unavailable, then an identifier is automatically generated in the canonical namespace and is used for the access. An internal database is updated with the automatically generated identifier, providing a mechanism to add mappings without administrative intervention. To access resources requiring an identifier from another particular namespace, a canonical namespace identifier may be mapped to another identifier in the particular namespace, or a generic identifier may be used.

Abstract: Embodiments relate to policy-based, multi-scheme data reduction for a computer memory. An aspect includes receiving a plurality of policy rules by a policy engine of the computer memory, wherein a first policy rule specifies applying a first data reduction scheme to data in the computer memory based on the data matching first characteristics, wherein a second policy rule specifies applying a second data reduction scheme to data in the computer memory based on the data matching second characteristics, wherein the first data reduction scheme is different from the second data reduction scheme. Another aspect includes determining, by the policy engine, that first data in the computer memory matches the first characteristics, and that second data in the computer memory matches the second characteristics. Yet another aspect includes applying the first data reduction scheme to the first data, and applying the second data reduction scheme to the second data.

Abstract: A computer-implemented method according to one embodiment includes identifying one or more accessible server nodes within a plurality of nodes of a failed clustered system, retrieving a plurality of fragments of configuration data from the one or more accessible server nodes, and constructing a backup state for the failed clustered system, utilizing the plurality of fragments of the configuration data.

Abstract: A method, computer system, and a computer program product for enhanced application write operations is provided. The present invention may include performing a write operation by an application node. The present invention may then include committing the write operation to a local buffer cache. The present invention may then include sending the application node an indication that the write operation was successful. The present invention may then include flushing the local buffer cache to a disk based on a buffer flush trigger, and the present invention may lastly include sending a Remote Procedure Call (RPC) to at least one gateway node, wherein functionality of the gateway node is located on different physical nodes which are separated from the application node over the network.

Abstract: Embodiments of the invention relate to small write performance enhancements for parallel file systems. One embodiment includes flushing a received write transaction from a first memory device to a second memory device on a node in a file system based on one or more byte-ranges within a file system block. It is determined to flush the received write transaction to a recovery log that is stored in a non-volatile storage layer of the file system. If it is determined to flush the received write transaction to the recovery log: appending data associated with the received write transaction in the recovery log of the file system, replicating the data associated with the received write transaction in another non-volatile storage layer of another node if required, and marking the one or more byte-ranges as committed.

Abstract: A storage system in a clustered system may receive a first input/output (I/O) request. The storage system may include one or more storage nodes. Each of the one or more storage nodes may have a copy of a particular object stored thereon. The storage system may execute the first I/O request. Executing the first I/O request may modify data of a first object in a first storage node. The first object may be a copy of the particular object. The storage system may transfer the modified data of the first object to a master storage node. The master storage node may include a master object update descriptor list.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A first compression is performed on a first set of data to create first compressed data. The first compressed data is stored in one or more blocks of a first compression group. A size of free space of a last block of the first compression group is discovered and calculated. A second compression is performed on a second set of data to create second compressed data. At least a portion of the second compressed data is supplied to the first compression group for padding into the last block in response to determining that the size of the free space is sufficient. An unpadded portion of the second compressed data is stored in one or more blocks of a second compression group.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A compression is performed on a first set of data to create a first set of compressed data partitions in a compression group. A partition table is constructed, and partition entries for the first data block are added to the table in conjunction with the first set. A current size of the compression group is assessed. In response to a compression ratio being greater than a target compression ratio and internal fragmentation of the compression group being smaller than a threshold, the compression group is dynamically completed. The dynamic completion decides a size for the compression group. The partition table is added to the compression group by assigning space within the first compression group for the table. The compression group is written to persistent storage.

Abstract: The embodiments described herein relate to synchronization of data in a shared pool of configurable computer resources. One or more consistency points are created in a source filesystem. A first consistency point is compared with a second consistency point to detect a directory change at the source filesystem, which includes identifying at least one difference between the first and second consistency points. A file level change associated with an established directory at a target filesystem is identified responsive to the detection of the directory change. A link is established between the source filesystem and the target filesystem, and the established directory is updated based on the file level change.

Abstract: A system facilitates access to data in a network and includes a cache that stores instructions. A processor executes the instructions including: caching processing configured to integrate caching into a local cluster file system, and cache local file data in the cache based on fetching file data on demand from a remote cluster file system. The cache is visible to file system clients as a Portable Operating System Interface (POSIX) compliant file system. Applications execute on a multi-node cache cluster using POSIX semantics via a POSIX compliant file system interface. Data cache is locally and remotely consistent for updates.

Abstract: One embodiment provides a method for re-balancing data and metadata across multiple sub-file systems of a file system. The method includes determining sub-file systems including an amount of data that exceeds a threshold. At least one cell in the sub-file systems is identified as a candidate for re-balancing. A re-balance process is performed on the at least one cell that includes performing a flush operation to flush dirty data from file system buffers, copying an inode table for an independent set of files in the at least one cell to a destination sub-file system, notifying an allocation manager for the destination sub-file system of a new storage pool to manage, and performing an un-quiesce operation on I/O operations to each element in the at least one cell.

Abstract: One embodiment provides a method for storage management in a hierarchical file system that includes creating a structure for the hierarchical file system including a top-file system and sub-file systems that each include files and directories. Directories for the sub-file systems of the hierarchical file system are tracked. A request for creation of a unique directory in the hierarchical file system is received. Upon a request by a particular sub-file system for more inodes without any inodes available, inode numbers are revoked from at least one other sub-file system having unused inodes.

Abstract: One embodiment provides a method for storage management in an aggregated file system. The method includes tracking inode numbers for sub-file systems of the aggregated file system. A request for a unique range of inode numbers in the aggregated file system is received. The unique range of inode numbers is provided to one or more sub-file systems of the aggregated file system while providing that each sub-file system of the aggregated file system has a required range of inode numbers and that no two sub-file systems have overlapping inode numbers.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical set of file systems and maintaining a corresponding directory in a top-file system and a number of symbolic pointers in the corresponding directory which each point from the corresponding directory to a sub-file system where any portion of the tracked directory resides. If available space in a first sub-file system is below a threshold, the system identifies a second sub-file system with available space above the threshold, wherein the first sub-file system contains a first portion of the directory. If a second portion of the directory does not exist in the second sub-file system, the system creates the second portion of the directory in the second sub-file system and creates a link from the first portion of the directory to the second portion of the directory.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical file system. A request for a unique directory in the hierarchical file system is received. The sub-file system of the hierarchical file system containing the unique directory is provided while ensuring that each directory resides in only one sub-file system. The system also maintains, in a top-file system, a corresponding directory and a symbolic pointer which points from the corresponding directory to a sub-file system where a given directory resides. Creation of new directories in sub-file systems also includes creation of corresponding directories in the top-file system and symbolic pointers from corresponding directory to new directory.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical file system. A request for creation of an additional directory in the hierarchical file system is received. In response to the request, the additional directory is created in a sub-file system of the hierarchical file system while providing that each directory resides in only one sub-file system. If the additional directory is a child directory, the additional directory is created in the sub-file system containing the parent directory of the child directory.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical set of file systems and maintaining a directory while providing the directory is capable of residing in more than one sub-file system within the hierarchical file system. If available space in a first sub-file system is below a threshold, the system identifies a second sub-file system with available space above the threshold, wherein the first sub-file system contains a first portion of the directory. If a second portion of the directory exists in the second sub-file system, the system creates or expands a file in the second portion of the directory, otherwise the system first creates the second portion of the directory in the second sub-file system and then creates a link from the first portion of the directory to the second portion of the directory.

Abstract: An identification system that may be used in heterogeneous computing environments provides a fail-free path to providing identifiers from a single canonical namespace. Objects or gateways requiring an identifier for access are accessed using an identifier for the canonical namespace. If an entity requests access using an identifier from another namespace, an external database is consulted to determine if a mapping exists for the identifier to another identifier the canonical namespace. If no mapping exists, or the external database is unavailable, then an identifier is automatically generated in the canonical namespace and is used for the access. An internal database is updated with the automatically generated identifier, providing a mechanism to add mappings without administrative intervention. To access resources requiring an identifier from another particular namespace, a canonical namespace identifier may be mapped to another identifier in the particular namespace, or a generic identifier may be used.