Abstract: Embodiments of the invention relate to coordinated access to a clustered file system's shared storage subsystem using a shared-lock architecture. A particular file server is configured to coordinate the file system's file access layout particular server, in response to a client's file access layout request for byte-ranges of a file stored in the storage subsystem being received by a file server in cluster and sent to said particular server, generates a file access layout of a set of file servers for a parallel file access protocol to use and byte ranges of the file that particular individual file servers to use, to service an I/O request for the file. The file server (that received the request), sends the generated file access layout to said client, and requests coordinated access to said file from the particular server in response to receiving an I/O request to byte-ranges of said file.

Abstract: Embodiments relate to granular management of data storage blocks in a data storage system. In one aspect, status values are employed to track “used”, “free”, and “claimed free” storage blocks. A storage block having stored data is identified as used, a storage block available to store data is identified as free, and a storage block having previously stored data removed that has not been reclaimed is identified as claimed free. These values are maintained on a map to track each data block within the data storage system. One or more used blocks are de-allocated, which includes changing the status value of each used block to claimed free Available claimed free data blocks are prioritized for data block allocation over available free data blocks for efficient storage, including enabling efficient reclamation of data blocks and minimizing data movement needed for reclamation-oriented de-fragmentation.

Abstract: A method for managing storage allocation includes adaptively determining, by a storage device processor, a region width across disk spaces for a group of storage devices that is inversely proportional to a number of nodes sharing a particular storage device. An adaptive storage device allocation region of the particular storage device is created based on the determined region width across the disk spaces for the group of storage devices.

Abstract: Embodiments relate to policy-based, multi-scheme data reduction for a computer memory. An aspect includes receiving a plurality of policy rules by a policy engine of the computer memory, wherein a first policy rule specifies applying a first data reduction scheme to data in the computer memory based on the data matching first characteristics, wherein a second policy rule specifies applying a second data reduction scheme to data in the computer memory based on the data matching second characteristics, wherein the first data reduction scheme is different from the second data reduction scheme. Another aspect includes determining, by the policy engine, that first data in the computer memory matches the first characteristics, and that second data in the computer memory matches the second characteristics. Yet another aspect includes applying the first data reduction scheme to the first data, and applying the second data reduction scheme to the second data.

Abstract: Embodiments of the invention relate to storage allocation in a storage system. One embodiment includes adaptively determining, by a storage device processor, a region height across disk spaces of a group of storage devices that is proportional to a number of connected storage devices. The storage device processor adaptively determines a region width across the disk spaces for the group of storage devices that is inversely proportional to a number of nodes sharing a particular storage device.

Abstract: An identification system that may be used in heterogeneous computing environments provides a fail-free path to providing identifiers from a single canonical namespace. Objects or gateways requiring an identifier for access are accessed using an identifier for the canonical namespace. If an entity requests access using an identifier from another namespace, an external database is consulted to determine if a mapping exists for the identifier to another identifier the canonical namespace. If no mapping exists, or the external database is unavailable, then an identifier is automatically generated in the canonical namespace and is used for the access. An internal database is updated with the automatically generated identifier, providing a mechanism to add mappings without administrative intervention. To access resources requiring an identifier from another particular namespace, a canonical namespace identifier may be mapped to another identifier in the particular namespace, or a generic identifier may be used.

Abstract: A technique for restoring file systems by applying full and incremental backups together while streaming them from multiple devices provides a faster system restore from a full backup where incremental backups must also be applied. A first storage device streams the full backup while at least one second storage device streams at least one incremental backup. As files are detected in the incremental backup, they are written instead of the corresponding file in the full backup. Incremental backups can be pre-merged to reduce their number to one less than a number of storage devices available to stream the backups, so that the full backup can be streamed along with the pre-merged incremental backup(s) to restore the file system.

Abstract: A technique for restoring file systems by applying full and incremental backups together while streaming them from multiple devices provides a faster system restore from a full backup where incremental backups must also be applied. A first storage device streams the full backup while at least one second storage device streams at least one incremental backup. As files are detected in the incremental backup, they are written instead of the corresponding file in the full backup. Incremental backups can be pre-merged to reduce their number to one less than a number of storage devices available to stream the backups, so that the full backup can be streamed along with the pre-merged incremental backup(s) to restore the file system.

Abstract: An identification system that may be used in heterogeneous computing environments provides a fail-free path to providing identifiers from a single canonical namespace. Objects or gateways requiring an identifier for access are accessed using an identifier for the canonical namespace. If an entity requests access using an identifier from another namespace, an external database is consulted to determine if a mapping exists for the identifier to another identifier the canonical namespace. If no mapping exists, or the external database is unavailable, then an identifier is automatically generated in the canonical namespace and is used for the access. An internal database is updated with the automatically generated identifier, providing a mechanism to add mappings without administrative intervention. To access resources requiring an identifier from another particular namespace, a canonical namespace identifier may be mapped to another identifier in the particular namespace, or a generic identifier may be used.

Abstract: The embodiments described herein relate to synchronization of data in a shared pool of configurable computer resources. One or more consistency points are created in a source filesystem. A first consistency point is compared with a second consistency point to detect a directory change at the source filesystem, which includes identifying at least one difference between the first and second consistency points. A file level change associated with an established directory at a target filesystem is identified responsive to the detection of the directory change. A link is established between the source filesystem and the target filesystem, and the established directory is updated based on the file level change.

Abstract: Embodiments of the invention relate to small write performance enhancements for parallel file systems. One embodiment includes flushing a received write transaction from a first memory device to a second memory device on a node in a file system based on one or more byte-ranges within a file system block. It is determined to flush the received write transaction to a recovery log that is stored in a non-volatile storage layer of the file system. If it is determined to flush the received write transaction to the recovery log: appending data associated with the received write transaction in the recovery log of the file system, replicating the data associated with the received write transaction in another non-volatile storage layer of another node if required, and marking the one or more byte-ranges as committed.

Abstract: A method includes flushing a write transaction based on byte-ranges within a file system block. It is determined if the write transaction is a first write transaction to the file system block and whether to flush the received write transaction to a recovery log. If it is determined to flush the write transaction to the recovery log and the received write transaction is the first write transaction: the data associated with the write transaction is appended in the recovery log and byte-ranges remaining in the file system block are recorded, and also recording an associated particular bit value or particular bit pattern that the remaining byte-ranges must be set to before the remaining byte-ranges are accessed or when a recovery sequence for recovering data in the recovery log executes due to failure. The write transaction is replicated. The entire file system block byte-range is marked as committed.

Abstract: Aspects of the invention are provided to support partial file caching on a file system block boundary. All read requests are converted so that offset and count are aligned on a block boundary. Data associated with read requests is first satisfied from local cache, with cache misses supported with a call to persistent or remote system. Similarly, for a write request, any partial blocks are aligned to the block boundary. Data associated with the write request is performed on local cache and placed in a queue for replay to the persistent or remote system.

Abstract: Embodiments relate to granular management of data storage blocks in a data storage system. In one aspect, status values are employed to track “used”, “free”, and “claimed free” storage blocks. A storage block having stored data is identified as used, a storage block available to store data is identified as free, and a storage block having previously stored data removed that has not been reclaimed is identified as claimed free. These values are maintained on a map to track each data block within the data storage system. One or more used blocks are de-allocated, which includes changing the status value of each used block to claimed free Available claimed free data blocks are prioritized for data block allocation over available free data blocks for efficient storage, including enabling efficient reclamation of data blocks and minimizing data movement needed for reclamation-oriented de-fragmentation.

Abstract: Embodiments of the invention relate to synchronization of data in a shared pool of configurable computer resources. An image of the filesystem changes, including data and metadata, is captured in the form of a consistency point. Sequential consistency points are created, with changes to data and metadata in the filesystem between sequential consistency captured and placed in a queue for communication to a target filesystem at a target site. The changes are communicated as a filesystem operation, with the communication limited to the changes captured and reflected in the consistency point.

Abstract: A system facilitates access to data in a network. One implementations includes a caching layer function that is configured to integrate into a local cluster file system, to cache local file data in a cache based on fetching file data on demand from a remote cluster file system, and to operate on a multi-node cache cluster.

Abstract: Embodiments of the inventions relate to granular management of data storage blocks in a data storage system. In one aspect, status values are employed to track “used”, “free”, and “claimed free” storage blocks. A storage block having stored data is identified as used, a storage block available to store data is identified as free, and a storage block having previously stored data removed that has not been reclaimed is identified as claimed free. These values are maintained on a map to track each data block within the data storage system. Available claimed free data blocks are prioritized for data block allocation over available free data blocks for efficient storage, including enabling efficient reclamation of data blocks and minimizing data movement needed for reclamation-oriented de-fragmentation.

Abstract: Aspects of the invention are provided to support partial file caching on a file system block boundary. All read requests are converted so that offset and count are aligned on a block boundary. Data associated with read requests is first satisfied from local cache, with cache misses supported with a call to persistent or remote system. Similarly, for a write request, any partial blocks are aligned to the block boundary. Data associated with the write request is performed on local cache and placed in a queue for replay to the persistent or remote system.

Abstract: Scalable caching of remote file data in cluster file systems is provided. One implementation involves maintaining a cache in a local cluster file system and caching local file data in the cache by fetching file data on demand from the remote cluster file system into the local cached file system over the network. The local file data and metadata corresponds to the remote file data and metadata in the remote cluster file system. Updates made to the local file data and metadata are pushed back to the remote cluster file system asynchronously.

Abstract: Scalable caching of remote file data in cluster file systems is provided. One implementation involves maintaining a cache in a local cluster file system and caching local file data in the cache by fetching file data on demand from the remote cluster file system into the local cached file system over the network. The local file data and metadata corresponds to the remote file data and metadata in the remote cluster file system. Updates made to the local file data and metadata are pushed back to the remote cluster file system asynchronously.