Abstract: A method and apparatus is disclosed herein for maintaining deduplication metadata on disk separate from an in-memory deduplication file system for a deduplication computing system. The method may include reading deduplication metadata from a first memory of a deduplication computing system. Furthermore, the method may include maintaining deduplicated data for a file system in a second memory, and maintaining the deduplication metadata that describes the deduplicated data of the file system in the first memory of the deduplication computing system, wherein the first memory is a different type of memory and separated from the second memory in the deduplication computing system. Furthermore, the method may include accessing the deduplication metadata in the second memory in response to receipt of a request by the deduplication computing system to perform a first operation with deduplicated data of the file system in the second memory.

Abstract: A method and apparatus is disclosed herein for checking the consistency of deduplication metadata after an unclean shutdown of a deduplication computing system. The method may include mounting a deduplication file system of a deduplication computing system after an unclean shutdown of the deduplication computing system. Furthermore, the method may include initiating a reconstruction and consistency check of deduplication metadata associated with the deduplication file system. Furthermore, the method may include receiving a user operation prior to completion of the reconstruction and consistency check of the deduplication metadata, and processing the user operation in parallel with the reconstruction and consistency check of the deduplication metadata.

Abstract: A method and apparatus is disclosed herein for maintaining data consistency in persistent memory in an in-place-update file system of a deduplication computing system. The method may include receiving a data write request to write a block of data from a volatile memory to a non-volatile memory in an in-place-update file system of deduplication computing system. Furthermore, the method may include determining whether the data write request is to perform an in-place update of an existing block in the non-volatile memory, and converting a type of write operation associated with the request from a requested write operation type to a new data allocation write operation when the data write request modifies the existing block. Furthermore, the method may include processing the new data allocation write operation to write the block of data from the volatile memory to the non-volatile memory using an ordered sequence of file system data journaling and writes to the non-volatile memory.

Abstract: Various of the disclosed embodiments provide for recovery following inadvertent errors in a Log-Structured File System (LSFS). Particularly, embodiments mitigate inadvertent errors introduced by a file system consistency check operation by creating file system images at an appropriate time. The images may be stored within the portion of the file system accessible to a user. The images may be created in conjunction with the mounting of the file system and in such a fashion as to preserve the file system data should an error occur. Restoring the system to one of the images may remove any errors introduced by the consistency check, or similar, operation.

Abstract: A method and apparatus is disclosed herein for maintaining data consistency in persistent memory in an in-place-update file system of a deduplication computing system. The method may include receiving a data write request to write a block of data from a volatile memory to a non-volatile memory in an in-place-update file system of deduplication computing system. Furthermore, the method may include determining whether the data write request is to perform an in-place update of an existing block in the non-volatile memory, and converting a type of write operation associated with the request from a requested write operation type to a new data allocation write operation when the data write request modifies the existing block. Furthermore, the method may include processing the new data allocation write operation to write the block of data from the volatile memory to the non-volatile memory using an ordered sequence of file system data journaling and writes to the non-volatile memory.

Abstract: A method and apparatus is disclosed herein for maintaining deduplication metadata on disk separate from an in-memory deduplication file system for a deduplication computing system. The method may include reading deduplication metadata from a first memory of a deduplication computing system. Furthermore, the method may include maintaining deduplicated data for a file system in a second memory, and maintaining the deduplication metadata that describes the deduplicated data of the file system in the first memory of the deduplication computing system, wherein the first memory is a different type of memory and separated from the second memory in the deduplication computing system. Furthermore, the method may include accessing the deduplication metadata in the second memory in response to receipt of a request by the deduplication computing system to perform a first operation with deduplicated data of the file system in the second memory.

Abstract: A method and apparatus is disclosed herein for checking the consistency of deduplication metadata after an unclean shutdown of a deduplication computing system. The method may include mounting a deduplication file system of a deduplication computing system after an unclean shutdown of the deduplication computing system. Furthermore, the method may include initiating a reconstruction and consistency check of deduplication metadata associated with the deduplication file system. Furthermore, the method may include receiving a user operation prior to completion of the reconstruction and consistency check of the deduplication metadata, and processing the user operation in parallel with the reconstruction and consistency check of the deduplication metadata.

Abstract: A persistent random-access, byte-addressable storage device may be simulated on a persistent random-access, block-addressable storage device of a storage system configured to enable asynchronous buffered access to information persistently stored on the block-addressable device. Buffered access to the information is provided, in part, by a portion of kernel memory within the storage system allocated as a staging area for the simulated byte-addressable storage device to temporarily store the information destined for persistent storage. One or more asynchronous interfaces may be employed by a user of the simulated byte-addressable device to pass metadata describing the information to a driver of the device, which may process the metadata to copy the information to the staging area. The driver may organize the staging area as one or more regions to facilitate buffering of the information (data) prior to persistent storage on the block-addressable storage device.

Abstract: A recovery consumer framework provides for execution of recovery actions by one or more recovery consumers to enable efficient recovery of information (e.g., data and metadata) in a storage system after a failure event (e.g., a power failure). The recovery consumer framework permits concurrent execution of recovery actions so as to reduce recovery time (i.e., duration) for the storage system. The recovery consumer framework may coordinate (e.g., notify) the recovery consumers to serialize execution of the recovery actions by those recovery consumers having a dependency while allowing concurrent execution between recovery consumers having no dependency relationship. Each recovery consumer may register with the framework to associate a dependency on one or more of the other recovery consumers. The dependency association may be represented as a directed graph where each vertex of the graph represents a recovery consumer and each directed edge of the graph represents a dependency. The framework may traverse (i.e.

Abstract: A recovery consumer framework provides for execution of recovery actions by one or more recovery consumers to enable efficient recovery of information (e.g., data and metadata) in a storage system after a failure event (e.g., a power failure). The recovery consumer framework permits concurrent execution of recovery actions so as to reduce recovery time (i.e., duration) for the storage system. The recovery consumer framework may coordinate (e.g., notify) the recovery consumers to serialize execution of the recovery actions by those recovery consumers having a dependency while allowing concurrent execution between recovery consumers having no dependency relationship. Each recovery consumer may register with the framework to associate a dependency on one or more of the other recovery consumers. The dependency association may be represented as a directed graph where each vertex of the graph represents a recovery consumer and each directed edge of the graph represents a dependency. The framework may traverse (i.e.

Abstract: A persistent random-access, byte-addressable storage device may be simulated on a persistent random-access, block-addressable storage device of a storage system configured to enable asynchronous buffered access to information persistently stored on the block-addressable device. Buffered access to the information is provided, in part, by a portion of kernel memory within the storage system allocated as a staging area for the simulated byte-addressable storage device to temporarily store the information destined for persistent storage. One or more asynchronous interfaces may be employed by a user of the simulated byte-addressable device to pass metadata describing the information to a driver of the device, which may process the metadata to copy the information to the staging area. The driver may organize the staging area as one or more regions to facilitate buffering of the information (data) prior to persistent storage on the block-addressable storage device.

Abstract: Various of the disclosed embodiments provide for recovery following inadvertent errors in a Log-Structured File System (LSFS). Particularly, embodiments mitigate inadvertent errors introduced by a file system consistency check operation by creating file system images at an appropriate time. The images may be stored within the portion of the file system accessible to a user. The images may be created in conjunction with the mounting of the file system and in such a fashion as to preserve the file system data should an error occur. Restoring the system to one of the images may remove any errors introduced by the consistency check, or similar, operation.

Abstract: Methods and a processing system directed to a network assisted file system checker are described. In one embodiment the checker system is a network assisted checker that employs virtual storage devices that is storage is backed by files, and optionally the files backing the virtual storage devices may be remote files accessed over a network through a network file sharing protocol such as, but not limited to, NFS or CIFS. A device driver module introduces the virtual storage device to the local operating system supporting the checker process, and the device driver maps read and write requests made by the checker process to the virtual storage device onto files being supported by the network file sharing system.