Abstract: Techniques are described for checkpointing multiple key ranges in parallel and concurrently during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, multiple range threads processing multiple key ranges, one thread per key range, create checkpoints for their respective key ranges in parallel and concurrently after processing a per-determined number of B-tree keys. In certain embodiments, each thread requests a lock from a central checkpoint record and takes turns for updating a status byte while continuing processing the B-tree keys in its responsible key range. In certain embodiments, upon encountering a failure event, either a system crash or a thread failure, each thread restarts its B-tree key processing from a B-tree key after the most recent checkpoint.

Abstract: Techniques are described for enabling concurrent and non-blocking replication object deletion during cross-region replications. In some embodiments, in a target file system, a target replication pipeline as part of a cross-region replication, and a deletion pipeline operate in parallel. The deletion pipeline deletes processed objects reaching the last pipeline stage of the target replication pipeline after each checkpoint in the target replication pipeline. In some embodiments, after a non-recoverable failure during the cross-region replication, the cross-region replication can be restarted from the beginning (i.e., fresh restart) without waiting for its unused objects in the Object Store to be deleted by utilizing a generation number associated with each object to delete the unused objects in a background process while allowing deleting processed objects as normal for the freshly restarted cross-region replication.

Abstract: Techniques are described for checkpointing multiple key ranges in parallel and concurrently during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, multiple range threads processing multiple key ranges, one thread per key range, create checkpoints for their respective key ranges in parallel and concurrently after processing a per-determined number of B-tree keys. In certain embodiments, each thread requests a lock from a central checkpoint record and takes turns for updating a status byte while continuing processing the B-tree keys in its responsible key range. In certain embodiments, upon encountering a failure event, either a system crash or a thread failure, each thread restarts its B-tree key processing from a B-tree key after the most recent checkpoint.

Abstract: Techniques are described for partitioning B-tree keys of file systems into key ranges for parallel processing in delta generation during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, a delta generation processing for cross-region replication may utilize a key-range splitting mechanism involving a recursive algorithm that partitions B-tree keys of a source file system into roughly equal-size key ranges. All the partitioned key ranges may be processed in parallel and concurrently by different processing threads, one thread per key range, to improve the performance of the delta generation and achieve scalability.

Abstract: Techniques are described for maintaining data consistency when failure events occur during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, two generation numbers (or different identifications) are assigned to two groups of processed B-tree key-value pairs, one before and one after a failure event, within a key range. In some embodiments, the two generation numbers are assigned to a group of B-tree key-value pairs processed by a failed thread and another group of B-tree key-value pairs processed by a substitute thread taking over the failed thread to avoid potential data corruption.

Abstract: Methods, non-transitory computer readable media, and computing devices that facilitate copy-free data migrations across filesystems. In a first step with this technology, a first set of filesystem metadata associated with a first filesystem is received. At least a portion of the first set of filesystem metadata is retrieved from a first data structure associated with the first filesystem. The first set of filesystem metadata includes a first identifier and a physical location associated with user data. A second identifier, associated with a second filesystem having a different addressing scheme than the first filesystem, is generated from at least the first identifier. A second set of filesystem metadata including the second identifier and the physical location is stored such that at least the second identifier is stored in a second data structure associated with the second filesystem.

Abstract: Methods, non-transitory computer readable media, and computing devices that facilitate copy-free data migrations across filesystems. In a first step with this technology, a first set of filesystem metadata associated with a first filesystem is received. At least a portion of the first set of filesystem metadata is retrieved from a first data structure associated with the first filesystem. The first set of filesystem metadata includes a first identifier and a physical location associated with user data. A second identifier, associated with a second filesystem having a different addressing scheme than the first filesystem, is generated from at least the first identifier. A second set of filesystem metadata including the second identifier and the physical location is stored such that at least the second identifier is stored in a second data structure associated with the second filesystem.