Abstract: Techniques are described for ensuring end-to-end fair-share resource utilization during cross-region replication. In certain embodiments, a fair-share architecture is used for communication among pipeline stages performing a cross-region replication between different cloud infrastructure regions. Cross-region replication-related jobs are distributed evenly from a pipeline stage into a temporary buffer in the fair-share architecture, and then further distributed evenly form the fair-share architecture to parallel running threads of next pipeline stage for execute. Techniques for static and dynamic resource allocations are also disclosed.

Abstract: Techniques are described for checkpointing multiple key ranges in parallel and concurrently during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, multiple range threads processing multiple key ranges, one thread per key range, create checkpoints for their respective key ranges in parallel and concurrently after processing a per-determined number of B-tree keys. In certain embodiments, upon encountering a failure event, either a system crash or a thread failure, each thread restarts its B-tree key processing from a B-tree key after the most recent checkpoint.

Abstract: Techniques are described for checkpointing multiple key ranges in parallel and concurrently during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, multiple range threads processing multiple key ranges, one thread per key range, create checkpoints for their respective key ranges in parallel and concurrently after processing a per-determined number of B-tree keys. In certain embodiments, each thread requests a lock from a central checkpoint record and takes turns for updating a status byte while continuing processing the B-tree keys in its responsible key range. In certain embodiments, upon encountering a failure event, either a system crash or a thread failure, each thread restarts its B-tree key processing from a B-tree key after the most recent checkpoint.

Abstract: Techniques are described for checkpointing multiple key ranges in parallel and concurrently during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, multiple range threads processing multiple key ranges, one thread per key range, create checkpoints for their respective key ranges in parallel and concurrently after processing a per-determined number of B-tree keys. In certain embodiments, each thread requests a lock from a central checkpoint record and takes turns for updating a status byte while continuing processing the B-tree keys in its responsible key range. In certain embodiments, upon encountering a failure event, either a system crash or a thread failure, each thread restarts its B-tree key processing from a B-tree key after the most recent checkpoint.

Abstract: Techniques are described for maintaining data consistency when failure events occur during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, two generation numbers (or different identifications) are assigned to two groups of processed B-tree key-value pairs, one before and one after a failure event, within a key range. In some embodiments, the two generation numbers are assigned to a group of B-tree key-value pairs processed by a failed thread and another group of B-tree key-value pairs processed by a substitute thread taking over the failed thread to avoid potential data corruption.

Abstract: Techniques are described for ensuring end-to-end fair-share resource utilization during cross-region replication. In certain embodiments, a fair-share architecture is used for communication among pipeline stages performing a cross-region replication between different cloud infrastructure regions. Cross-region replication-related jobs are distributed evenly from a pipeline stage into a temporary buffer in the fair-share architecture, and then further distributed evenly form the fair-share architecture to parallel running threads of next pipeline stage for execute. Techniques for static and dynamic resource allocations are also disclosed.