Abstract: Techniques are described for a hierarchical caching mechanism enabling efficient cross-region replications. In some embodiments, replication-related information (e.g., key-value pairs) is stored in a particular layout in a binary tree (B-tree) of a file system for replication processing. A hierarchy of caches storing a first type of information (e.g., crypto keys associated with iNodes) may be arranged to match the particular layout in the B-tree to enable efficient parallel processing of a second type of information (e.g., files, file data, or symbolic links), where the replication-related information in the B-tree is partitioned into multiple key ranges for parallel processing. In some embodiments, the caches in different hierarchies may be shared by different parallel-processing key ranges and replication jobs in a file system.

Abstract: Techniques are described for snapshot key inter-dependency resolution during cross-region replications. Dependency between a first type of replication-related information (e.g., crypto keys associated with a parent directory iNode or a file iNode) and a second type of replication-related information (e.g., files, file data/FMAPs, or symbolic links) during a cross-region replication may be resolved to enable non-blocking delta application in a target file system. In some embodiments, temporary dummy entries for the first type of information may be created in the B-tree of the target file system for the out-of-order download (e.g., the second type being downloaded before the first type) of these two types of information. In some embodiments, a consolidation process may be performed between the dummy entries and the later-arriving first type of information.

Abstract: Techniques are described for enabling concurrent and non-blocking replication object deletion during cross-region replications. In some embodiments, in a target file system, a target replication pipeline as part of a cross-region replication, and a deletion pipeline operate in parallel. The deletion pipeline deletes processed objects reaching the last pipeline stage of the target replication pipeline after each checkpoint in the target replication pipeline. In some embodiments, after a non-recoverable failure during the cross-region replication, the cross-region replication can be restarted from the beginning (i.e., fresh restart) without waiting for its unused objects in the Object Store to be deleted by utilizing a generation number associated with each object to delete the unused objects in a background process while allowing deleting processed objects as normal for the freshly restarted cross-region replication.

Abstract: Techniques are described for enabling replication-aware resource management and task management in a cloud infrastructure for cross-region replication. In some embodiments, each replication job is associated with a set of replication-related information. In certain embodiments, the replication-aware resource management allocates resources, using a combination of various resource allocation schemes, to a fleet of replicators to allow the fleet to select replication jobs in a job queue, and perform resource scaling based on monitored performance metrics reported by the fleet. In some embodiments, the replication-aware task management enables replication job selection based on the set of replication-related information to optimize the performance of all cross-region replications in the region.

Abstract: Techniques are described for enabling concurrent cross-region replications and garbage collection while maintaining consistency and data integrity among file systems. In some embodiments, techniques for garbage collection fencing utilize a system-level garbage fencing key (GC fencing key) and one or more job-level GC fencing keys in a source file system that perform one or more cross-region replications with one or more target file systems, one replication and one job-level GC fencing key per target file system. In some embodiments, one job-level GC fencing key in a source file system and one job-level GC fencing key in a source file system together provide garbage fencing for a cross-region replication. In certain embodiments, the metadata information in a GC fencing key can inform, instruct, or be used to configure garbage collectors to skip garbage collection for a range of snapshots in a file system.

Abstract: Techniques are described for checkpointing multiple key ranges in parallel and concurrently during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, multiple range threads processing multiple key ranges, one thread per key range, create checkpoints for their respective key ranges in parallel and concurrently after processing a per-determined number of B-tree keys. In certain embodiments, each thread requests a lock from a central checkpoint record and takes turns for updating a status byte while continuing processing the B-tree keys in its responsible key range. In certain embodiments, upon encountering a failure event, either a system crash or a thread failure, each thread restarts its B-tree key processing from a B-tree key after the most recent checkpoint.

Abstract: Novel techniques for end-to-end file storage replication and security between file systems in different cloud infrastructure regions are disclosed herein. In one embodiment, a file storage service generates deltas between snapshots in a source file system, and transfers the deltas and associated data through a high-throughput object storage to recreate a new snapshot in a target file system located in a different region during disaster recovery. The file storage service utilizes novel techniques to achieve scalable, reliable, and restartable end-to-end replication. Novel techniques are also described to ensure a secure transfer of information and consistency during the end-to-end replication.

Abstract: Novel techniques are described for performing hierarchical key management involving an end-to-end file storage replication between different cloud infrastructure regions. The novel hierarchical key management comprises three different keys, a first security key for the source region, a session key, valid only for a session, for the transfer of data between two different regions, and a second security key for the target region. Novel techniques are also described for using different file keys for different files of a file system in each region.

Abstract: Novel techniques are described for maintaining file consistency between different cloud infrastructure regions during cross-region file storage replication. The novel techniques comprise upon detecting a file deletion status (i.e., a file directory entry version marked as deletion) in a source file system during a replication cycle, retrieving file name from a valid file directory entry of a prior snapshot to construct a temporary file directory entry containing necessary information for a target file system to reconstruct a duplicate file directory entry version marked as deletion. The temporary file directory entry is transferred from the source file system to the target file system during the same replication cycle.

Abstract: Techniques are described for performing different types of restart operations for a file storage replication between a source file system and a target file system in different cloud infrastructure regions. In certain embodiments, the disclosed techniques perform a restart operation to terminate a current cross-region replication by synchronizing resource cleanup operations in the source file system and the target file system, respectively. In other embodiments, disclosed techniques perform a restart operation to allow a customer to reuse the source file system by identifying a restartable base snapshot in the source file system without dependency on the target file system.

Abstract: Techniques are described for performing different types of restart operations for a file storage replication between a source file system and a target file system in different cloud infrastructure regions. In certain embodiments, the disclosed techniques perform a restart operation to terminate a current cross-region replication by synchronizing resource cleanup operations in the source file system and the target file system, respectively. In other embodiments, disclosed techniques perform a restart operation to allow a customer to reuse the source file system by identifying a restartable base snapshot in the source file system without dependency on the target file system.

Abstract: Techniques are described for partitioning B-tree keys of file systems into key ranges for parallel processing in delta generation during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, a delta generation processing for cross-region replication may utilize a key-range splitting mechanism involving a recursive algorithm that partitions B-tree keys of a source file system into roughly equal-size key ranges. All the partitioned key ranges may be processed in parallel and concurrently by different processing threads, one thread per key range, to improve the performance of the delta generation and achieve scalability.

Abstract: Techniques are described for maintaining data consistency when failure events occur during file storage replications between file systems in different cloud infrastructure regions. In certain embodiments, two generation numbers (or different identifications) are assigned to two groups of processed B-tree key-value pairs, one before and one after a failure event, within a key range. In some embodiments, the two generation numbers are assigned to a group of B-tree key-value pairs processed by a failed thread and another group of B-tree key-value pairs processed by a substitute thread taking over the failed thread to avoid potential data corruption.

Abstract: Techniques are described for efficient replication and maintaining snapshot data consistency during file storage replication between file systems in different cloud infrastructure regions. In certain embodiments, provenance IDs are used to efficiently identify a starting point (e.g., a base snapshot) for a cross-region replication process, conserve cloud resources while reducing network and IO traffic.

Abstract: Techniques are described for ensuring end-to-end fair-share resource utilization during cross-region replication. In certain embodiments, a fair-share architecture is used for communication among pipeline stages performing a cross-region replication between different cloud infrastructure regions. Cross-region replication-related jobs are distributed evenly from a pipeline stage into a temporary buffer in the fair-share architecture, and then further distributed evenly form the fair-share architecture to parallel running threads of next pipeline stage for execute. Techniques for static and dynamic resource allocations are also disclosed.

Abstract: Techniques for managing overwrites when archiving data in cloud/object storage are provided. In one set of embodiments, a computer system can overwrite a chunk of data residing in cloud/object storage, where the cloud/object storage only guarantees eventual consistency for overwrites, and where the overwriting is associated with a sequence number. Upon overwriting the chunk of data, the computer system create a new instance of a file in the cloud/object storage, where the new instance of the file has a version number corresponding to the sequence number.

Abstract: According to one set of embodiments, a computer system can receive a request or command to delete a snapshot from among a plurality of snapshots of a dataset, where the plurality of snapshots are stored in cloud/object storage. In response, the computer system can add the snapshot to a batch of pending snapshots to be deleted and can determine whether the size of the batch has reached a threshold. If the size of the batch has not reached the threshold, the computer system return a response to an originator of the request or command indicating that the snapshot has been deleted, without actually deleting the snapshot from the cloud/object storage.

Abstract: Techniques for archiving data in cloud/object storage using local metadata staging are provided. In one set of embodiments, a computer system residing at an on-premises site comprising on-premises storage can receive a snapshot of a dataset to be archived. The computer system can package data in the snapshot into one or more fixed-size data chunks and upload the one or more fixed-size data chunks to cloud/object storage. Further, concurrently with the packaging and the uploading, the computer system can stage metadata for the snapshot in the on-premises storage. Then, upon uploading all of the data of the snapshot, the computer system can upload the metadata staged in the on-premises storage to the cloud/object storage.

Abstract: Techniques for supporting glacier tiering of archived data in cloud/object storage are provided. In one set of embodiments, a computer system can identify one or more snapshots of a data set archived in the cloud/object storage that are scheduled to be migrated from a standard tier of the cloud/object storage to a glacier tier of the cloud/object storage. The computer system can further determine one or more data objects of the one or more snapshots that are referred to by at least one child snapshot and clone the one or more data objects in the standard tier.

Abstract: Techniques for handling fragmentation of archived data in cloud/object storage are provided. In one set of embodiments, a computer system can upload a new snapshot of a data set to the cloud/object storage, where the new snapshot comprises a plurality of data blocks, and where the new snapshot is uploaded as one or more data objects and one or more metadata objects. For each data block in the plurality of data blocks, the computer system can identify an existing data object in the cloud/object storage where the data block is currently stored. The computer system can further select, from among the identified existing data objects, a subset of the existing data objects that are part of a snapshot created in the cloud/object storage at least a predefined number of snapshots before the new snapshot, and select one or more data blocks of one or more data objects in the subset that have not be overwritten by another snapshot.