Abstract: A smart prefetching of operations technique prefetches metadata and/or data of objects stored in a multi-cloud snapshot technology (MST) service associated with an object store. One or more data objects (e.g., an application) at a primary site may be designated for backup or failover to a secondary site, e.g., in the event of failure of the primary site. Smart prefetching logic of the MST is configured to prefetch the object metadata and/or object data from the object store to serve subsequent requests without accessing the object store. To that end, when storing data to the object store as one or more objects, MST maintains specific metadata along with the data of the objects. The technique utilizes the specific metadata to prefetch the object metadata and/or object data before receiving actual read requests for that data to improve the read latency and throughput when restoring the application as soon as possible.

Abstract: A technique enables coordination of unrelated software components to facilitate extensive recovery point management on a snapshot or recovery point through the use of a flexible tag structure. The tag is organized and arranged as a {key=value,[value] . . . } structure wherein the key denotes an operation that requires coordination between the unrelated software components and the value(s) denote multi-cardinality that provide parameters for coordination of the operation. The multi-cardinality aspect of the flexible tag structure provides a set of values associated with the key of the tag that enables a software component and/or protocol to insert its value(s) into the tag structure for its interpretation. The technique thus provides an extensible model where multiple components/protocols use the tag to coordinate operations on the RP by conveying certain meaning/interpretations of the tag and its values.

Abstract: A technique enables recovery of failover data used to generate one or more High Frequency Snapshots (HFSs) at a source and replicated to a target for storage and recovery. The target is illustratively an intermediary repository embodied as a long-term storage service (LTSS) configured to organize and store the HFSs as recovery points (RPs) in an object store. The LTSS stores a HFS identifier (ID), a logical offset in an object of the object store storing data of the HFS, and a logical timestamp associated with each replicated HFS as a key of a segment descriptor in a key-value database configured to store metadata describing the failover data of the HFS stored as one or more objects in the object store. Upon recovery of the failover data, the technique enables identification of the HFS stored in the object store and creation of a HFS index metadata structure (B+ tree) to extract the identified HFS as a RP.

Abstract: A technique allows instantiation and running on demand of long-term snapshot storage services of an archival storage system at various geographical locations. Storage service instances are configured to provide storage and retrieval of large amounts of point-in-time images or snapshots (e.g., recovery points) of application workloads stored as objects on one or more buckets of a shared object store. The storage service instances may contemporaneously serve snapshots of a same set of buckets on the shared object store without interfering with each other in a contention-free manner. That is, the technique enables storage service instances that are associated with snapshot workload data and/or metadata stored, e.g., as objects of a recovery point, on the same set of buckets to coexist without knowledge of each other. The storage service instances can be created and destroyed on-demand by splitting and merging the existing instances.

Abstract: A lazy index construction technique efficiently and cost effectively manages creation and storage of an index data structure based on characteristics of storage media used by an archival storage system. The index data structure (index) is configured to reference snapshot data of snapshots stored in the archival storage system. The technique is configured to defer creation and storage of the index on the archival storage system in a lazy manner until all snapshot data is received by a replication receiver and stored on the storage media so that updates/changes to the index on the storage media are minimized. The technique may be used with any type or combination of (i) “overwrite” data structure embodied as an index (i.e., an index data structure with overwrite capabilities) stored in an (ii) archival storage system having storage media (e.g., an object store) that is not conducive to overwrite capabilities.

Abstract: A site and storage tier aware technique replicates data as one or more recovery points (RPs) from a primary site to a secondary site in a multi-site data replication environment. A storage tier aware reference resolver determines (i) an amount of RP data transfer associated with the replication and (ii) location information associated with a cloud storage tier storing the RP data in an object store. The storage tier aware reference resolution aspect provides two additional factors to consider when retrieving data of a reference RP from cloud storage: (iii) the time (duration) needed to retrieve the data and (iv) the cost (financial expense) needed to retrieve the data. In addition, a site aware reference resolution aspect of the technique determines an optimal RP to use as the reference RP and involves consideration of (v) which RPs have been replicated from the primary site to the secondary site and (vi) which RPs have been retained for storage at the sites.

Abstract: A technique enables coordination of unrelated software components to facilitate extensive recovery point management on a snapshot or recovery point through the use of a flexible tag structure. The tag is organized and arranged as a {key=value,[value] . . . } structure wherein the key denotes an operation that requires coordination between the unrelated software components and the value(s) denote multi-cardinality that provide parameters for coordination of the operation. The multi-cardinality aspect of the flexible tag structure provides a set of values associated with the key of the tag that enables a software component and/or protocol to insert its value(s) into the tag structure for its interpretation. The technique thus provides an extensible model where multiple components/protocols use the tag to coordinate operations on the RP by conveying certain meaning/interpretations of the tag and its values.

Abstract: A differencing technique enables efficient retrieval of data from one of a substantial number of point-in-time images (e.g., snapshots) maintained over substantially long periods of time in a long-term storage service (LTSS) of an archival storage system. The LTSS efficiently retrieves the data by computing differences or deltas between any two arbitrary snapshots in accordance with a differencing procedure. According to the technique, the differencing procedure operates on one or more index tree structures configured to translate a logical offset range of snapshot data in a snapshot address space (e.g., of a file system) to a data object address space (e.g., of an object store hosting the snapshot data).

Abstract: A bypassing technique bypasses an indexing service and provides a bypass data path for transferring/retrieving snapshots from a production cluster to an object store. In an embodiment, the production cluster may determine how extents of the snapshots are packed into objects of the object store and transfers the snapshots directly to the object store over the bypass data path. Once the snapshot transfer is completed, the production cluster provides location metadata as to how the snapshot extents are packed into objects to the indexing service. The indexing service is invoked to create an index of the location metadata and is not involved in the data transfer of the snapshots. In another embodiment, the production cluster identifies a snapshot to restore and queries the indexing service to compute the deltas between the snapshot to be restored and a reference snapshot. The indexing service returns a set of segments that indicates the changed delta regions between the two snapshots.

Abstract: A technique enables recovery of failover data used to generate one or more High Frequency Snapshots (HFSs) at a source and replicated to a target for storage and recovery. The target is illustratively an intermediary repository embodied as a long-term storage service (LTSS) configured to organize and store the HFSs as recovery points (RPs) in an object store. The LTSS stores a HFS identifier (ID), a logical offset in an object of the object store storing data of the HFS, and a logical timestamp associated with each replicated HFS as a key of a segment descriptor in a key-value database configured to store metadata describing the failover data of the HFS stored as one or more objects in the object store. Upon recovery of the failover data, the technique enables identification of the HFS stored in the object store and creation of a HFS index metadata structure (B+ tree) to extract the identified HFS as a RP.

Abstract: A technique improves storage efficiency of an object store configured to maintain numerous snapshots for long-term storage in an archival storage system by efficiently determining data that is exclusively owned by an expiring snapshot to allow deletion of the expiring snapshot from the object store. The technique involves managing index data structures to enable efficient garbage collection across a very large number of data objects. When a snapshot expires, the technique obviates the need to scan the numerous snapshot data objects to determine which index structures are no longer needed and can be reclaimed (garbage collected). The technique is directed to management of underlying storage based on different sets of policies. When certain snapshots expire and are ready for deletion, the technique is directed to finding those data blocks that are no longer referenced (used) by any valid snapshots.

Abstract: A reference snapshot selection technique is configured to select a reference snapshot resolution algorithm used to determine an appropriate reference snapshot that may be employed to perform incremental snapshot replication of workload data between primary and secondary sites in a data replication environment. A reference resolution procedure is configured to process a set of constraints from the data replication environment to dynamically select the reference snapshot resolution algorithm based on a figure of merit that satisfies administrative constraints to reduce or optimize resource utilization in the data replication environment.

Abstract: A reference snapshot selection technique is configured to select a reference snapshot resolution algorithm used to determine an appropriate reference snapshot that may be employed to perform incremental snapshot replication of workload data between primary and secondary sites in a data replication environment. A reference resolution procedure is configured to process a set of constraints from the data replication environment to dynamically select the reference snapshot resolution algorithm based on a figure of merit that satisfies administrative constraints to reduce or optimize resource utilization in the data replication environment.

Abstract: A technique improves storage efficiency of an object store configured to maintain numerous snapshots for long-term storage in an archival storage system by efficiently determining data that is exclusively owned by an expiring snapshot to allow deletion of the expiring snapshot from the object store. The technique involves managing index data structures to enable efficient garbage collection across a very large number of data objects. When a snapshot expires, the technique obviates the need to scan the numerous snapshot data objects to determine which index structures are no longer needed and can be reclaimed (garbage collected). The technique is directed to management of underlying storage based on different sets of policies. When certain snapshots expire and are ready for deletion, the technique is directed to finding those data blocks that are no longer referenced (used) by any valid snapshots.

Abstract: An indexing technique provides an index data structure for efficient retrieval of a snapshot from a long-term storage service (LTSS) of an archival storage system. The snapshot is generated from typed data of a logical entity, such as a virtual disk (vdisk). The data of the snapshot is replicated to a frontend data service of the LTSS sequentially and organized as one or more data objects for storage by a backend data service of LTSS in an object store of the archival storage system. Metadata associated with the snapshot (i.e., snapshot metadata) is recorded as a log and persistently stored on storage media local to the frontend data service. The snapshot metadata includes information describing the snapshot data, e.g., a logical offset range of a snapshot of the vdisk and, thus, is used to construct the index data structure. Notably, construction of the index data structure is deferred until after the entirety of the snapshot data has been replicated and received by the frontend data service.

Abstract: Methods, systems and computer program products for high-availability computing. In a computing configuration comprising a primary node, a first backup node, and a second backup node, a particular data state is restored to the primary node from a backup snapshot at the second backup node. Firstly, a snapshot coverage gap is identified between a primary node snapshot at the primary node and the backup snapshot at the second backup node. Next, intervening snapshots at the first backup node that fills the snapshot coverage gap are identified and located. Having both the backup snapshot from the second backup node and the intervening snapshots from the first backup node, the particular data state at the primary node is restored by performing differencing operations between the primary node snapshot, the backup snapshot from the second backup node, and the intervening snapshots of the first backup node.

Abstract: A system and method determining a Chain Identification Number (CID) of a source snapshot to be replicated from a source site to a target site of a virtual computing system, determining a predetermined number of potential reference snapshots based on the CID of the source snapshot, computing a closeness value between the source snapshot and each of the potential reference snapshots, and creating a list of the potential reference snapshots based on the closeness value of each of the potential reference snapshots. One snapshot from the list is selected as a reference snapshot. The source snapshot is replicated to the target site based on the reference snapshot.

Abstract: A system and method determining a Chain Identification Number (CID) of a source snapshot to be replicated from a source site to a target site of a virtual computing system, determining a predetermined number of potential reference snapshots based on the CID of the source snapshot, computing a closeness value between the source snapshot and each of the potential reference snapshots, and creating a list of the potential reference snapshots based on the closeness value of each of the potential reference snapshots. One snapshot from the list is selected as a reference snapshot. The source snapshot is replicated to the target site based on the reference snapshot.

Abstract: Methods, systems and computer program products for high-availability computing. In a computing configuration comprising a primary node, a first backup node, and a second backup node, a particular data state is restored to the primary node from a backup snapshot at the second backup node. Firstly, a snapshot coverage gap is identified between a primary node snapshot at the primary node and the backup snapshot at the second backup node. Next, intervening snapshots at the first backup node that fills the snapshot coverage gap are identified and located. Having both the backup snapshot from the second backup node and the intervening snapshots from the first backup node, the particular data state at the primary node is restored by performing differencing operations between the primary node snapshot, the backup snapshot from the second backup node, and the intervening snapshots of the first backup node.