Abstract: Multi-site distributed storage systems and computer-implemented methods are described for providing an automatic unplanned failover (AUFO) feature to guarantee non-disruptive operations (e.g., operations of business enterprise applications, operations of software application) even in the presence of failures including, but not limited to, network disconnection between multiple data centers and failures of a data center or cluster.

Abstract: Systems and methods for making a cross-site storage solution resilient towards mediator unavailability are provided. According to one embodiment, a stretched storage system is operable to bring a mediator associated with a primary and secondary distributed storage system back into the role of an arbitrator for peered consistency groups (CGs). A mediator reseed status indicator is maintained for multiple CGs to identify when the mediator's status information for a CG is stale. When the mediator becomes available and a local CG is identified as the subject of a mediator reseed process, the master node of the primary that hosts a master copy of a dataset for the local CG performs the reseed process, including: (i) causing relationship status information for the local CG to be updated on the mediator to the current state maintained by the primary; and (ii) resetting the mediator reseed status indicator.

Abstract: Techniques are provided for resynchronizing a synchronous replication relationship. Asynchronous incremental transfers are performed to replicate data of a storage object to a replicated storage object. Incoming write requests, targeting the storage object, are logged into a dirty region log during a last asynchronous incremental transfer. Metadata operations, executed on the storage object, are logged into a metadata log during the last asynchronous incremental transfer. Sequence numbers are assigned to the metadata operations based upon an order of execution. The metadata operations are replicated to the replicated storage object for execution according to the sequence numbers, and the dirty regions are replicated to the replicated storage object in response to the metadata operations having been replicated to the replicated storage object.

Abstract: One or more techniques and/or computing devices are provided for managing an arbitrary set of storage items using a granset. For example, a storage controller may host a plurality of storage items and/or logical unit numbers (LUNs). A subset of the storage items are grouped into a consistency group. A granset is created for tracking, managing, and/or providing access to the storage items within the consistency group. For example, the granset comprises application programming interfaces (APIs) and/or properties used to provide certain levels of access to the storage items (e.g., read access, write access, no access), redirect operations to access either data of an active file system or to a snapshot, fence certain operations (e.g., rename and delete operations), and/or other properties that apply to each storage item within the consistency group. Thus, the granset provides a persistent on-disk layout used to manage an arbitrary set of storage items.

Abstract: Systems and methods are described for a non-disruptive planned failover from a primary copy of data at a primary storage cluster to a mirror copy of the data at a cross-site secondary storage cluster without using an external mediator. According to an example, a planned failover feature of a multi-site distributed storage system provides an order of operations such that a primary copy of a first data center continues to serve I/O operations until a mirror copy of a second data center is ready. This planned failover feature improves functionality and efficiency of the distributed storage system by providing non-disruptiveness during planned failover without using an external mediator based on a primary storage cluster being selected as an authority to implement a state machine with a persistent configuration database to track a planned failover state for the planned failover.

Abstract: A system and method for enabling data replication is described. A set of protocol messages can be associated with a set of corresponding set of requests. The associated set of protocol messages can be provided to each of a source storage system and a destination storage system. The source storage system and the destination storage system can be heterogeneous storage systems that implement different types of file systems or file layouts. The source storage system and the destination storage system are enabled to perform a data replication process, in which at least a set of data is to be replicated from the source storage system to the destination storage system, using at least a protocol message from the associated set of protocol messages. The protocol message can correspond to a request for a list of snapshots stored at the source storage system.

Abstract: One or more techniques and/or computing devices are provided for synchronous replication. For example, synchronous replication relationships are established between a first storage object (e.g., a file, a logical unit number (LUN), a consistency group, etc.), hosted by a first storage controller, and a plurality of replication storage objects hosted by other storage controllers. In this way, a write operation to the first storage object is implemented in parallel upon the first storage object and the replication storage objects in a synchronous manner, such as using a zero-copy operation to reduce overhead otherwise introduced by performing copy operations. Reconciliation is performed in response to a failure so that the first storage object and the replication storage objects comprise consistent data. Failed write operations and replication write operations are retried, while enforcing a single write semantic.

Abstract: Systems and methods for re-aligning data replication configuration of a cross-site storage solution after a failover are provided. According to one embodiment, after a failover, the new primary distributed storage system orchestrates flipping of the data replication configuration of a peered consistency group (CG) to reestablish zero RPO and zero RTO protections for the peered CG. The primary causes the secondary distributed storage system to perform an atomic database operation on its remote configuration database to (i) delete an existing source configuration that identifies the secondary as a source of data replication; and (ii) persist a new destination configuration identifying the secondary as a destination of data replication.

Abstract: One or more techniques and/or computing devices are provided for utilizing snapshots for data integrity validation and/or faster application recovery. For example, a first storage controller, hosting first storage, has a synchronous replication relationship with a second storage controller hosting second storage. A snapshot replication policy rule is defined to specify that a replication label is to be used for snapshot create requests, targeting the first storage, that are to be replicated to the second storage. A snapshot creation policy is created to issue snapshot create requests comprising the replication label. Thus a snapshot of the first storage and a replication snapshot of the second storage are created based upon a snapshot create request comprising the replication label. The snapshot and the replication snapshot may be compared for data integrity validation (e.g., determine whether the snapshots comprise the same data) and/or quickly recovering an application after a disaster.

Abstract: One or more techniques and/or computing devices are provided for implementing synchronous replication. For example, a synchronous replication relationship may be established between a first storage controller hosting local storage and a second storage controller hosting remote storage (e.g., replication may be specified at a file, logical unit number (LUN), or any other level of granularity). Data operations and offloaded operations may be implemented in parallel upon the local storage and the remote storage. Error handling operations may be implemented upon the local storage and implement in parallel as a best effort on the remote storage, and a reconciliation may be performed to identify any data divergence from the best effort parallel implementation. Storage area network (SAN) operations may be implemented upon the local storage, and upon local completion may be remotely implemented upon the remote storage.

Abstract: Techniques are provided for neutralizing replication errors. An operation is executed upon a first storage object and is replicated as a replicated operation for execution upon a second storage object. A first error may be received for the replicated operation. Instead of transitioning to an out of sync state and aborting the operation, a wait is performed until a result of the attempted execution of the operation is received. If the first error is the same as a second error returned for the operation, then the operation and replicated operation are considered successful and a synchronous replication relationship is kept in sync. If the first error and the second error are different errors, then an error response is returned for the operation and the synchronous replication relationship is transitioned to out of sync.

Abstract: One or more techniques and/or computing devices are provided for resilient replication of storage operations. For example, a first storage controller may host first storage having a replication relationship with second storage hosted by a second storage controller. To improve resiliency against transient network issues of a network between the storage controllers, the first storage controller may implement a queue and retry mechanism to retry replication operations not acknowledge back by the second storage controller within a threshold time. The second storage controller may maintain a cumulative sequence number of a latest replication operation performed in order, an operation response map of replication operations performed out of order, and an operation finder map identifying currently implemented replication operations, which may be used to process incoming replication operations. Single write semantics, write order consistency, and reduction of write amplification may be provided.

Abstract: One or more techniques and/or computing devices are provided for synchronous replication. For example, synchronous replication relationships are established between a first storage object (e.g., a file, a logical unit number (LUN), a consistency group, etc.), hosted by a first storage controller, and a plurality of replication storage objects hosted by other storage controllers. In this way, a write operation to the first storage object is implemented in parallel upon the first storage object and the replication storage objects in a synchronous manner, such as using a zero-copy operation to reduce overhead otherwise introduced by performing copy operations. Reconciliation is performed in response to a failure so that the first storage object and the replication storage objects comprise consistent data. Failed write operations and replication write operations are retried, while enforcing a single write semantic.

Abstract: One or more techniques and/or computing devices are provided for replicating virtual machine disk clones. For example, a first storage controller, hosting first storage, may have a synchronous replication relationship with a second storage controller hosting second storage. A virtual machine, within the first storage, may be specified as having synchronous replication protection. Accordingly, virtual machine disk clones of a virtual machine disk of the virtual machine may be replicated from the first storage to the second storage. For example, virtual machine disk clones may be synchronous replicated, replicated by a resync process invoked by a hypervisor agent, and/or stored and replicated from a clone backup directory.

Abstract: Techniques are provided for synchronous replication for synchronous mirror copy guarantee. A file system dependent technique for synchronous mirror copy guarantee is provided by overriding default behavior of a persistent fence so that the persistent fence is activated to block operations targeting a storage object having a synchronous replication relationship based upon the synchronous replication relationship being out of sync. The default behavior of the persistent fence is overridden to allow operations to be executed upon the storage object based upon the synchronous replication relationship being in sync. A file system independent technique for synchronous mirror copy guarantee is provided by intercepting operations before the operations are received by a file system. The operations are selectively forwarded to the file system or not based upon a state of a synchronous replication relationship.

Abstract: One or more techniques and/or computing devices are provided for resynchronization. For example, a request may be received to create pseudo snapshots of a first consistency group, hosted by a first storage controller, and a second consistency group, hosted by a second storage controller, having a synchronous replication relationship with the first consistency group. Incoming client write requests are logged within an intercept tracking log at the first storage controller. After a first drain without hold of incoming write requests is performed, a first pseudo common snapshot of the second consistency group is created. After a second drain without hold of incoming write operations is performed, a second pseudo common snapshot of the first consistency group and the intercept tracking log is created. The pseudo snapshots and the intercept tracking log (e.g., indicating a delta between the pseudo snapshots) are used to resynchronize the first and second consistency groups.

Abstract: Techniques are provided for synchronous replication for synchronous mirror copy guarantee. A file system dependent technique for synchronous mirror copy guarantee is provided by overriding default behavior of a persistent fence so that the persistent fence is activated to block operations targeting a storage object having a synchronous replication relationship based upon the synchronous replication relationship being out of sync. The default behavior of the persistent fence is overridden to allow operations to be executed upon the storage object based upon the synchronous replication relationship being in sync. A file system independent technique for synchronous mirror copy guarantee is provided by intercepting operations before the operations are received by a file system. The operations are selectively forwarded to the file system or not based upon a state of a synchronous replication relationship.

Abstract: One or more techniques and/or computing devices are provided for implementing synchronous replication. For example, a synchronous replication relationship may be established between a first storage controller hosting local storage and a second storage controller hosting remote storage (e.g., replication may be specified at a file, logical unit number (LUN), or any other level of granularity). Data operations and offloaded operations may be implemented in parallel upon the local storage and the remote storage. Error handling operations may be implemented upon the local storage and implement in parallel as a best effort on the remote storage, and a reconciliation may be performed to identify any data divergence from the best effort parallel implementation. Storage area network (SAN) operations may be implemented upon the local storage, and upon local completion may be remotely implemented upon the remote storage.

Abstract: Techniques are provided for transitioning a first storage object and a second storage object from an asynchronous replication state to a synchronous replication state. Metadata operations are logged into a metadata log. Dirty data, modified by data operations executed upon the first storage object, are tracked using a dirty region log. Metadata operations are replicated from the metadata log to the second storage object during a metadata drain phase. During the metadata drain phase, incoming operations are logged into the metadata log when the metadata log has a non-zero size, otherwise, the metadata operations are replicated to the second storage object. After the metadata drain phase, a data drain phase is performed using the dirty region log.

Abstract: One or more techniques and/or computing devices are provided for utilizing snapshots for data integrity validation and/or faster application recovery. For example, a first storage controller, hosting first storage, has a synchronous replication relationship with a second storage controller hosting second storage. A snapshot replication policy rule is defined to specify that a replication label is to be used for snapshot create requests, targeting the first storage, that are to be replicated to the second storage. A snapshot creation policy is created to issue snapshot create requests comprising the replication label. Thus a snapshot of the first storage and a replication snapshot of the second storage are created based upon a snapshot create request comprising the replication label. The snapshot and the replication snapshot may be compared for data integrity validation (e.g., determine whether the snapshots comprise the same data) and/or quickly recovering an application after a disaster.