Abstract: Systems and methods for making a cross-site storage solution resilient towards mediator unavailability are provided. According to one embodiment, a stretched storage system is operable to bring a mediator associated with a primary and secondary distributed storage system back into the role of an arbitrator for peered consistency groups (CGs). A mediator reseed status indicator is maintained for multiple CGs to identify when the mediator's status information for a CG is stale. When the mediator becomes available and a local CG is identified as the subject of a mediator reseed process, the master node of the primary that hosts a master copy of a dataset for the local CG performs the reseed process, including: (i) causing relationship status information for the local CG to be updated on the mediator to the current state maintained by the primary; and (ii) resetting the mediator reseed status indicator.

Abstract: Systems and methods are described for efficiently performing various operations at the granularity of a consistency group (CG) within a cross-site storage solution. An example of one of the various operations includes an independent and parallel resynchronization approach that independently brings individual volumes of a CG to a steady state of in-synchronization (InSync), thereby contributing to scalability of CGs by supporting CGs having a large number of member volumes without requiring a change to the resynchronization process. Another example includes preserving dependent write-order consistency when a remote mirror copy goes out-of-synchronization (OOS) for any reason by driving all member volumes OOS responsive to any member volume becoming OOS. Yet another example includes independent creation of snapshots by member volumes to support efficient and on-demand creation by an application of a common snapshots of all or a subset of peered member volumes of a CG with which the application is associated.

Abstract: Systems and methods are described for efficiently processing events related to a relationship between a primary copy of data at a primary storage system and a mirror copy of the data at a cross-site secondary storage system of a multi-site distributed storage system. According to an example, a mediator agent that is configured on both primary and secondary storage systems provides coordination and serialization for various events generated in the relationship. The multi-site distributed storage system performs actions based on the event processing such as performing a failover operation from the primary storage system to the secondary storage system or resynchronizing the relationship to ensure application protection and availability.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for improving a resumption time of input/output (I/O) operations during an automatic unplanned failover (AUFO). A computer-implemented method includes monitoring, with a second cluster, heartbeat information received at ultra-short time intervals from a first connection of one or more storage objects of the first cluster, determining, with the second cluster, whether the heartbeat information from the first connection is received during an ultra-short time interval, and intelligently routing heartbeat information from the one or more storage objects of the first cluster from the first connection to a second connection when the heartbeat information from the first connection is not received during the ultra-short time interval.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for improving a resumption time for processing of input/output (I/O) operations during an automatic unplanned failover (AUFO). A first storage cluster includes a first set of consistency groups (CGs) and a second storage cluster includes a second mirrored set of CGs. A computer-implemented method includes prefetching, with a user space of the second storage cluster, configuration information from a replicated database prior to starting the AUFO workflow, sending the configuration information to a kernel space of the second storage cluster on a per CG level while queuing the AUFO workflow, and determining if any in progress workflows conflict with the AUFO workflow.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for improving a resumption time of input/output (I/O) operations during an automatic unplanned failover (AUFO). A computer-implemented method includes determining, with a second storage cluster, whether heartbeat information from one or more storage objects of a CG of a first set of CGs is received during a time period, determining an out of sync state for a data replication relationship between the CG of the first set of CGs and a mirrored CG of a second set of CGs when the heartbeat information is not received during the time period and sending a single bulk role change call with a cluster identifier from the second cluster to an external mediator to provide a role change from follower to leader in the second set of CGs.

Abstract: Systems and methods are described for a non-disruptive planned failover from a primary copy of data at a primary storage system to a mirror copy of the data at a cross-site secondary storage system. According to an example, a planned failover feature of a multi-site distributed storage system provides an order of operations such that a primary copy of a first data center continues to serve I/O operations until a mirror copy of a second data center is ready. This planned failover feature improves functionality and efficiency of the distributed storage system by providing non-disruptiveness during planned failover—even if various failures occur. The planned failover feature also includes a persistent fence to avoid serving I/O operations during a timing window when both primary data storage and secondary data storage are attempting to have a master role to serve I/O operations and this avoids a split-brain situation.

Abstract: Systems and methods for making a cross-site storage solution resilient towards mediator unavailability are provided. According to one embodiment, a stretched storage system is operable to bring a mediator associated with a primary and secondary distributed storage system back into the role of an arbitrator for peered consistency groups (CGs). A mediator reseed status indicator is maintained for multiple CGs to identify when the mediator’s status information for a CG is stale. When the mediator becomes available and a local CG is identified as the subject of a mediator reseed process, the master node of the primary that hosts a master copy of a dataset for the local CG performs the reseed process, including: (i) causing relationship status information for the local CG to be updated on the mediator to the current state maintained by the primary; and (ii) resetting the mediator reseed status indicator.

Abstract: Systems and methods are described for efficiently performing various operations at the granularity of a consistency group (CG) within a cross-site storage solution. An example of one of the various operations includes an independent and parallel resynchronization approach that independently brings individual volumes of a CG to a steady state of in-synchronization (InSync), thereby contributing to scalability of CGs by supporting CGs having a large number of member volumes without requiring a change to the resynchronization process. Another example includes preserving dependent write-order consistency when a remote mirror copy goes out-of-synchronization (OOS) for any reason by driving all member volumes OOS responsive to any member volume becoming OOS. Yet another example includes independent creation of snapshots by member volumes to support efficient and on-demand creation by an application of a common snapshots of all or a subset of peered member volumes of a CG with which the application is associated.

Abstract: Systems and methods for making a cross-site storage solution resilient towards mediator unavailability are provided. According to one embodiment, a stretched storage system is operable to bring a mediator associated with a primary and secondary distributed storage system back into the role of an arbitrator for peered consistency groups (CGs). A mediator reseed status indicator is maintained for multiple CGs to identify when the mediator's status information for a CG is stale. When the mediator becomes available and a local CG is identified as the subject of a mediator reseed process, the master node of the primary that hosts a master copy of a dataset for the local CG performs the reseed process, including: (i) causing relationship status information for the local CG to be updated on the mediator to the current state maintained by the primary; and (ii) resetting the mediator reseed status indicator.

Abstract: Systems and methods are described for a non-disruptive planned failover from a primary copy of data at a primary storage cluster to a mirror copy of the data at a cross-site secondary storage cluster without using an external mediator. According to an example, a planned failover feature of a multi-site distributed storage system provides an order of operations such that a primary copy of a first data center continues to serve I/O operations until a mirror copy of a second data center is ready. This planned failover feature improves functionality and efficiency of the distributed storage system by providing non-disruptiveness during planned failover without using an external mediator based on a primary storage cluster being selected as an authority to implement a state machine with a persistent configuration database to track a planned failover state for the planned failover.

Abstract: Systems and methods are described for a non-disruptive planned failover from a primary copy of data at a primary storage system to a mirror copy of the data at a cross-site secondary storage system. According to an example, a planned failover feature of a multi-site distributed storage system provides an order of operations such that a primary copy of a first data center continues to serve I/O operations until a mirror copy of a second data center is ready. This planned failover feature improves functionality and efficiency of the distributed storage system by providing non-disruptiveness during planned failover - even if various failures occur. The planned failover feature also includes a persistent fence to avoid serving I/O operations during a timing window when both primary data storage and secondary data storage are attempting to have a master role to serve I/O operations and this avoids a split-brain situation.

Abstract: Systems and methods are provided for bringing a volume of a consistency group (CG) into an in-synchronization (InSync) state while other volumes of the CG remain in the InSync state. According to an example, in order to support recovery from disruptive events in a manner that ensures a zero recovery point objective (RPO) guarantee and insulates an application making use of the CG from adverse impacts, responsive to a triggering event, a Fast Resync process may first be attempted to promptly bring an affected volume back into an in-synchronization (InSync) state from an out of synchronization (OOS) state while allowing other members of the CG to remain in the InSync state. Should the Fast resync process be unsuccessful in bringing the volume back into the InSync state within a predetermined or configurable time threshold, then a second type of resynchronization process may be employed at the CG level.

Abstract: Systems and methods are described for a non-disruptive planned failover from a primary copy of data at a primary storage system to a mirror copy of the data at a cross-site secondary storage system. According to an example, a planned failover feature of a multi-site distributed storage system provides an order of operations such that a primary copy of a first data center continues to serve I/O operations until a mirror copy of a second data center is ready. This planned failover feature improves functionality and efficiency of the distributed storage system by providing non-disruptiveness during planned failover—even if various failures occur. The planned failover feature also includes a persistent fence to avoid serving I/O operations during a timing window when both primary data storage and secondary data storage are attempting to have a master role to serve I/O operations and this avoids a split-brain situation.

Abstract: Systems and methods are provided for bringing a volume of a consistency group (CG) into an in-synchronization (InSync) state while other volumes of the CG remain in the InSync state. According to an example, in order to support recovery from disruptive events in a manner that ensures a zero recovery point objective (RPO) guarantee and insulates an application making use of the CG from adverse impacts, responsive to a triggering event, a Fast Resync process may first be attempted to promptly bring an affected volume back into an in-synchronization (InSync) state from an out of synchronization (OOS) state while allowing other members of the CG to remain in the InSync state. Should the Fast resync process be unsuccessful in bringing the volume back into the InSync state within a predetermined or configurable time threshold, then a second type of resynchronization process may be employed at the CG level.

Abstract: Systems and methods are described for a non-disruptive planned failover from a primary copy of data at a primary storage cluster to a mirror copy of the data at a cross-site secondary storage cluster without using an external mediator. According to an example, a planned failover feature of a multi-site distributed storage system provides an order of operations such that a primary copy of a first data center continues to serve I/O operations until a mirror copy of a second data center is ready. This planned failover feature improves functionality and efficiency of the distributed storage system by providing non-disruptiveness during planned failover without using an external mediator based on a primary storage cluster being selected as an authority to implement a state machine with a persistent configuration database to track a planned failover state for the planned failover.

Abstract: Systems and methods are described for efficiently processing events related to a relationship between a primary copy of data at a primary storage system and a mirror copy of the data at a cross-site secondary storage system of a multi-site distributed storage system. According to an example, a mediator agent that is configured on both primary and secondary storage systems provides coordination and serialization for various events generated in the relationship. The multi-site distributed storage system performs actions based on the event processing such as performing a failover operation from the primary storage system to the secondary storage system or resynchronizing the relationship to ensure application protection and availability.

Abstract: Systems and methods are described for a non-disruptive planned failover from a primary copy of data at a primary storage system to a mirror copy of the data at a cross-site secondary storage system. According to an example, a planned failover feature of a multi-site distributed storage system provides an order of operations such that a primary copy of a first data center continues to serve I/O operations until a mirror copy of a second data center is ready. This planned failover feature improves functionality and efficiency of the distributed storage system by providing non-disruptiveness during planned failover—even if various failures occur. The planned failover feature also includes a persistent fence to avoid serving I/O operations during a timing window when both primary data storage and secondary data storage are attempting to have a master role to serve I/O operations and this avoids a split-brain situation.

Abstract: Systems and methods are described for efficiently processing events related to a relationship between a primary copy of data at a primary storage system and a mirror copy of the data at a cross-site secondary storage system of a multi-site distributed storage system. According to an example, a mediator agent that is configured on both primary and secondary storage systems provides coordination and serialization for various events generated in the relationship. The multi-site distributed storage system performs actions based on the event processing such as performing a failover operation from the primary storage system to the secondary storage system or resynchronizing the relationship to ensure application protection and availability.

Abstract: Systems and methods for making a cross-site storage solution resilient towards mediator unavailability are provided. According to one embodiment, a stretched storage system is operable to bring a mediator associated with a primary and secondary distributed storage system back into the role of an arbitrator for peered consistency groups (CGs). A mediator reseed status indicator is maintained for multiple CGs to identify when the mediator's status information for a CG is stale. When the mediator becomes available and a local CG is identified as the subject of a mediator reseed process, the master node of the primary that hosts a master copy of a dataset for the local CG performs the reseed process, including: (i) causing relationship status information for the local CG to be updated on the mediator to the current state maintained by the primary; and (ii) resetting the mediator reseed status indicator.