Abstract: A technique includes causing an agent device to setup a replication partnership between a first storage node and a second storage. Causing the agent device to setup the replication partnership includes configuring a proxy server that is associated with the second storage node to establish a secure communication channel for the replication partnership over a public network. Configuring the proxy server includes storing in the proxy server credentials for authenticating the first storage node to use the secure communication channel; and establishing port translations to be used in the secure communication channel in communicating replication data between the first storage node and the second storage node. Causing the agent device to setup the replication partnership may also include communicating replication partnership information to the second node.

Abstract: A technique for reducing recovery time in a computing apparatus includes: storing data in a key-value store residing in a persistent memory, the key-value store including a plurality of persistent nodes including key-value pairs, the key-value pairs of the persistent nodes being unsorted; caching selected persistent nodes in a volatile memory, the key-value pairs of the cached nodes being sorted, the caching including instantiating a dual pointer for each cached node, the dual pointer including a volatile pointer to a child node of the cached node and a persistent pointer to the child node of the corresponding persistent node in the key-value store; and rebuilding the cache of selected nodes upon demand using the dual pointers in the event of recovery.

Abstract: A technique includes causing an agent device to setup a replication partnership between a first storage node and a second storage. Causing the agent device to setup the replication partnership includes configuring a proxy server that is associated with the second storage node to establish a secure communication channel for the replication partnership over a public network. Configuring the proxy server includes storing in the proxy server credentials for authenticating the first storage node to use the secure communication channel; and establishing port translations to be used in the secure communication channel in communicating replication data between the first storage node and the second storage node. Causing the agent device to setup the replication partnership may also include communicating replication partnership information to the second node.

Abstract: Examples herein relate to re-synchronizing data between an upstream volume and a downstream volume, including by communicating data to the downstream volume, from a start point to an end point of the upstream volume, maintaining a first cursor to indicate data, from the start point to the first cursor, that has been communicated to and committed to persistent storage of the downstream volume, and maintaining a second cursor to indicate data, from the first cursor to the second cursor, that has been communicated to the downstream volume and buffered without yet being confirmed as committed to persistent storage of the downstream volume.

Abstract: One witness node may provide quorum witness functionality for multiple quorums concurrently (e.g., while multiple quorums are executing independently and simultaneously). An aggregated heartbeat message from a first quorum actor of the first quorum may include a first quorum heartbeat message that has a unique identifier for the first quorum. This identifier may allow update for the quorum data store with respect to the first quorum. The aggregated heartbeat message may also include information representing a second quorum heartbeat message from a second quorum actor of a second quorum. The second quorum heartbeat message may include a second unique identifier for the second quorum. The second identifier may allow an update for the quorum data store independently of the first update. More than two quorums may be supported by a single witness node by extracting individualized information from an aggregated heartbeat message.

Abstract: This disclosure is directed to heartbeat messages for a high-availability redundant distributed computing environment (e.g., a quorum data storage implementation). Heartbeat messages may be sent from a node to other members of a duster as an indication of hardware state, network connectivity, and possibly application status on the sending node. If a member of a cluster (or quorum) goes silent (e.g., misses heartbeats), other members of the duster (or quorum) may consider that member (or the node that hosts that member) to be non-functional and may initiate a recovery action. Techniques are disclosed for using low-latency non-persistent storage for some heartbeat messages (referred to herein as a “non-persistent heartbeat messages”) to replace a portion of typical persistent (e.g., disk-based) heartbeat messages to reduce overall processing for periodic heartbeat messages. Further, implementations that aggregate multiple heartbeat messages from a node into a fewer number of heartbeat messages are disclosed.

Abstract: Examples described herein include receiving a first data write request from a host computing device, assigning a first identification to the first data write request, recording the first identification in in-flight data, transmitting the first data write request to a second storage array, receiving a first acknowledgement of the first data write request from the second storage array, and recording an indicator representing a group of acknowledgements. In some examples, the group of acknowledgements comprises the first acknowledgment.

Abstract: A method, system, and non-transitory computer readable medium are disclosed to regarding remote procedure call (RPC) functionality from a requestor node to a processing node using a quorum state store communication protocol. Records from a quorum state store may be augmented to include an indication of both the processing node and the requestor node; remote procedure call (RPC) request information; and information including RPC coordination information. Heartbeat messages may propagate updated quorum state store records throughout the quorum. Requestor nodes and processing nodes may perform requested functions and supply status information via updates to their own quorum state store records. Quorum state store records may be propagated throughout the quorum even when direct communication between requestor nodes and processing nodes may not be available.

Abstract: According to examples, a system may include an upstream volume controller having: a processor and a non-transitory machine-readable storage medium. The storage medium may include instructions executable by the processor to freeze an upstream volume, the upstream volume being in a replication set with a downstream volume, receive a snapshot creation request, create a snapshot of the upstream volume, and send one of a snapshot permit message or a snapshot abort message to a downstream volume processor. The instructions may also be executable by the processor to unfreeze the upstream volume responsive to at least one of the sending of the one of the snapshot permit message or the snapshot abort message or expiration of a timeout corresponding to a maximum time period during which the upstream volume is to remain frozen.

Abstract: According to examples, an apparatus may include a processor and a non-transitory machine-readable storage medium comprising instructions executable by the processor to assign a first object identifier and a data identifier to a first volume, the first object identifier being assigned exclusively to the first volume. The instructions may also be executable by the processor to identify an identifier of a second volume, determined whether the identifier of the second volume matches the data identifier, and based on a determination that the identifier of the second volume matches the data identifier of the first volume, configure a replication relationship between the first volume and the second volume.

Abstract: A technique for reducing recovery time in a computing apparatus includes: storing data in a key-value store residing in a persistent memory, the key-value store including a plurality of persistent nodes including key-value pairs, the key-value pairs of the persistent nodes being unsorted; caching selected persistent nodes in a volatile memory, the key-value pairs of the cached nodes being sorted, the caching including instantiating a dual pointer for each cached node, the dual pointer including a volatile pointer to a child node of the cached node and a persistent pointer to the child node of the corresponding persistent node in the key-value store; and rebuilding the cache of selected nodes upon demand using the dual pointers in the event of recovery.

Abstract: This disclosure is directed to heartbeat messages for a high-availability redundant distributed computing environment (e.g., a quorum data storage implementation). Heartbeat messages may be sent from a node to other members of a duster as an indication of hardware state, network connectivity, and possibly application status on the sending node. If a member of a cluster (or quorum) goes silent (e.g., misses heartbeats), other members of the duster (or quorum) may consider that member (or the node that hosts that member) to be non-functional and may initiate a recovery action. Techniques are disclosed for using low-latency non-persistent storage for some heartbeat messages (referred to herein as a “non-persistent heartbeat messages”) to replace a portion of typical persistent (e.g., disk-based) heartbeat messages to reduce overall processing for periodic heartbeat messages. Further, implementations that aggregate multiple heartbeat messages from a node into a fewer number of heartbeat messages are disclosed.

Abstract: One witness node may provide quorum witness functionality for multiple quorums concurrently (e.g., while multiple quorums are executing independently and simultaneously). An aggregated heartbeat message from a first quorum actor of the first quorum may include a first quorum heartbeat message that has a unique identifier for the first quorum. This identifier may allow update for the quorum data store with respect to the first quorum. The aggregated heartbeat message may also include information representing a second quorum heartbeat message from a second quorum actor of a second quorum. The second quorum heartbeat message may include a second unique identifier for the second quorum. The second identifier may allow an update for the quorum data store independently of the first update. More than two quorums may be supported by a single witness node by extracting individualized information from an aggregated heartbeat message.

Abstract: According to examples, an apparatus may include a processor and a non-transitory machine-readable storage medium comprising instructions executable by the processor to assign a first object identifier and a data identifier to a first volume, the first object identifier being assigned exclusively to the first volume. The instructions may also be executable by the processor to identify an identifier of a second volume, determined whether the identifier of the second volume matches the data identifier, and based on a determination that the identifier of the second volume matches the data identifier of the first volume, configure a replication relationship between the first volume and the second volume.

Abstract: A method, system, and non-transitory computer readable medium are disclosed to regarding remote procedure call (RPC) functionality from a requestor node to a processing node using a quorum state store communication protocol. Records from a quorum state store may be augmented to include an indication of both the processing node and the requestor node; remote procedure call (RPC) request information; and information including RPC coordination information. Heartbeat messages may propagate updated quorum state store records throughout the quorum. Requestor nodes and processing nodes may perform requested functions and supply status information via updates to their own quorum state store records. Quorum state store records may be propagated throughout the quorum even when direct communication between requestor nodes and processing nodes may not be available.

Abstract: Examples described herein include receiving a first data write request from a host computing device, assigning a first identification to the first data write request, recording the first identification in in-flight data, transmitting the first data write request to a second storage array, receiving a first acknowledgement of the first data write request from the second storage array, and recording an indicator representing a group of acknowledgements. In some examples, the group of acknowledgements comprises the first acknowledgment.

Abstract: According to examples, a system may include an upstream volume controller having: a processor and a non-transitory machine-readable storage medium. The storage medium may include instructions executable by the processor to freeze an upstream volume, the upstream volume being in a replication set with a downstream volume, receive a snapshot creation request, create a snapshot of the upstream volume, and send one of a snapshot permit message or a snapshot abort message to a downstream volume processor. The instructions may also be executable by the processor to unfreeze the upstream volume responsive to at least one of the sending of the one of the snapshot permit message or the snapshot abort message or expiration of a timeout corresponding to a maximum time period during which the upstream volume is to remain frozen.

Abstract: Methods, systems, and programs are presented for replicating data across scale-out storage systems. One method includes replicating, from an upstream to a downstream system, a volume snapshot having one or more bins. Locations for the bins of the snapshot are identified, the location for each bin including the upstream array storing the bin and the downstream array storing a replicated version of the bin. Each bin is validated by comparing an upstream bin checksum of the bin with a downstream bin checksum of the replicated version of the bin. When the checksums are different, a plurality of chunks are defined in the bin, and for each chunk in the bin an upstream chunk checksum calculated by the upstream array is compared with a downstream chunk checksum calculated by the downstream array. The chunk is sent from the upstream to the downstream array when the chunk checksums are different.

Abstract: Examples herein relate to re-synchronizing data between an upstream volume and a downstream volume, including by communicating data to the downstream volume, from a start point to an end point of the upstream volume, maintaining a first cursor to indicate data, from the start point to the first cursor, that has been communicated to and committed to persistent storage of the downstream volume, and maintaining a second cursor to indicate data, from the first cursor to the second cursor, that has been communicated to the downstream volume and buffered without yet being confirmed as committed to persistent storage of the downstream volume.

Abstract: A technique includes causing an agent device to setup a replication partnership between a first storage node and a second storage. Causing the agent device to setup the replication partnership includes configuring a proxy server that is associated with the second storage node to establish a secure communication channel for the replication partnership over a public network. Configuring the proxy server includes storing in the proxy server credentials for authenticating the first storage node to use the secure communication channel; and establishing port translations to be used in the secure communication channel in communicating replication data between the first storage node and the second storage node. Causing the agent device to setup the replication partnership may also include communicating replication partnership information to the second node.