Abstract: Techniques are disclosed relating to restarting a database node. A database node may allocate memory segments that include a restart segment for storing data records. The database node may spawn processes to read a log and replay log records of the log to update the restart segment to store data records. The database node may determine to perform a restart operation to transition from a first mode to a second mode. Performing the restart operation may include ceasing reading the log at a stop position and storing, based on the stop position, database state information that enables the processes to resume reading the log from the stop position. The database node may further deallocate the memory segments except for the restart segment and terminate the processes. After performing the restart operation, the database node may spawn the processes, which may resume reading the log based on the database state information.

Abstract: A storage server includes a first controller, a second controller and physical drives. The first controller receives a request to store data in a virtual block address (VBA) of a virtual drive, determines a physical block address (PBA) of a physical drive on which to store the data, and sends to the physical drive a command to store the data at the PBA. The first controller generates a first metadata update for a first metadata table associated with the virtual drive, wherein the first metadata update indicates a mapping of the VBA to the PBA. The first controller generates a second metadata update for a second metadata table that identifies statuses of PBAs, wherein the second metadata update indicates that the PBA is valid. The first controller writes an entry to a cache in a memory, the entry comprising the first metadata update and the second metadata update.

Abstract: A method of converting a data format in a storage system is provided. The method includes identifying a condition for when to make a data conversion from a write-optimized virtual disk format to a read-optimized virtual disk format; and locating, in response to the identified condition, a version of data. The method may include moving the version of data to a thin-provisioned format in-place virtual disk and detecting a set of overwrites from a remaining set of versions of the data, including blocks of data having been previously overwritten. The method may further include compacting each block within the set of overwrites to create a contiguous block such that an associated physical memory space is reclaimed, wherein the read-optimized virtual disk format of the data includes the thin-provisioned format in-place virtual disk and the contiguous block.

Abstract: The disclosed computer-implemented method for performing data replication in distributed cluster environments may include (1) identifying a distributed cluster environment that includes (A) a plurality of compute nodes that execute a plurality of virtual machines and (B) a data node that stores data that has been replicated from storage devices used by the virtual machines, (2) determining, at the data node, storage-utilization totals for the virtual machines that represent amounts of storage space on the storage devices used by the virtual machines, (3) identifying, based at least in part on the storage-utilization totals, a virtual machine whose storage-utilization total is highest among the plurality of virtual machines, (4) prioritizing the virtual machine and then in response to the prioritization, (5) directing the compute node to initiate a data replication process with the data node in connection with the virtual machine. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: The disclosed computer-implemented method for making snapshots available may include (i) identifying a writeback log that records input/output operations of a compute node within a high-availability environment, (ii) placing, in the writeback log, a marker that indicates a start of a snapshot to be stored on a data node, (iii) marking, after placing the marker and before all data within the snapshot has been transferred to the data node, the snapshot as available, (iv) receiving, from an additional compute node, a request to read from the snapshot, and (v) sending, from the compute node to the additional compute node, metadata indicating which portion of data from the snapshot is stored on the data node and which portion of the data from the snapshot is not stored on the data node but is stored in the writeback log. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A storage server includes an IO controller, a management controller and physical drives. The IO controller generates multiple metadata updates and writes a cache entry that includes the multiple metadata updates to a first cache in memory of the management controller. The IO controller additionally writes a copy of the cache entry to a second cache in a memory of the IO controller and increments a commit pointer in the first and second caches to indicate that the metadata updates are committed.

Abstract: A storage server includes an IO controller, a management controller and physical drives. The IO controller generates multiple metadata updates and writes a cache entry that includes the multiple metadata updates to a first cache in memory of the management controller. The IO controller additionally writes a copy of the cache entry to a second cache in a memory of the IO controller and increments a commit pointer in the first and second caches to indicate that the metadata updates are committed.

Abstract: A storage server includes a first controller, a second controller and physical drives. The first controller receives a request to store data in a virtual block address (VBA) of a virtual drive, determines a physical block address (PBA) of a physical drive on which to store the data, and sends to the physical drive a command to store the data at the PBA. The first controller generates a first metadata update for a first metadata table associated with the virtual drive, wherein the first metadata update indicates a mapping of the VBA to the PBA. The first controller generates a second metadata update for a second metadata table that identifies statuses of PBAs, wherein the second metadata update indicates that the PBA is valid. The first controller writes an entry to a cache in a memory, the entry comprising the first metadata update and the second metadata update.

Abstract: Write operations are scheduled for multiple nodes in a shared storage cluster that supports volume replication. Requests are received from nodes for allocation of space for write operations in a replication log. In response to a received request, the current capacity of the requesting node to manage a backlog can be determined. The amount of space in the replication log allocated to the node is then calibrated to the node's capacity, thereby preventing self-throttling. A separate priority can be assigned to each volume, and space in the replication log assigned to each volume based on its priority. Nodes can target synchronous and other latency sensitive operations to higher priority volumes. A single global queue can be maintained to schedule write operations for all nodes, thereby providing a fair scheduling. A separate local queue can be maintained for each node, thereby providing specific levels of preference to specific nodes.

Abstract: Various systems, methods, and processes to perform adaptive throttling of write operations in hybrid storage environments are disclosed. A credit value is assigned to a virtual machine. Input/output (I/O) operations generated by an application executing on the virtual machine are throttled based on the credit value assigned to the virtual machine. The credit value is assigned to the virtual machine in proportion to static measures and dynamic conditions. The static measures and the dynamic conditions are used to calculate a quality of service provided to the virtual machine by a storage system.

Abstract: A system and method for minimizing duplicate data transfer in a clustered storage system, having compute nodes in a compute plane coupled to data nodes in a data plane is provided. The method may include generating a hash key relating to content of a virtual disk associated with a compute node. During a data replication phase, the method may detect duplicate data stored in respective storage units of the compute node and the data node using the hash key. Further, the method may eliminate redundant data transfers through the use of an index and mapping scheme, where only non-duplicate data is transferred along with a set of logical block addresses associated with duplicate data from the replicating compute node to the data node. During a data recovery phase, the method may transfer duplicate data from a peer compute node or from a virtual machine to the requesting compute node, eliminating excess data transfer.

Abstract: Write operations are scheduled for multiple nodes in a shared storage cluster that supports volume replication. Requests are received from nodes for allocation of space for write operations in a replication log. In response to a received request, the current capacity of the requesting node to manage a backlog can be determined. The amount of space in the replication log allocated to the node is then calibrated to the node's capacity, thereby preventing self-throttling. A separate priority can be assigned to each volume, and space in the replication log assigned to each volume based on its priority. Nodes can target synchronous and other latency sensitive operations to higher priority volumes. A single global queue can be maintained to schedule write operations for all nodes, thereby providing a fair scheduling. A separate local queue can be maintained for each node, thereby providing specific levels of preference to specific nodes.

Abstract: Write operations are scheduled for multiple nodes in a shared storage cluster that supports volume replication. Requests are received from nodes for allocation of space for write operations in a replication log. In response to a received request, the current capacity of the requesting node to manage a backlog can be determined. The amount of space in the replication log allocated to the node is then calibrated to the node's capacity, thereby preventing self-throttling. A separate priority can be assigned to each volume, and space in the replication log assigned to each volume based on its priority. Nodes can target synchronous and other latency sensitive operations to higher priority volumes. A single global queue can be maintained to schedule write operations for all nodes, thereby providing a fair scheduling. A separate local queue can be maintained for each node, thereby providing specific levels of preference to specific nodes.

Abstract: Write operations are scheduled for multiple nodes in a shared storage cluster that supports volume replication. Requests are received from nodes for allocation of space for write operations in a replication log. In response to a received request, the current capacity of the requesting node to manage a backlog can be determined. The amount of space in the replication log allocated to the node is then calibrated to the node's capacity, thereby preventing self-throttling. A separate priority can be assigned to each volume, and space in the replication log assigned to each volume based on its priority. Nodes can target synchronous and other latency sensitive operations to higher priority volumes. A single global queue can be maintained to schedule write operations for all nodes, thereby providing a fair scheduling. A separate local queue can be maintained for each node, thereby providing specific levels of preference to specific nodes.

Abstract: A computer-implemented method for distributing replication tasks within computing clusters may include (1) identifying a primary volume that is replicated to a secondary volume, (2) identifying a computing cluster with access to the primary volume that includes at least a first node and a second node, (3) receiving a request to write data to the primary volume, (4) logging, via the first node, the request to write the data to the primary volume to a replication log, and (5) using the replication log to replicate, via the second node, the data to the secondary volume. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A computer-implemented method for distributing replication tasks within computing clusters may include (1) identifying a primary volume that is replicated to a secondary volume, (2) identifying a computing cluster with access to the primary volume that includes at least a first node and a second node, (3) receiving a request to write data to the primary volume, (4) logging, via the first node, the request to write the data to the primary volume to a replication log, and (5) using the replication log to replicate, via the second node, the data to the secondary volume. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A reconfiguration is distributed among multiple nodes of a cluster. Upon detecting an initiation of a reconfiguration of the cluster, reconfiguration functionality is delegated from the master node to one or more slave nodes in the cluster. Thus, multiple nodes perform reconfiguration functionality in parallel, such that one or more slave nodes perform delegated reconfiguration tasks that would conventionally be performed by the master node. The cluster reconfiguration can be in the form of one or more nodes joining or leaving the cluster. Data to facilitate the cluster reconfiguration can be transmitted from the master node to a slave node to which reconfiguration functionality is being delegated. Such data can include, for example, identifiers of nodes joining or leaving the cluster and/or information concerning the architecture and shared storage media of the cluster.

Abstract: A computer-implemented method for achieving file-level data-protection operations using block-level technologies may include: 1) identifying an I/O operation directed to at least one data block on a primary storage system, 2) accessing a data-protection list that uniquely identifies files that are to be protected by a data-protection operation, 3) determining that the I/O operation is directed to at least a portion of at least one file identified on the data-protection list, and then, in response to the determination, 4) performing the data-protection operation on the I/O operation. Various additional methods, systems, and configured computer-readable media are also disclosed.