Abstract: An apparatus comprises a processing device configured to receive a write request to write a given portion of data to a storage system comprising a multiple-instance write cache, the multiple-instance write cache comprising a first write cache instance that utilizes replica-based data protection and a second write cache instance that utilizes data striping-based data protection, and to determine a size of the given data portion and to compare the size of the given data portion to at least one size threshold. The processing device is also configured, responsive to a first comparison result, to write the given data portion to the first write cache instance. The processing device is further configured, responsive to a second comparison result different than the first comparison result, to write at least part of the given data portion to the second write cache instance.

Abstract: A storage control system receives a request to access target data and searches a primary metadata structure to find an associated metadata entry. The primary metadata structure comprises a log-structured merge tree comprising a first level of segments which comprises a group of segments having an associated group filter, and individual filters associated with respective segments of the group of segments. The storage control system searches the primary metadata structure by performing a lookup operation using the group filter to determine whether the associated metadata entry is potentially present in a given segment of the group of segments, and performing a lookup operation using the individual filters to identify the given segment of the group of segments which potentially includes the metadata entry, when the metadata entry is determined to be potentially present in a given segment of group of segments, using the group filter.

Abstract: A storage control system manages a storage metadata structure which comprises first and second tree structures. The first tree structure is configured to accumulate metadata entries associated with newly written data items, and sort the accumulated metadata entries by index keys. The second tree structure is configured to organize metadata entries using an index structure that enables random-access to the metadata entries using the index keys. The storage control system performs a merging process to merge metadata entries in leaf levels of the first and second tree structures, and performs a tree construction process to construct a third tree structure by populating a leaf level of the third tree structure with merged metadata entries from the leaf levels of the first and second tree structures. The storage metadata structure is updated to comprise the first tree structure, and the third tree structure in place of the second tree structure.

Abstract: A storage control system manages a storage metadata structure which comprises first and second tree structures. The first tree structure is configured to accumulate metadata entries associated with newly written data items, and sort the accumulated metadata entries by index keys. The second tree structure is configured to organize metadata entries using an index structure that enables random-access to the metadata entries using the index keys. The storage control system performs a merging process to merge metadata entries in leaf levels of the first and second tree structures, and performs a tree construction process to construct a third tree structure by populating a leaf level of the third tree structure with merged metadata entries from the leaf levels of the first and second tree structures. The storage metadata structure is updated to comprise the first tree structure, and the third tree structure in place of the second tree structure.

Abstract: Techniques are provided for managing metadata of a storage system. A storage control system manages a storage metadata structure which comprises metadata items for accessing data items in a storage system comprising a persistent write cache and a primary storage. The storage metadata structure comprises a first metadata structure that is configured to organize metadata items received from the write cache at a first granularity, and a second metadata structure that is configured to organize metadata items received from the first metadata structure at a second granularity, wherein the second granularity is greater than the first granularity. The storage control system utilizes the storage metadata structure to access data items in the persistent write cache and the primary storage.

Abstract: A garbage collection process is performed in a storage system which comprises a storage control node, and storage nodes which implement a striped volume comprising a plurality of stripes having strips that are distributed over the storage nodes. The storage control node selects a victim stripe for garbage collection, and an empty stripe in the striped volume. The storage control node determines a data strip of the victim stripe having predominantly valid data based on a specified threshold, and sends a copy command to a target storage node which comprises the predominantly valid data strip, to cause the target storage node to copy the predominantly valid data strip to a data strip of the empty stripe which resides on the target storage node. The storage control node writes valid data blocks of the victim stripe to remaining data strips of the empty stripe, and releases the victim stripe for reuse.

Abstract: A storage control system receives a request to access target data and searches a primary metadata structure to find an associated metadata entry. The primary metadata structure comprises a log-structured merge tree comprising a first level of segments which comprises a group of segments having an associated group filter, and individual filters associated with respective segments of the group of segments. The storage control system searches the primary metadata structure by performing a lookup operation using the group filter to determine whether the associated metadata entry is potentially present in a given segment of the group of segments, and performing a lookup operation using the individual filters to identify the given segment of the group of segments which potentially includes the metadata entry, when the metadata entry is determined to be potentially present in a given segment of group of segments, using the group filter.

Abstract: Techniques are provided for managing load distribution in a data storage system which comprises storage sever nodes and storage nodes, wherein each storage server node comprises one or more distribution targets, wherein each distribution target comprises a logical processing unit that is configured to manage input/output operations for accessing data in the storage nodes. A process for managing the load distribution comprises: partitioning a logical address space into a plurality of distribution items; mapping the logical address space to a sequence of one or more distribution templates, wherein each distribution template comprises an ordered list of the distribution targets of the storage server nodes, which is uniquely different for each of the one or more distribution templates; and utilizing the one or more distribution templates to map the distribution items of the logical address space to the distribution targets of the storage server nodes.

Abstract: An apparatus comprises a processing device configured to maintain a snapshot mapping structure for a storage volume. The snapshot mapping structure comprises objects representing the storage volume and snapshots thereof. The objects are organized in a tree structure representing data dependencies between the storage volume and its snapshots. Each of the objects comprises a dynamic bitmap indicating whether that object contains new data for different offset positions relative to its ancestor objects in the tree structure. The processing device is further configured to detect operations directed to a specified one of the storage volume or one of its snapshots, to parse the snapshot mapping structure to identify objects to utilize for processing the operations, and to processing the operations utilizing the dynamic bitmaps of the identified objects.

Abstract: An apparatus comprises a processing device configured to maintain a snapshot mapping structure for a storage volume. The snapshot mapping structure comprises objects representing the storage volume and snapshots thereof. The objects are organized in a tree structure representing data dependencies between the storage volume and its snapshots. Each of the objects comprises a dynamic bitmap indicating whether that object contains new data for different offset positions relative to its ancestor objects in the tree structure. The processing device is further configured to detect operations directed to a specified one of the storage volume or one of its snapshots, to parse the snapshot mapping structure to identify objects to utilize for processing the operations, and to processing the operations utilizing the dynamic bitmaps of the identified objects.

Abstract: An apparatus comprises a processing device configured to receive a request to perform merging in a snapshot mapping structure comprising objects representing a given storage volume and its snapshots. The objects are organized in a tree structure representing data dependencies between the given storage volume and its snapshots. The processing device is also configured to analyze the snapshot mapping structure to identify ones of the objects that are candidates for deletion and to determine merge groups. A given one of the merge groups includes (i) at least a subset of the objects identified as candidates for deletion and (ii) a given object that is a child object of at least one of the subset of the objects identified as candidates for deletion. The processing device is further configured to perform merging in the snapshot mapping structure by merging data items in the given merge group to one of its objects.

Abstract: A storage control system receives a first data block to be written to a primary storage, and generates a content signature for the first data block. The storage control system adds a first entry for the first data block into a persistent deduplication database. The first entry comprises a key which comprises the content signature for the first data block. The persistent deduplication database comprises a tree data structure which comprises elements that are configured to store entries for data blocks. The storage control system merges the entries of at least two elements of the tree data structure to generate a set of merged entries which comprises the first entry for the first data block, and a second entry for a second data block, and commences a deduplication process in response to determining that the first entry and the second entry in the set of merged entries have matching keys.

Abstract: A garbage collection process is performed in a storage system which comprises a storage control node, and storage nodes which implement a striped volume comprising a plurality of stripes having strips that are distributed over the storage nodes. The storage control node selects a victim stripe for garbage collection, and an empty stripe in the striped volume. The storage control node determines a data strip of the victim stripe having predominantly valid data based on a specified threshold, and sends a copy command to a target storage node which comprises the predominantly valid data strip, to cause the target storage node to copy the predominantly valid data strip to a data strip of the empty stripe which resides on the target storage node. The storage control node writes valid data blocks of the victim stripe to remaining data strips of the empty stripe, and releases the victim stripe for reuse.

Abstract: A storage control system receives a first data block to be written to a primary storage, and generates a content signature for the first data block. The storage control system adds a first entry for the first data block into a persistent deduplication database. The first entry comprises a key which comprises the content signature for the first data block. The persistent deduplication database comprises a tree data structure which comprises elements that are configured to store entries for data blocks. The storage control system merges the entries of at least two elements of the tree data structure to generate a set of merged entries which comprises the first entry for the first data block, and a second entry for a second data block, and commences a deduplication process in response to determining that the first entry and the second entry in the set of merged entries have matching keys.

Abstract: At least one processor is configured to obtain measurement information comprising an indication of an amount of utilization of a hardware resource of a first server node by a plurality of processing groups and to determine that the amount of utilization of the hardware resource is above a threshold amount of utilization. The at least one processor is further configured to select a given processing group for redistribution based at least in part on the determination that the amount of utilization of the hardware resource is above the threshold amount and on an amount of utilization of the hardware resource by the given processing group. The at least one processor is further configured to determine that a second server node comprises enough available capacity of the hardware resource and to redistribute the given processing group to the second server node based at least in part on the determination.

Abstract: A storage control node receives data to be written to a striped volume, allocates first and second stripes, writes the data to at least one data strip of the first stripe, computes parity data based on the data written to the first stripe, and writes the parity data to the first stripe. The storage control node sends a copy command to a target storage node which comprises the at least one data strip of the first stripe to thereby cause the at least one data strip to be copied to a data strip of the second stripe which resides on the target storage node. The storage control node writes additional data to the second stripe, computes updated parity data based on the additional data and the parity data of the first stripe, writes the updated parity data the second stripe, and releases the first stripe for reuse.

Abstract: A storage control system manages a utilization of data blocks of a storage volume which is partitioned into data blocks having a unique block identifier (ID) and a same block size. The storage control system receives data items and assigns a respective unique data ID to each data item, which include consecutive data IDs. The data items are written to a free data block as a whole, and a record for the written data block is inserted into a node of a first tree structure. The record includes the unique block ID of the written data block, a first data ID of the data items, and a bitmap which maps the consecutive data IDs of the data items in the written data block, starting from the first data ID, to a respective bit whose value indicates whether the data item associated with the data ID is valid or invalid.

Abstract: A storage control system maintains a write cache in a non-volatile memory device of primary memory of a storage node. The write cache comprises a cyclic buffer and pointers to manage the write cache and track a tail location and head location of the write cache. The storage control system receives a write request from a host system, which comprises a data item to be written to primary storage. The received data item is written together with an associated metadata item at the head location of the write cache. The items in the write cache are arranged in a cyclic write order from the tail location to the head location. The storage control system sends an acknowledgment to the host system that the data item is successfully written to the primary storage, in response to the received data item and the associated metadata item being stored in the write cache.

Abstract: A storage control system maintains a write cache in a non-volatile memory device of primary memory of a storage node. The write cache comprises a cyclic buffer and pointers to manage the write cache and track a tail location and head location of the write cache. The storage control system receives a write request from a host system, which comprises a data item to be written to primary storage. The received data item is written together with an associated metadata item at the head location of the write cache. The items in the write cache are arranged in a cyclic write order from the tail location to the head location. The storage control system sends an acknowledgment to the host system that the data item is successfully written to the primary storage, in response to the received data item and the associated metadata item being stored in the write cache.

Abstract: A storage control system receives a first write request and a second write request following the first write request. The first and second write requests comprise respective first and second data items for storage to a primary storage. First and second cache write operations are performed in parallel to write the first and second data items a persistent write cache. The first cache write operation comprises a split write operation which comprises writing a parsing header for the first data item to the write cache, and writing a payload of the first data item to the write cache. The second cache write operation comprises storing the second data item and associated metadata in the write cache, and waiting for an acknowledgment that the parsing header for the first data item has been successfully stored in the write cache before returning an acknowledgment indicating successful storage of the second data item.