Abstract: Techniques are provided for managing metadata of a storage system. A storage control system manages a storage metadata structure which comprises metadata items for accessing data items in a storage system comprising a persistent write cache and a primary storage. The storage metadata structure comprises a first metadata structure that is configured to organize metadata items received from the write cache at a first granularity, and a second metadata structure that is configured to organize metadata items received from the first metadata structure at a second granularity, wherein the second granularity is greater than the first granularity. The storage control system utilizes the storage metadata structure to access data items in the persistent write cache and the primary storage.

Abstract: An apparatus comprises a processing device configured to receive a write request to write a given portion of data to a storage system comprising a multiple-instance write cache, the multiple-instance write cache comprising a first write cache instance that utilizes replica-based data protection and a second write cache instance that utilizes data striping-based data protection, and to determine a size of the given data portion and to compare the size of the given data portion to at least one size threshold. The processing device is also configured, responsive to a first comparison result, to write the given data portion to the first write cache instance. The processing device is further configured, responsive to a second comparison result different than the first comparison result, to write at least part of the given data portion to the second write cache instance.

Abstract: An apparatus comprises a processing device configured to initiate garbage collection for data pages stored in local storage of a storage node of a storage system. The processing device is also configured to determine, for a given data page stored in the local storage of the storage node, a validity score characterizing a size of changed data in the given data page, and to compare the validity score for the given data page to at least one designated threshold. The processing device is further configured to update a given page object for the given data page in an object store of persistent storage responsive to a first comparison result, and to generate, in the object store of the persistent storage, a page delta object for the given data page responsive to a second comparison result, the page delta object comprising the changed data in the given data page.

Abstract: A storage control system receives a request to access target data and searches a primary metadata structure to find an associated metadata entry. The primary metadata structure comprises a log-structured merge tree comprising a first level of segments which comprises a group of segments having an associated group filter, and individual filters associated with respective segments of the group of segments. The storage control system searches the primary metadata structure by performing a lookup operation using the group filter to determine whether the associated metadata entry is potentially present in a given segment of the group of segments, and performing a lookup operation using the individual filters to identify the given segment of the group of segments which potentially includes the metadata entry, when the metadata entry is determined to be potentially present in a given segment of group of segments, using the group filter.

Abstract: An apparatus comprises a processing device configured to receive, from one of two or more storage nodes of a software-defined storage system comprising a write cache and persistent storage, a request to read a given portion of data. One or more portions of data in the persistent storage are also stored as read-only data in local storage of virtual computing instances implementing the storage nodes. The processing device is configured to read the given portion of data from the local storage of the virtual computing instances responsive to determining that the given portion of data is part of the read-only data in the local storage of the virtual computing instances, and to read the given portion of data from the write cache or persistent storage responsive to determining that the given portion of data is not part of the read-only data in the local storage of the virtual computing instances.

Abstract: A storage control system manages a storage metadata structure which comprises first and second tree structures. The first tree structure is configured to accumulate metadata entries associated with newly written data items, and sort the accumulated metadata entries by index keys. The second tree structure is configured to organize metadata entries using an index structure that enables random-access to the metadata entries using the index keys. The storage control system performs a merging process to merge metadata entries in leaf levels of the first and second tree structures, and performs a tree construction process to construct a third tree structure by populating a leaf level of the third tree structure with merged metadata entries from the leaf levels of the first and second tree structures. The storage metadata structure is updated to comprise the first tree structure, and the third tree structure in place of the second tree structure.

Abstract: A storage control system manages a storage metadata structure which comprises first and second tree structures. The first tree structure is configured to accumulate metadata entries associated with newly written data items, and sort the accumulated metadata entries by index keys. The second tree structure is configured to organize metadata entries using an index structure that enables random-access to the metadata entries using the index keys. The storage control system performs a merging process to merge metadata entries in leaf levels of the first and second tree structures, and performs a tree construction process to construct a third tree structure by populating a leaf level of the third tree structure with merged metadata entries from the leaf levels of the first and second tree structures. The storage metadata structure is updated to comprise the first tree structure, and the third tree structure in place of the second tree structure.

Abstract: Techniques are provided for managing metadata of a storage system. A storage control system manages a storage metadata structure which comprises metadata items for accessing data items in a storage system comprising a persistent write cache and a primary storage. The storage metadata structure comprises a first metadata structure that is configured to organize metadata items received from the write cache at a first granularity, and a second metadata structure that is configured to organize metadata items received from the first metadata structure at a second granularity, wherein the second granularity is greater than the first granularity. The storage control system utilizes the storage metadata structure to access data items in the persistent write cache and the primary storage.

Abstract: A garbage collection process is performed in a storage system which comprises a storage control node, and storage nodes which implement a striped volume comprising a plurality of stripes having strips that are distributed over the storage nodes. The storage control node selects a victim stripe for garbage collection, and an empty stripe in the striped volume. The storage control node determines a data strip of the victim stripe having predominantly valid data based on a specified threshold, and sends a copy command to a target storage node which comprises the predominantly valid data strip, to cause the target storage node to copy the predominantly valid data strip to a data strip of the empty stripe which resides on the target storage node. The storage control node writes valid data blocks of the victim stripe to remaining data strips of the empty stripe, and releases the victim stripe for reuse.

Abstract: A storage control system receives a request to access target data and searches a primary metadata structure to find an associated metadata entry. The primary metadata structure comprises a log-structured merge tree comprising a first level of segments which comprises a group of segments having an associated group filter, and individual filters associated with respective segments of the group of segments. The storage control system searches the primary metadata structure by performing a lookup operation using the group filter to determine whether the associated metadata entry is potentially present in a given segment of the group of segments, and performing a lookup operation using the individual filters to identify the given segment of the group of segments which potentially includes the metadata entry, when the metadata entry is determined to be potentially present in a given segment of group of segments, using the group filter.

Abstract: Techniques are provided to recover from partial device errors of storage devices in a data storage system. A storage control system manages a storage device which comprises storage capacity that is logically partitioned into segments of equal size. The storage control system groups at least some of the segments of the storage device into a segment group. Each segment of the segment group is configured to store one or more data items and associated metadata items. The storage control system generates a parity data segment based on the segments of the segment group, and persistently stores the parity data segment in association with the segment group. In response to detecting a storage device error associated with a corrupted segment of the segment group, the storage control system utilizes the parity data segment and non-corrupted segments of the segment group to recover at least one missing data item of the corrupted segment.

Abstract: Techniques are provided for managing load distribution in a data storage system which comprises storage sever nodes and storage nodes, wherein each storage server node comprises one or more distribution targets, wherein each distribution target comprises a logical processing unit that is configured to manage input/output operations for accessing data in the storage nodes. A process for managing the load distribution comprises: partitioning a logical address space into a plurality of distribution items; mapping the logical address space to a sequence of one or more distribution templates, wherein each distribution template comprises an ordered list of the distribution targets of the storage server nodes, which is uniquely different for each of the one or more distribution templates; and utilizing the one or more distribution templates to map the distribution items of the logical address space to the distribution targets of the storage server nodes.

Abstract: An apparatus comprises a processing device configured to maintain a snapshot mapping structure for a storage volume. The snapshot mapping structure comprises objects representing the storage volume and snapshots thereof. The objects are organized in a tree structure representing data dependencies between the storage volume and its snapshots. Each of the objects comprises a dynamic bitmap indicating whether that object contains new data for different offset positions relative to its ancestor objects in the tree structure. The processing device is further configured to detect operations directed to a specified one of the storage volume or one of its snapshots, to parse the snapshot mapping structure to identify objects to utilize for processing the operations, and to processing the operations utilizing the dynamic bitmaps of the identified objects.

Abstract: An apparatus comprises a processing device configured to maintain a snapshot mapping structure for a storage volume. The snapshot mapping structure comprises objects representing the storage volume and snapshots thereof. The objects are organized in a tree structure representing data dependencies between the storage volume and its snapshots. Each of the objects comprises a dynamic bitmap indicating whether that object contains new data for different offset positions relative to its ancestor objects in the tree structure. The processing device is further configured to detect operations directed to a specified one of the storage volume or one of its snapshots, to parse the snapshot mapping structure to identify objects to utilize for processing the operations, and to processing the operations utilizing the dynamic bitmaps of the identified objects.

Abstract: An apparatus comprises a processing device configured to receive a request to perform merging in a snapshot mapping structure comprising objects representing a given storage volume and its snapshots. The objects are organized in a tree structure representing data dependencies between the given storage volume and its snapshots. The processing device is also configured to analyze the snapshot mapping structure to identify ones of the objects that are candidates for deletion and to determine merge groups. A given one of the merge groups includes (i) at least a subset of the objects identified as candidates for deletion and (ii) a given object that is a child object of at least one of the subset of the objects identified as candidates for deletion. The processing device is further configured to perform merging in the snapshot mapping structure by merging data items in the given merge group to one of its objects.

Abstract: A storage control system receives a first data block to be written to a primary storage, and generates a content signature for the first data block. The storage control system adds a first entry for the first data block into a persistent deduplication database. The first entry comprises a key which comprises the content signature for the first data block. The persistent deduplication database comprises a tree data structure which comprises elements that are configured to store entries for data blocks. The storage control system merges the entries of at least two elements of the tree data structure to generate a set of merged entries which comprises the first entry for the first data block, and a second entry for a second data block, and commences a deduplication process in response to determining that the first entry and the second entry in the set of merged entries have matching keys.

Abstract: An apparatus comprises at least a first storage node of a plurality of storage nodes of a distributed storage system. The first storage node comprises a processor coupled to a memory. The first storage node is configured to receive from a host device a read operation directed to a particular portion of a logical address space of the storage system, where the logical address space is divided among the storage nodes. The first storage node is further configured to determine that the read operation is potentially part of a stream of multiple read operations directed to respective particular portions of the logical address space, and to send, to at least a second one of the storage nodes, an indication of the determination made by the first storage node. The stream of multiple read operations may comprise a sequential read stream directed to respective contiguous portions of the logical address space.

Abstract: A garbage collection process is performed in a storage system which comprises a storage control node, and storage nodes which implement a striped volume comprising a plurality of stripes having strips that are distributed over the storage nodes. The storage control node selects a victim stripe for garbage collection, and an empty stripe in the striped volume. The storage control node determines a data strip of the victim stripe having predominantly valid data based on a specified threshold, and sends a copy command to a target storage node which comprises the predominantly valid data strip, to cause the target storage node to copy the predominantly valid data strip to a data strip of the empty stripe which resides on the target storage node. The storage control node writes valid data blocks of the victim stripe to remaining data strips of the empty stripe, and releases the victim stripe for reuse.

Abstract: An apparatus includes at least one processing device configured to establish a plurality of paths between at least one initiator of a host device and a plurality of targets of respective storage nodes of a distributed storage system, and for each of a plurality of input-output operations generated in the host device for delivery to the distributed storage system: to access a target lookup service of the host device to determine a particular one of the storage nodes that stores data for a logical storage volume and offset targeted by the input-output operation, to select a particular one of the plurality of paths from the initiator to one of the targets on the particular storage node, and to send the input-output operation to the particular storage node over the selected path. The initiator and the targets are illustratively configured in accordance with a designated standard storage access protocol.

Abstract: A storage control system receives a first data block to be written to a primary storage, and generates a content signature for the first data block. The storage control system adds a first entry for the first data block into a persistent deduplication database. The first entry comprises a key which comprises the content signature for the first data block. The persistent deduplication database comprises a tree data structure which comprises elements that are configured to store entries for data blocks. The storage control system merges the entries of at least two elements of the tree data structure to generate a set of merged entries which comprises the first entry for the first data block, and a second entry for a second data block, and commences a deduplication process in response to determining that the first entry and the second entry in the set of merged entries have matching keys.