Abstract: Techniques are provided for compacting indirect blocks. For example, an object is represented as a structure comprising data blocks within which data of the object is stored and indirect blocks comprising block numbers of where the data blocks are located in storage. Block numbers within a set of indirect blocks are compacted into a compacted indirect block comprising a base block number, a count of additional block numbers after the base block number in the compacted indirect block, and a pattern of the block numbers in the compacted indirect block. The compacted indirect block is stored into memory for processing access operations to the object. Storing compacted indirect blocks into memory allows for more block numbers to be stored within memory. In this way, the block numbers are read from memory is faster than loading the block numbers from disk.

Abstract: Techniques are provided for combining data block and checksum block I/O into a single I/O operation. Many storage systems utilize checksums to verify the integrity of data blocks stored within storage devices managed by a storage stack. However, when a storage system reads a data block from a storage device, a corresponding checksum must also be read to verify integrity of the data in the data block. This results in increased latency because two read operations are being processed through the storage stack and are being executed upon the storage device. To reduce this latency and improve I/O operations per second, a single combined I/O operation corresponding to a contiguous range of blocks including the data block and the checksum block is processed through the storage stack instead of two separate I/O operations. Additionally, I/O operation may be combined into a single request that is executed upon the storage device.

Abstract: The disclosed technology enables quicker initialization of a new master node for a cluster when a previous master node fails by tracking node state in the cluster prior to being designated the new master node. In a particular example, a method includes, in a first node, designated as a current master node for the cluster, managing the cluster based on states of the nodes determined by the first node. While the first node is designated the master node, the method includes each of the nodes collecting, and storing locally, the states of the nodes. In response to a failure of the first node, the method includes selecting a second node of the nodes a new master node. Upon being designated the new master node, the method includes the second node managing the cluster of nodes based on the states of the nodes that the second node collected and stored locally.

Abstract: A technique is configured to provide various data protection schemes, such as replication and erasure coding, for data blocks of volumes served by storage nodes of a cluster configured to perform deduplication of the data blocks. Additionally, the technique is configured to ensure that each deduplicated data block complies with data redundancy guarantees of the data protection schemes, while improving storage space of the storage nodes. In order to satisfy the data integrity guarantees while improving available storage space, the storage nodes perform periodic garbage collection for data blocks to optimize storage in accordance with currently applicable data protection schemes.

Abstract: Techniques are provided for persistent memory file system reconciliation. As part of the persistent memory file system reconciliation, high level file system metadata associated with a persistent memory file system of persistent memory is reconciled. Client access to the persistent memory file system is inaccessible until reconciliation of the high level file system metadata has completed. A first scanner is executed to traverse pages of the persistent memory in order to fix local inconsistencies associated with the pages. A local inconsistency of a first set of metadata or data of a page is fixed using a second set of metadata or data of the page. The first scanner is executed asynchronously in parallel with processing client I/O directed to the persistent memory file system.

Abstract: Systems and methods are described for a streamlined garbage collection process during which data integrity checking is also performed for a distributed key-value (KV) store utilized by a distributed storage management system. According to one embodiment, by making use of full or truncated block IDs (rather than an intermediate probabilistic data structure, such as a Bloom filter) for garbage collection, data integrity checking can be performed concurrently almost for free. During garbage collection, a block ID compare list is compared to block IDs within the distributed KV store. If a particular block ID is present in the distributed KV store but is missing from the block ID compare list, the corresponding data block represents garbage to be collected. If the particular block ID is present in the block ID compare list but missing from the distributed KV store, a data integrity error has been identified.

Abstract: Systems and methods for creation and retention of immutable snapshots to facilitate ransomware protection are provided. According to one embodiment, multiple use cases for retention of snapshots are supported, including (i) maintaining a locked snapshot on a source volume of a first storage system on which it was originally created for at least an associated immutable retention time; (ii) replicating the locked snapshot to a destination volume of a second storage system and also maintaining the replica of the locked snapshot on the destination volume for at least the associated immutable retention time; and (iii) maintaining an unlocked snapshot on the source volume, replicating the unlocked snapshot to the destination volume, locking the replicated snapshot on the destination volume when it has an associated non-zero immutable retention time, and thereafter maintaining the replica on the destination volume in accordance with the immutable retention time.

Abstract: Systems and methods for creation and retention of immutable snapshots to facilitate ransomware protection are provided. According to one embodiment, multiple use cases for retention of snapshots are supported, including (i) maintaining a locked snapshot on a source volume of a first storage system on which it was originally created for at least an associated immutable retention time; (ii) replicating the locked snapshot to a destination volume of a second storage system and also maintaining the replica of the locked snapshot on the destination volume for at least the associated immutable retention time; and (iii) maintaining an unlocked snapshot on the source volume, replicating the unlocked snapshot to the destination volume, locking the replicated snapshot on the destination volume when it has an associated non-zero immutable retention time, and thereafter maintaining the replica on the destination volume in accordance with the immutable retention time.

Abstract: Techniques are provided for a snapshot difference interface integrated into an object store data management container. The snapshot difference interface is capable of interpreting an object format and snapshot file system format of snapshots backed up to an object store within objects formatted according to the object format. The snapshot difference interface can identify differences between snapshots, such as files that changed between the snapshots, while the snapshots are still resident within the object store. Because the snapshot difference interface does not retrieve the snapshots from the object store, security is improved, resource and network consumption is reduced, there is less of an impact upon client I/O processing, and a catalog of the snapshots can be more efficiently built and recovered in the event of corruption.

Abstract: One or more systems, devices, computer program products, and/or computer-implemented methods provided herein to use a redundant array of disks. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise a control component that directs, for n physical drives of a redundant array of disks (RAID) storing data for at least n logical volumes, log-structured writing of data of each logical volume of the at least n logical volumes vertically across chunks of only a single physical drive of the n physical drives, wherein the control component further directs writing of parity data at each of the physical drives, which parity data at each physical drive of the subset respectively corresponds to other ones of the physical drives of the n physical drives.

Abstract: Methods and systems for a networked storage system are provided. One method includes: generating, by a first node, a dummy entry in a storage location cache of the first node, the dummy entry associated with a read request received by the first node for data stored using a logical object owned by a second node; receiving, by the first node, an invalidation request to invalidate any storage location entry associated with the data, the invalidation request sent in response to the second node receiving a write request to modify the data; invalidating, by the first node, the dummy entry; receiving, by the first node, a response to the read request from the second node with the requested data; and replacing, by the first node, the dummy entry with a storage location entry and invalidating the storage location entry based on the invalidated dummy entry.

Abstract: A system is described. The system includes a processing resource and a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the processing resource to detect an unrecognized Internet Protocol Security (IPsec) packet associated with an IP address at a first node within a cluster, retrieve one or more selector fields from the IPsec packet, query of a security policy database to determine whether a destination IP address included in the one or more retrieved selector fields matches one or more matching outbound IPsec policies associated with a destination IP address, determine whether a matching outbound IPsec policy includes an IPsec policy associated with the destination address entry and establish the first IPsec SA communication session between the first node and the client based on the outbound IPsec policy.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for providing common snapshot retention and automatic fanout reconfiguration for an asynchronous leg after a failure event that causes a failover from a primary storage site to a secondary storage site. A computer-implemented method comprises providing an asynchronous replication relationship with an asynchronous update schedule from one or more storage objects of the first storage node to one or more replicated storage objects of the third storage node, creating a snapshot copy of the one or more storage objects of the first storage node, transferring the snapshot copy to the third storage node based on an asynchronous mirror policy, and intercepting the snapshot create operation on the primary storage site and transferring the snapshot copy to the second storage node to provide a common snapshot between the second storage node and the third storage node.

Abstract: A computer-implemented method comprises providing a synchronous replication relationship from one or more storage objects of a first storage node to one or more replicated storage objects of a second storage node, providing an asynchronous replication relationship with an asynchronous update schedule from the one or more storage objects of the first storage node to one or more replicated storage objects of the third storage node to provide a protection configuration, tracking, with the third storage node of the tertiary site, a state of the secondary storage site, automatically performing a failover from the primary storage site to the secondary storage site and activating a synchronous mirror copy for the one or more replicated storage objects of the second storage node, and automatically initiating realignment and reconfiguration of the protection configuration to the tertiary storage site based upon the state of the secondary storage site.

Abstract: Methods, non-transitory machine-readable media, and computing devices that manage resources between multiple hosts coupled to dual-port solid-state disks (SSDs) are disclosed. With this technology, in-core conventional namespace (CNS) and zoned namespace (ZNS) mapping tables are synchronized by a host flash translation layer with on-disk CNS and ZNS mapping tables, respectively. An entry in one of the in-core CNS or ZNS mapping tables is identified based on whether a received storage operation is directed to a CNS or a ZNS of the dual-port SSD. The entry is further identified based on a logical address extracted from the storage operation. The storage operation is serviced using a translation in the identified entry for the logical address, when the storage operation is directed to the CNS, or a zone identifier in the identified entry for a zone of the ZNS, when the storage operation is directed to the ZNS.

Abstract: Synchronization metadata is read from non-volatile storage. The synchronization metadata comprises indications of one or more synchronization targets. A synchronization target is a node of a clustered storage system. A synchronization cache is populated with the synchronization metadata. After populating the synchronization cache with at least a portion of the synchronization metadata, a connection to a network is established. After the connection to the network is established, a connection to a first of the one or more synchronization targets is established.

Abstract: Systems and methods for reducing delays between the time at which a need for a resynchronization of data replication between a volume of a local CG and its peer volume of a remote CG is detected and the time at which the resynchronization is triggered (Reseed Time Period) are provided. According to an example, information indicative of the direction of data replication between the volume and the peer volume is maintained within a cache of a node. Responsive to a disruptive operation (e.g., relocation of the volume from a first node to a second node), the Reseed Time Period is lessened by proactively adding a passive cache entry to a cache within the second node at the time the CG relationship is created when the second node represents an HA partner of the first node and prior to the volume coming online when the second node represents a non-HA partner.

Abstract: Systems, methods, and machine-readable media for monitoring a storage system and correcting demand imbalances among nodes in a cluster are disclosed. A performance manager for the storage system may detect performance imbalances that occur over a period of time. When operating below an optimal performance capacity, the manager may cause a volume to be moved from a node with a high load to a node with a lower load to achieve a preventive result. When operating at or near optimal performance capacity, the manager may cause a QOS limit to be imposed to prevent the workload from exceeding the performance capacity, to achieve a proactive result. When operating abnormally, the manager may cause a QOS limit to be imposed to throttle the workload to bring the node back within the optimal performance capacity of the node, to achieve a reactive result. These actions may be performed independently, or in cooperation.

Abstract: Systems, methods, and machine-readable media for organizing resources within a context, controlling access to resources, providing auditing capabilities based on the context, and granting permission to perform work using a single record object. A resource provider service may receive a request from a client. A record may be created in response to receiving the client request. The record may be associated with the requested work. The record may provide context for and permission to perform the requested work. The resource provider service may respond to the client acknowledging receiving the request. The resource provider service may record work to be done in response to receiving the request. The resource provider service may perform the work. The resource provider service may update the record indicating that work is being performed. The resource provider service may archive and/or remove the record when it has expired.

Abstract: Techniques are provided for supporting a lookup structure for a file system implementing hierarchical reference counting. A write operation to write data to a page maintained by the file system is received. A lookup within a lookup structure is performed using information related to the page in order to identify a lookup entry within the lookup structure. A hash generation count within the lookup entry is compared to a file system info generation count within a file system info object for a volume associated with the page. In response to the lookup entry generation count not matching the file system info generation count, a file system tree of the file system is traversed to determine a reference count for the page, and the write operation is implemented based upon the reference count. Otherwise, the lookup entry is utilized to access the page for processing the write operation.