Abstract: Systems and methods for quality of service management are provided. According to one embodiment, a non-transitory computer-readable medium comprises instructions that when executed by the processing resource cause the processing resource to generate an inertial parameter that represents a resistance to moving a first volume from a first protection domain in a distributed storage system and rebalance, when the inertial parameter indicates the first volume should not be removed from the protection domain, a compute load between one or more available protection domains having capacity for additional volumes. The compute load includes one of a plurality of volumes other than the volume.

Abstract: Techniques are provided for creating file clones of multipart files. Creating clones of files is an integral part of providing backup, restore, and other storage services. However, conventional file cloning techniques are unable to create clones of multipart files that are composed of multiple parts stored across different volumes and/or nodes in a constant time. The disclosed techniques are capable of cloning multipart files by creating a clone parent file into which catalog entries from a source multiple file are moved. A destination multipart file is initially created as an empty clone of the source multipart file. Block sharing of the catalog entries from the clone parent file to the source and destination multipart files is performed, and cloning of the source multipart file is declared complete in a constant time such as within a few seconds or less.

Abstract: A method for reducing write latency in a distributed file system. A write request that includes a volume identifier is received at a data management subsystem deployed on a node within a distributed storage system. The data management subsystem maps the volume identifier to a file system volume and maps the file system volume to a set of logical block addresses in a logical block device in a storage management subsystem deployed on the node. The storage management subsystem maps the logical block device to a metadata object for the logical block device on the node that is used to process the write request. The mapping of the file system volume to the set of logical block addresses in the logical block device enables co-locating the metadata object with the file system volume on the node, which reduces the write latency associated with processing the write request.

Abstract: Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.

Abstract: Techniques, equipment, and systems for enhanced storage systems and storage drive interfacing are presented herein. In one example, a storage interposer includes a storage device connector configured to couple a dual port interface selected among a first interface protocol and a second interface protocol, and a protocol unit configured to transfer storage transactions received over the dual port interface in a storage format. The storage interposer also includes a transaction unit configured to obtain the storage transactions in the storage format and process indications of which port among the dual port interface supplied each of the storage transactions against one or more criteria to order the storage transactions into a queue shared among the ports. A single port storage drive coupled to the storage interposer can be issued the storage transactions from the queue according to the order.

Abstract: OP logs are utilized for fast recovery of synchronous replication in a storage system. According to one embodiment, a computer-implemented method comprises establishing synchronous replication between one or more storage objects of a first consistency group (CG1) of a primary storage site and one or more storage objects of a second consistency group (CG2) of a secondary storage site, initiating a peer health check to determine a health condition of the primary storage site and concurrently initiating a create snapshot process to create a snapshot of one or more storage objects of the CG2 of the secondary storage site in response to detection of missing heartbeat communications from the primary storage site, and performing inflight tracking replay and reconciliation between a first Op log of the primary storage site and a second Op log of the secondary storage site when an out of sync state or failover occurs.

Abstract: A global policy-driven framework for managing, revising, and implementing data coherency policies in a distributed data storage system. An edge node of the distributed data storage system receives a request from an application to store a data object in the system. The edge node, in response to receiving the request to store the data object, generates a coherency policy request, which is then submitted to the primary node of the system. The primary node of the system generates a coherency policy response, which is returned to the edge node. The edge node then stores the data object in the system in accordance with the coherency policy response.

Abstract: In one embodiment, a method comprises providing a first dynamically extensible file system (DEFS) of a first node and a second DEFS of a second node of a storage cluster having a disaggregated storage space within a storage pod, detecting, with the first DEFS of the first node, blocks and associated physical volume block numbers (PVBNs) to be freed, determining whether the first DEFS owns or does not own an allocation area (AA) having the PVBNs to be freed, and transferring non-owned PVBNs to be freed to a remote free log of the first DEFS when the AA having the PVBNs to be freed is not owned or assigned to the first DEFS.

Abstract: Systems, methods, and software are disclosed herein relating to processing telemetry data of a storage environment for anomaly detection in various implementations. In an implementation, a computing apparatus detects a sequence of missing data in telemetry data associated with a user of a data storage environment and verifies the user was active during a time period of the sequence of missing data. Upon verification, the computing apparatus generates synthetic data to replace the sequence of missing data; the synthetic data is based on activity data of users similar to the given user. The computing apparatus generates augmented data with the telemetry data and the synthetic data and incorporates the augmented data into a historical user activity profile. The computing apparatus detects anomalous behavior in the storage environment based on a comparison of new telemetry data with the historical user activity profile and takes corrective action associated with the anomalous behavior.

Abstract: The disclosure describes systems, devices, and methods for tracking operations of controllers in a data storage environment on a per-controller basis. In an implementation, a method for re-assigning responsibility of individual controller-owned parity bitmap sections and re-performing an incomplete operation is provided. In the method, a controller detects a failure of a first controller and responsively assign responsibility for a second of a parity bitmap previously assigned to the first controller to the second controller. The controller directs the second controller to read the assigned section of the parity bitmap. The controller further directs the second controller to, for a status indicator in the section indicating incomplete, compute new parity data based on source data associated with the status indicator, store the new parity data in the section, and update the status indicator from incomplete to complete.

Abstract: Methods and systems involve sequentially storing compressed data segments received at a storage system in disk blocks such that at least one of the disk blocks includes compressed data from at least two of the compressed data segments.

Abstract: Techniques are provided for atomic writes for persistent memory. In response to receiving a write operation, a new per-page structure with a new page block number is allocated. New data of the write operation is persisted to a new page of the persistent memory having the new page block number, and the new per-page structure is persisted to the persistent memory. If the write operation targets a hole after the new data and the new per-page structure have been persisted, then a new per-page structure identifier of the new per-page structure is inserted into a parent indirect page of a page comprising the new data. If the write operation targets old data after the new data and the new per-page structure have been persisted, then an old per-page structure of the old data is updated with the new page block number.

Abstract: Disclosed herein are methods and systems for the operation of a resource management service. The resource management service deploys reclaimable compute instances from a resource pool and continuously generates predicted remaining lifespans for the deployed reclaimable compute instances. The predicted remaining lifespan is monitored to determine if the predicted remaining lifespan is below a threshold value. In response to the predicted remaining lifespan for a reclaimable compute instance falling below a threshold value, the resource management service instructs the reclaimable compute instance to create an application state snapshot of an application running thereon. A subsequent compute instance is deployed from the resource pool, on which the application can be restored to a previous state using the application state snapshot.

Abstract: Systems and methods for identifying anomalous activities in a cloud computing environment are provided. According to one embodiment, a customer's infrastructure may be fortified by leveraging deep learning technology (e.g., an encoder-decoder machine-learning (ML) model) to predict events in the cloud environment. During a training phase, the ML model may be trained to make a prediction regarding a next event based on a predetermined or configurable length of a sequence of contextual events. For example, historical events (e.g., cloud application programming interface (API) events logged to a cloud activity trace) observed within the customer's cloud infrastructure over the course of a particular date range may be split into appropriate event/context pairs and fed to the ML model. Subsequently, during a run-time anomaly detection phase, the ML model may be used to predict a next event based on a sequence of immediately preceding events to facilitate identification of anomalous activity.

Abstract: Systems and methods for sharing a namespace of an ephemeral storage device by multiple consumers are provided. In an example, a storage driver (e.g., a non-volatile memory express (NVMe) driver) of a virtual storage system deployed within a compute instance of a cloud environment facilitates sharing of the namespace by exposing an application programming interface (AP)I through which the multiple consumers access an ephemeral storage device associated with the compute instance. During initialization processing performed by each consumer, for example, during boot processing of the virtual storage system, the consumers may share the namespace by reserving for their own use respective partitions within the namespace via the API and thereafter restrict their usage of the namespace to their respective partitions.

Abstract: The technology disclosed herein enhances data protection in a distributed storage system. In a particular example, a method includes determining a drive in a subject node of the distributed storage system has failed while storing first data of a data set distributed across nodes of the distributed storage system by a data protection mechanism. The method further includes broadcasting failure information indicating the data set from the subject node to other nodes of the distributed storage system. At the other nodes, in response to receiving the failure information, the method includes identifying a subset of the other nodes that also store a portion of the data set. In each identified node of the subset, the method includes identifying second data of the data set stored on a local drive and copying the second data to a different local drive to protect the data set from further drive failure.

Abstract: Systems and methods for enhancing API security by identifying anomalous activities in a cloud environment are provided. In one example, the lack of awareness of an external API with respect to how calls to the external API may affect a cluster of a container orchestration platform is addressed. For instance, the views of the external and internal APIs may be combined to achieve better API security by correlating external API calls with undesirable behavior or other anomalies arising in the internal API. Responsive to identifying such undesirable behavior, information (e.g., a host, a source IP, a user, a specific payload) associated with the offending external API call may be added to a network security feature (e.g., a deny list, an IPS, or a WAF) utilized by the external API to facilitate performance of enhanced filtering of subsequent external API calls by the external API on behalf of the internal API.

Abstract: The technology disclosed herein enables enforcement of Information Lifecycle Management (ILM) policies across tenants in an object storage system using tags associated with the ILM policies. In a particular example, a method includes identifying ILM policies for an object storage system having multiple tenants and associating respective tags with the ILM policies. The method further includes enabling a subset of the tags to be assigned to data objects of a tenant of the multiple tenants and enforcing a portion of the ILM policies on the data objects, wherein the portion of the ILM policies corresponds to tags of the subset assigned to the data objects.

Abstract: Systems, methods, and machine-readable media are disclosed for a global scheduler that manages jobs across nodes of one or more clusters. The global scheduler receives a request to replicate a volume from a first storage node to a second storage node, the global scheduler handling job scheduling for one or more storage operating systems across one or more clusters. The global scheduler stores a new job in a job queue based on the request to replicate the volume. The global scheduler determines that the new job has a higher priority than other jobs in the job queue. The global scheduler allocates one or more resources of the first storage node to the new job in response to determining that the new job has a higher priority than other jobs in the job queue. The global scheduler sends an instruction to run the new job.

Abstract: Systems and methods for performing journal swapping are provided. In one example, the backing storage used for virtual non-volatile random access memory (vNVRAM) is dynamically switched based on the high-availability (HA) state of an HA pair of nodes of a virtual storage system. By default, in-memory logging may be used for write journaling when the HA pair is operating in a normal HA state. When the HA pair is in an HA degraded state, the write journaling may be performed to local ephemeral storage. This allows the file system of the virtual storage system to run in a more performant configuration most of the time (e.g., during which HA is enabled and healthy). At the same time, the file system has the ability to swap to a less performant but more resilient configuration during planned events (e.g., scheduled maintenance and throughput scaling) in which HA is in a degraded state.