Abstract: Systems and methods for supporting dynamic disk growth within a storage appliance are provided. According to one embodiment, a portion of a logical size of each of multiple disks (e.g., hyperscale disks or Logical Unit Numbers (LUNs)) are provisioned for use by a storage system as backing for respective file system disks. To accommodate growth, block numbers for the file system disks are pre-allocated within a sparse space of a contiguous sequence of block numbers corresponding to a number of blocks represented by the logical size. Metadata is maintained for the file system disks regarding a range of the pre-allocated block numbers that are available for use. Responsive to a triggering condition, the provisioned portion of a disk is increased and subsequently, responsive to detecting a change in a size of the disk by the storage system, a size of the corresponding file system disk is updated within the metadata.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for improving a resumption time of input/output (I/O) operations during a common snapshot procedure for storage objects. A computer-implemented method includes initiating a snapshot multi create operation to selectively form a batch of first and second synchronous replicated datasets that belong to a first group of storage disks at the primary storage site and corresponding second group of storage disks at the secondary storage site, performing a batch snapshot create operation on the primary storage site by executing snapshots of storage objects on the primary storage site of the batch of first and second synchronous replicated datasets in parallel multiple threads to effectively utilize processing resources on the primary storage site, and initiating an independent workflow and state machine for each storage object of the batch of first and second synchronous replicated datasets.

Abstract: Techniques are provided for replicating clone files from a source file system to a destination file system in a computing system. A file system of a first storage node may be replicated to a second storage node such as for data protection and backup purposes. The file system may include clone files that were created as clones of parent files. The clone files may have been created in a storage efficient manner where a clone file refers to data of a parent file, instead physically storing duplicate instances of the parent data of the parent file. The techniques provided herein are capable of replicating the clone files to the second node while preserve the storage efficiency of the clone files.

Abstract: Managing access to stored objects includes receiving, by a storage OS in a computing system, an access key identifier (ID) and access key associated with a user of an object storage service (OSS); determining, by a directory service, whether the user is a member of a group authorized to access a storage object in at least one of a network accessible storage (NAS) volume and an OSS bucket, based at least in part on the access key ID; in response to the user being a group member, attempting to authenticate the user by the storage OS with the directory service based at least on the access key ID and the access key; and in response to the user being authenticated by the directory service, allowing, by the storage OS, access by the user to the storage object stored in at least one of the NAS volume and the OSS bucket.

Abstract: Methods and systems for applications and/or virtual machines (“VM”) are provided. As an example, one method includes registering a storage system configured to store data for a VM; creating a policy for the VM for generating a VM backup by a storage management system interfacing with the storage system and a cloud-based microservice; storing, by the cloud-based micro-service, a copy of the VM backup at a cloud-based storage; receiving, by the cloud-based micro-service, an indication from the storage management system of a successful execution of a pre-restore operation upon unregistering the VM from a VM management system; and copying, by the cloud-based micro-service, VM data from the VM backup stored at the cloud-based storage to the storage system for executing a restore operation to restore the VM backup at the storage system.

Abstract: The technology disclosed herein enables access to a file system by a portable executable program. In a particular example, a method includes, in a host executing the portable executable program, recognizing the portable executable program is executing on the one or more processing systems and determining the portable executable program is configured to access the file system. The method also includes directing the portable executable program to create a module therein for file system access and creating an abstraction layer with which the module exchanges file system commands. In the abstraction layer, the method includes translating the file system commands to translated commands for the file system and exchanging translated commands between the abstraction layer and the file system.

Abstract: Systems and methods are described for performing persistent inflight tracking of operations (Ops) within a cross-site storage solution. According to one embodiment, a method comprises maintaining state information regarding a data synchronous replication status for a first storage object of a primary storage cluster and a second storage object of a secondary storage cluster. The method includes performing persistent inflight tracking of I/O operations with a first Op log of the primary storage cluster and a second Op log of the secondary storage cluster, establishing and comparing Op ranges for the first and second Op logs, and determining a relation between the Op range of the first Op log and the Op range of the second Op log to prevent divergence of Ops in the first and second Op logs and to support parallel split of the Ops.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for providing common snapshot retention and automatic fanout reconfiguration for an asynchronous leg after a failure event that causes a failover from a primary storage site to a secondary storage site. A computer-implemented method comprises providing an asynchronous replication relationship with an asynchronous update schedule from one or more storage objects of the first storage node to one or more replicated storage objects of the third storage node, creating a snapshot copy of the one or more storage objects of the first storage node, transferring the snapshot copy to the third storage node based on an asynchronous mirror policy, and intercepting the snapshot create operation on the primary storage site and transferring the snapshot copy to the second storage node to provide a common snapshot between the second storage node and the third storage node.

Abstract: Systems, methods, and machine-readable media are disclosed for running an executable object on an object storage system. An executable object including executable code is stored on a first storage node of an object storage system. The first storage node receives a request to run the executable object. The first storage node identifies the physical location of one or more data objects of interest to be processed by the executable object. The first storage node runs the executable object to process the identified one or more data objects. The first storage node receives a request from a second storage node of the object storage system for the executable object in response to one or more data objects of interest being located at the second storage node. The first storage node sends a copy of the executable object to the second storage node.

Abstract: Techniques are provided for implementing asynchronous region-aware caching for snapshot prefetching and restore. A first application within a first container of a container orchestration platform may be hosted at a first region of a multi-cloud environment. A second application within a second container of the container orchestration platform may be hosted at a second region of the multi-cloud environment. Data of the first application may be stored within storage located within the first region of the multi-cloud environment. In response to determining that the second application has a threshold probability of accessing the data of the first application (e.g., a snapshot that the second application can restore), the data may be cached as cached data within a network edge of the second region for access by the second application.

Abstract: Systems and methods are described for a cross-site high availability distributed storage system. According to one embodiment, a computer implemented method includes providing a remote direct memory access (RDMA) request for a RDMA stream, and generating, with an interconnect (IC) layer of the first storage node, multiple IC channels and associated IC requests for the RDMA request. The method further includes mapping an IC channel to a group of multiple transport layer sessions to split data traffic of the IC channel into multiple packets for the group of multiple transport layer sessions using an IC transport layer of the first storage node and assigning, with the IC transport layer, a unique transaction identification (ID) to each IC request and assigning a different data offset to each packet of a transport layer session.

Abstract: Systems and methods are provided for implementation and use of disaggregated storage by a distributed storage system. In various examples described herein, the association of blocks of storage space within a storage pod with a given dynamically extensible file system (DEFS) may be in large chunks of one or more gigabytes (GB), which may be referred to herein as “allocation areas” (AAs) that each include multiple RAID stripes. The use of large, multi-GB chunks, as the unit of space allocation/assignment to DEFSs of a distributed storage system facilitates ease of management and independence of write allocation. The use of AAs described herein further allows disk space to be used more fluidly across individual (nodes) storage systems of a distributed storage system (cluster of nodes working together), thereby eliminating silos of storage; and processing resource (e.g., CPU) load may be distributed across the cluster.

Abstract: Techniques are provided for dynamically implementing quality of service policies for a distributed storage system based upon resources saturation. A quality of service policy is defined for throttling I/O operations received by a node of the distributed storage system based upon whether resources of the node have become saturated. The quality of service policy is dynamically implemented based upon ever changing resource utilization and saturation. Dynamically implementing the quality of service policy improves the ability to efficiently utilize resources of the node compared to conventional static polices that cannot adequately react to such changing considerations and resource utilization/saturation. With conventional static policies, an administrator manually defines a minimum amount of guaranteed resources and/or a maximum resource usage cap that could be set to values that result in inefficient operation and resource starvation.

Abstract: A computer implemented method includes maintaining information indicative of whether a data replication relationship between a dataset associated with the local CG (CG1) and a mirror copy of the dataset stored on a remote CG (CG2) of a remote distributed storage system is in an in-synchronization (InSync) state or an out-of-synchronization (OOS) state, determining whether a CG storage expansion process is in progress, and in response to determining the OOS state and whether the primary storage site has a failure, initiating an automatic unplanned failover (AUFO) workflow without manual intervention on the original volumes in the CG1 of a first storage cluster of the primary storage site and the original volumes in the CG2 of a second storage cluster when the CG storage expansion process is in progress with a new source volume to be a member of CG1 and a new destination volume to be a member of CG2.

Abstract: The disclosure describes artificial intelligence (AI) data platform that utilizes snapshots obtained from a storage node to update a vector database. The AI data platform compares snapshots to generate differential snapshots that identify changed data in storage volumes. The AI data platform uses the differential snapshots to update vector embeddings in a vector database for retrieval-augmented generation (RAG) workflows.

Abstract: The disclosure describes a node management service that determines to deploy a primary workload to a new instance in a compute cluster. The node management service projects an allocatable capacity for the new instance based on historical capacity data. The allocatable capacity is the amount of compute resources available for running the primary workload in the new instance after deployment of a supporting workload to the new instance. The node management service associates the primary workload with the new instance upon determining that the projected allocatable capacity is sufficient for running the primary workload.

Abstract: Techniques are provided for implementing a distributed control plane to facilitate communication between a container orchestration platform and a distributed storage architecture. The distributed storage architecture hosts worker nodes that manage distributed storage that can be made accessible to applications within the container orchestration platform through the distributed control plane. However, the worker nodes may support an imperative model of program commands, but the container orchestration platform and applications therein utilize a declarative model of programming commands not supported by the distributed storage architecture. Accordingly, the distributed control plane is configured with control plane controllers that are paired with the worker nodes and are configured to reformat commands between the imperative model and the declarative model.

Abstract: Techniques are provided for combining data block and checksum block I/O into a single I/O operation. Many storage systems utilize checksums to verify the integrity of data blocks stored within storage devices managed by a storage stack. However, when a storage system reads a data block from a storage device, a corresponding checksum must also be read to verify integrity of the data in the data block. This results in increased latency because two read operations are being processed through the storage stack and are being executed upon the storage device. To reduce this latency and improve I/O operations per second, a single combined I/O operation corresponding to a contiguous range of blocks including the data block and the checksum block is processed through the storage stack instead of two separate I/O operations. Additionally, I/O operation may be combined into a single request that is executed upon the storage device.

Abstract: Systems and methods for enhancing container security are provided by reducing the attack surface. In one example, the exposure of containers to potential security vulnerabilities is reduced by identifying dynamically loaded symbols by an application via performance of static symbol analysis by examining a section of an executable to identify dynamically loaded symbols corresponding to functions contained within shared libraries. Based on a given shared library's usage of functions within standard libraries and a known mapping between functions of standard libraries and system calls, those system calls potentially accessed by the application may be identified and a security policy may be generated and configured for enforcement by a kernel security module to limit system call usage accordingly. Thereafter, the security policy enforced by the kernel security module may be refined based on performance of dynamic symbol analysis to identify system calls that are actually called by the application during runtime.

Abstract: Systems and methods are provided for data migration including cloning a multiple-logical unit number (LUN) volume into a plurality of cloned single-LUN volumes on a first storage node of a computing system; creating a plurality of new volumes on a second storage node of the computing system based at least in part on the plurality of cloned single-LUN volumes; selectively copying snapshot data from the plurality of cloned single-LUN volumes to the plurality of new volumes; establishing snapshot mirror relationships between the plurality of cloned single-LUN volumes and the plurality of new volumes; synchronizing the plurality of cloned single-LUN volumes and the plurality of new volumes; and performing a migration of data logical interface failovers (LIFs) from the plurality of cloned single-LUN volumes to the plurality of new volumes.