Abstract: Systems and methods for making use of non-persistent storage as the journaling storage media for a virtual storage system are provided. According to one embodiment, in order to meet the needs of Extreme Low Latency Workloads while also seeking to provide predictable performance and the lowest possible latency, ephemeral storage of the virtual storage system is used to preserve state information (e.g., in the form of boot arguments and an operation log journal) across a host failure recovery scenario in which the virtual storage system is expected to be redeployed within a compute instance brought up by a cloud environment of a hyperscaler on the same host, thereby providing improved data durability (fewer host failure scenarios that result in lost data) as compared to the use of ephemeral memory of the compute instance and lower write latency than the use of persistent storage provided by the cloud environment.

Abstract: Systems and methods for managing data storage using a distributed file system are provided. In one example, a relocation event is detected that indicates a relocation is to be initialized. The relocation is initialized by identifying a destination node of a distributed storage system for the relocation of a set of objects in a cluster database, including a logical block device, a corresponding logical aggregate, and a corresponding file system volume. A state of each of the set of objects is changed to offline. The set of objects are then relocated from an originating node of the distributed storage system to the destination node in which the corresponding logical aggregate is relocated after the logical block device and the corresponding file system volume is relocated after the logical aggregate. Finally, the state of each of the set of objects is changed to online.

Abstract: Techniques are provided for persistent hole reservation. For example, hole reservation flags of operations targeting a first storage object of a first node are replicated into replication operations targeting a second storage object of a second node during a transition operation to transition the first storage object and the second storage object from an asynchronous replication state to a synchronous replication state. In another example, the second storage object is grown to a size of a replication punch hole operation that failed due to targeting a file block number greater than an end of size of the second storage object.

Abstract: Techniques are provided for on-demand serverless disaster recovery. A primary node may host a primary volume. Snapshots of the primary volume may be backed up to an object store. In response to failure, a secondary node and/or an on-demand volume may be created on-demand. The secondary node may provide clients with failover access to the on-demand volume while a restore process restores a snapshot of the primary volume to the on-demand volume. In some embodiments, there was no secondary node and/or on-demand volume while the primary node was operational. This conserves computing resources that would be wasted by otherwise hosting the secondary node and/or on-demand volume while clients were able to access the primary volume through the primary node. Modifications directed to the on-demand volume are incrementally backed up to the object store for subsequently restoring the primary volume after recovery.

Abstract: A storage area network (SAN)-attached storage system architecture is disclosed. The storage system provides strongly consistent distributed storage communication protocol semantics, such as SCSI target semantics. The system includes a mechanism for presenting a single distributed logical unit, comprising one or more logical sub-units, as a single logical unit of storage to a host system by associating each of the logical sub-units that make up the single distributed logical unit with a single host visible identifier that corresponds to the single distributed logical unit. The system further includes mechanisms to maintain consistent context information for each of the logical sub-units such that the logical sub-units are not visible to a host system as separate entities from the single distributed logical unit.

Abstract: Methods and systems for solid state drives are provided, including assigning a first namespace to a first instance of a storage operating system and a second instance of the storage operating system for enabling read access to a first portion of a flash storage system by the first instance, and read and write access to the second instance; allocating a second namespace to the first instance for exclusive read and write access within a second portion of the flash storage system; generating, by the first instance, a request for the second instance to transfer a data object from the second portion owned by the first instance to the first portion; storing, by the second instance, the data object at the first portion; and updating metadata of the data object at the second portion, the metadata indicating a storage location at the second portion where the data object is stored.

Abstract: Methods, non-transitory machine readable media, and computing devices that more securely facilitate data protection workflows are disclosed. With this technology, identification information for primary inbound and outbound queues is extracted from a registration token received from an administrator device. A registration request is inserted into the primary outbound queue using the identification information and one or more communication networks that are external to a data center. The primary inbound queue is polled using the identification information and the communication networks to retrieve messages from a backup service computing device that instantiated the queues and generated the registration token. Accordingly, nodes in a data center can communicate more securely with a cloud backup service via queues and without exposing any HTTP ports to the backup service.

Abstract: Techniques are provided for storage tier verification checks. A determination is made that a mount operation of an aggregate of a set of volumes stored within a multi-tier storage environment has completed. A first metafile and a second metafile are maintained to track information related to the storage of objects of a volume of the aggregate within a remote object store that is a tier of the multi-tier storage environment. A distributed verification is performed between the first metafile and the second metafile to identify an inconsistency. Accordingly, the first metafile and the second metafile are reconciled to address the inconsistency so that storage information within the first metafile and the second metafile are consistent.

Abstract: A system, method, and machine-readable storage medium for performing garbage collection in a distributed storage system are provided. In some embodiments, an efficiency level of a garbage collection process is monitored. The garbage collection process may include removal of one or more data blocks of a set of data blocks that is referenced by a set of content identifiers. The set of slice services and the set of data blocks may reside in a cluster, and a set of probabilistic filters (e.g., Bloom filters) may indicate whether the set of data blocks is in-use. At least one parameter of a probabilistic filter of the set of probabilistic filters may be adjusted (e.g., increased or reduced) if the efficiency level is below the efficiency threshold. Garbage collection may be performed on the set of data blocks in accordance with the set of probabilistic filters.

Abstract: Techniques are provided for lock reservations for shared storage. A reserve command to reserve a storage structure is received by a driver from a node. The reserve command is formatted according to a storage protocol. The driver translates the reserve command into a lease acquire command formatted according to an object store protocol and targeting an object stored within an object store and corresponding to the storage structure. A lease identifier derived from a node identifier of the node is inserted into the lease acquire command. The lease acquire command is routed to the object store for obtaining a lease on the object for granting the node exclusive write access to the object.

Abstract: In addition to caching I/O operations at a host, at least some data management can migrate to the host. With host side caching, data sharing or deduplication can be implemented with the cached writes before those writes are supplied to front end storage elements. When a host cache flush to distributed storage trigger is detected, the host deduplicates the cached writes. The host aggregates data based on the deduplication into a “change set file” (i.e., a file that includes the aggregation of unique data from the cached writes). The host supplies the change set file to the distributed storage system. The host then sends commands to the distributed storage system. Each of the commands identifies a part of the change set file to be used for a target of the cached writes.

Abstract: Data is replicated on a backup node, where the granularity of the replication can be less than a full volume. A data consistency group comprising a subset of data for a volume is defined for a primary node. A set of differences for the data consistency group is sent to a backup node. The backup node creates change logs in response to receiving the set of differences. In response to receiving a request to access a file having data in the data consistency group, the backup node creates a clone of the file. The backup node determines whether an update to a data block of the file exists in the change logs. In response to determining that the update to the data block exists in the change logs, the backup node updates a copy of the data block for the cloned file with data in the change logs.

Abstract: A method and system for co-locating journaling and data storage based on write requests. A write request that includes metadata and data is received from a client. A logical storage unit for storing the metadata and the data is identified. The logical storage unit is divided into a journal partition and a volume partition. The journal partition includes a first log and a second log. Which of the first log and the second log is an active log and which of the first log and the second log is an inactive log are identified. The metadata is recorded in a first location in the active log and the data is recorded in a second location in the active log during a single I/O operation. A reply is sent to the client after the metadata and the data are recorded in the journal partition.

Abstract: A method and system for co-locating journaling and data storage based on write requests. A write request that includes metadata and data is received from a client. A logical storage unit for storing the metadata and the data is identified. The logical storage unit is divided into a journal partition and a volume partition. The journal partition includes a first log and a second log. Which of the first log and the second log is an active log and which of the first log and the second log is an inactive log are identified. The metadata is recorded in a first location in the active log and the data is recorded in a second location in the active log during a single I/O operation. A reply is sent to the client after the metadata and the data are recorded in the journal partition.

Abstract: Methods and systems for a storage environment are provided, including generating a plurality of child (or tetris) write requests to write data for a write request using a plurality of subdivisions of a plurality of logical zones defined for a plurality of zoned solid state drives (ZNS SSDs) of a RAID array, each LZone mapped to one or more logical RAID zone (RZone) of the ZNS SSDs having a plurality of physical zones across a plurality of independent media units of each ZNS SSD; assigning a sequence number to each child (or tetris) write request corresponding to each subdivision, the sequence number indicating an order in which the child (or tetris) write requests are to be processed; and selecting, based on the assigned sequence number, one or more subdivisions for sequentially writing data to one or more RZones of the plurality of ZNS SSDs.

Abstract: A method, non-transitory computer readable medium, and device that assists with managing storage in a distributed deduplication system includes receiving an object to be stored from a client computing device. The received object is divided into a plurality of fragments. A plaintext hash value and a ciphertext hash value is determined for each of the plurality of fragments, wherein each of the plurality of fragments is renamed with the corresponding determined ciphertext hash value. Each of the renamed plurality of fragments are stored in a plurality of storage repositories.

Abstract: One or more techniques and/or computing devices are provided for moving a consistency group having a replication relation. For example, a first consistency group of storage objects (e.g., files, logical unit numbers (LUNs), etc.) within first storage may have a replication relationship with a second consistency group within second storage (e.g., the second consistency group is maintained as a synchronously replicated copy of the first consistency group). A volume copy operation, a single file move on demand command, a single file restore command, or other functionality is used to move the first consistency group from the first storage to third storage, such as for load balancing, to create a moved first consistency group within the third storage. A new replication relationship is established between the moved first consistency group and the second consistency group, and the moved first consistency group and the second consistency group are resynchronized.

Abstract: Techniques are provided for inofile management and access control list file handle parity. For example, operations targeting a first storage object of a first node are replicated to a second storage object of a second node. A size of an inofile maintained by the second node is increased if an inode number to be allocated by the replication operation is greater than a current size of the inofile. Access control list file handle parity is achieved by maintaining parity between inode number and generation number pairings of the first node and the second node.

Abstract: Methods and systems for a storage environment are provided. One method includes copying a data unit from a first temporary storage location corresponding to each zoned solid-state drive (ZNS SSD) of a first ZNS SSD set of a storage system to a first XOR module, while determining a first partial horizontal parity using the data unit stored in the first temporary storage location; and determining a vertical parity for each ZNS SSD of the first ZNS SSD set using the data unit provided to the first XOR module in a current cycle and vertical parity determined from a previous cycle.

Abstract: Techniques are provided for freeing and utilizing unused inodes. For example, an operation, targeting a first storage object of a first node having a replication relationship with a second storage object of a second node, is intercepted. A replication operation is created as a replication of the operation. The operation is implemented upon the first storage object and the replication operation is implemented upon the second storage object. A determination is made that the replication operation uses an inode no longer used by storage objects of the second node. The inode targeted by the replication operation is freed and utilized based upon the inode being a leaf inode. If the inode is a stream directory inode, then data streams of the stream directory inode are moved under a new private inode and the stream directory inode is released.