Abstract: Network address migration using a destination compute instance to update network configuration information in a cloud environment is disclosed. A network interface either using a private address within a subnet corresponding to the created network interface or using a floating address outside of the subnet corresponding to the created network interface is created. A first node of the HA pair with a service provider interface. The first node is an active data server of the HA pair, and the second node is a backup node of the HA pair. Requests are serviced the first node using the created network interface. Upon failure of the first node, the second node performs a failover, wherein if the first node was utilizing a floating address, the second node registers the second node with the service provider interface by adding an address of the second node to the route table.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for providing an automatic unplanned failover (AUFO) feature to guarantee non-disruptive operations (e.g., operations of business enterprise applications, operations of software application) even in the presence of failures including, but not limited to, network disconnection between multiple data centers and failures of a data center or cluster.

Abstract: One or more techniques and/or computing devices are provided for managing an arbitrary set of storage items using a granset. For example, a storage controller may host a plurality of storage items and/or logical unit numbers (LUNs). A subset of the storage items are grouped into a consistency group. A granset is created for tracking, managing, and/or providing access to the storage items within the consistency group. For example, the granset comprises application programming interfaces (APIs) and/or properties used to provide certain levels of access to the storage items (e.g., read access, write access, no access), redirect operations to access either data of an active file system or to a snapshot, fence certain operations (e.g., rename and delete operations), and/or other properties that apply to each storage item within the consistency group. Thus, the granset provides a persistent on-disk layout used to manage an arbitrary set of storage items.

Abstract: The disclosure describes a system for developing a forensic projection for data lost in a cyberattack. After identifying a cyberattack causing a loss of data in the data volume, the system identifies a snapshot of the portion of the data volume affected by the cyberattack. The system estimates, based on the snapshot, an amount of lost data caused by the cyberattack. The system then determines based at least on the amount of lost data, a data loss metric.

Abstract: Techniques are provided for caching data during an on-demand restore using a cloud block map. A client may be provided with access to an on-demand volume during a restore process that copies backup data from a snapshot within a remote object store to the on-demand volume stored within local storage. In response to receiving a request from the client for a block of the backup data not yet restored from the snapshot to the on-demand volume, the block may be retrieved from the snapshot in the remote object store. The block may be cached within a cloud block map stored within the local storage as a cached block. The client may be provided with access to the cached block.

Abstract: Techniques are provided for implementing garbage collection and bin synchronization for a distributed storage architecture of worker nodes managing distributed storage composed of bins of blocks. As the distributed storage architecture scales out to accommodate more storage and worker nodes, garbage collection used to free unused blocks becomes unmanageable and slow. Accordingly garbage collection is improved by utilizing heuristics to dynamically speed up or down garbage collection and set sizes for subsets of a bin to process instead of the entire bin. This ensures that garbage collection does not use stale information about what blocks are in-use, and ensures garbage collection does not unduly impact client I/O processing or conversely falls behind on garbage collection. Garbage collection can be incorporated into a bin sync process to improve the efficiency of the bin sync process so that unused blocks are not needlessly copied by the bin sync process.

Abstract: Techniques are provided for metadata management for enabling automated switchover in accordance with a configuration of storage solution that expresses a preference for either maintaining availability (e.g., a non-zero RPO mode) of the storage solution or avoiding data loss (e.g., a zero RPO mode). In one example, responsive to detecting a switchover trigger event, a node of a local cluster of a cross-site storage solution determines whether performance of an automated switchover from a failed cluster to a surviving cluster of the cross-site storage solution is enabled. Responsive to an affirmative determination, the node selectively proceeds with the automated switchover based on the configuration.

Abstract: Various embodiments of the present disclosure relate to compression techniques that can be used for data storage, indexing, and retrieval. In an example embodiment, a controller of or in communication with a data storage system can obtain a request to store a file in the data storage system. The controller identifies time-series data associated with input/output of the file and metrics of the file, generates a feature vector for the file, and compresses the feature vector using a compression technique, resulting in a compressed feature vector having a compression ratio. The controller performs a hash algorithm using the compression ratio to determine a storage device at which to store the compressed feature vector, the compression ratio associated with the file, and an indication of the compression technique. Upon receiving a query for a file, the controller compares compression ratios to identify a storage device storing the requested file.

Abstract: Techniques are provided for a snapshot difference interface integrated into an object store data management container. The snapshot difference interface is capable of interpreting an object format and snapshot file system format of snapshots backed up to an object store within objects formatted according to the object format. The snapshot difference interface can identify differences between snapshots, such as files that changed between the snapshots, while the snapshots are still resident within the object store. Because the snapshot difference interface does not retrieve the snapshots from the object store, security is improved, resource and network consumption is reduced, there is less of an impact upon client I/O processing, and a catalog of the snapshots can be more efficiently built and recovered in the event of corruption.

Abstract: Techniques are provided for performing a resync transfer to recover from a storage site failure. During normal operation of a first site hosting a first volume, data is replicated to a second volume hosted by a second site. If the first site fails, when clients are redirected to the second volume at the second site. When the first site recovers, data modifications made to the second volume are resynced back to the first volume. As part of synchronizing the first volume, a data warehouse is rebuilt at the first site in order to track the location of blocks present on the replication destination. Typically, the data modifications are transferred after the data warehouse is rebuilt, which results in significantly long resync times. The techniques provided herein decrease the resync time by either rebuilding the data warehouse in parallel with resyncing the data modifications or circumvent the need for rebuild.

Abstract: Techniques are provided for a data format for efficient management of checkpoint support. The data format corresponds to a base metafile and a set of instance metafiles used to track storage operations such as a directory restore operation. The base metafile and the set of instance metafiles can be used to resume the storage operation from where the storage operation left off in the event of a failure. The base metafile and the set of instance metafiles can be used to track progress of the storage operation processing objects stored within an object store of a cloud storage environment.

Abstract: Systems and methods are provided for data migration including cloning a multiple-logical unit number (LUN) volume into a plurality of cloned single-LUN volumes on a first storage node of a computing system; creating a plurality of new volumes on a second storage node of the computing system based at least in part on the plurality of cloned single-LUN volumes; selectively copying snapshot data from the plurality of cloned single-LUN volumes to the plurality of new volumes; establishing snapshot mirror relationships between the plurality of cloned single-LUN volumes and the plurality of new volumes; synchronizing the plurality of cloned single-LUN volumes and the plurality of new volumes; and performing a migration of data logical interface failovers (LIFs) from the plurality of cloned single-LUN volumes to the plurality of new volumes.

Abstract: Systems and methods are provided for data migration including cloning a multiple-logical unit number (LUN) volume into a plurality of cloned single-LUN volumes on a first storage node of a computing system; creating a plurality of new volumes on a second storage node of the computing system based at least in part on the plurality of cloned single-LUN volumes; selectively copying snapshot data from the plurality of cloned single-LUN volumes to the plurality of new volumes; establishing snapshot mirror relationships between the plurality of cloned single-LUN volumes and the plurality of new volumes; synchronizing the plurality of cloned single-LUN volumes and the plurality of new volumes; and performing a migration of data logical interface failovers (LIFs) from the plurality of cloned single-LUN volumes to the plurality of new volumes.

Abstract: Techniques are provided for compacting indirect blocks. For example, an object is represented as a structure comprising data blocks within which data of the object is stored and indirect blocks comprising block numbers of where the data blocks are located in storage. Block numbers within a set of indirect blocks are compacted into a compacted indirect block comprising a base block number, a count of additional block numbers after the base block number in the compacted indirect block, and a pattern of the block numbers in the compacted indirect block. The compacted indirect block is stored into memory for processing access operations to the object. Storing compacted indirect blocks into memory allows for more block numbers to be stored within memory. In this way, the block numbers are read from memory faster than loading the block numbers from disk.

Abstract: Various embodiments are directed to techniques for coordinating at least partially parallel performance and cancellation of data access commands between nodes of a storage cluster system. An apparatus may include a processor component of a first node coupled to a first storage device storing client device data; an access component to perform replica data access commands of replica command sets on the client device data, each replica command set assigned a set ID; a communications component to analyze a set ID included in a network packet to determine whether a portion of a replica command set in the network packet is redundant, and to reassemble the replica command set from the portion based if the portion is not redundant; and an ordering component to provide the communications component with set IDs of replica command sets of which the access component has fully performed the set of replica data access commands.

Abstract: Dynamic distribution of compression and decompression job requests to hardware accelerators is disclosed. A set of requests is evaluated to determine a number of compression jobs and a number of decompression jobs in the set of requests. A first set of hardware accelerator engines is allocated to perform compression jobs, and a second set of hardware accelerator engines is allocated to perform decompression jobs. Compression jobs are assigned to the first set of hardware accelerator engines based, at least in part, on a compressibility score of the corresponding job and a workload of the selected hardware accelerator engine. Decompression jobs are assigned to the second set of hardware accelerator engines based, at least in part, on a decompression weight of the corresponding job and a workload of the selected hardware accelerator engine.

Abstract: A method, computing device, and non-transitory machine-readable medium for managing incoming/outgoing flow through a staging area. A request to write data to an object based storage is received. A determination is made that the data is to be first staged within a transfer data structure. The transfer data structure is in a first storage tier and the object based storage is in a second storage tier. A determination is made that an amount of storage space used in the transfer data structure exceeds a start throttle threshold. In response to the determination that the amount of storage space exceeds the start throttle threshold, the request is sent into a queue selected from a set of throttling queues, wherein requests in the queue are handled in a first in, first out (FIFO) order. The request in the queue is processed based on the FIFO order and a dynamic throttle rate.

Abstract: Managing shutdown and reset of a network interface card (NIC) in response to an error condition is disclosed. An indication to initiate a network interface card (NIC) reset and reconnection sequence is received. A notification of a link down condition is transmitted. Pending connections are disconnected. Queue pairs corresponding to the interconnect channels are destroyed. Links corresponding to the NIC are disconnected. Packets are cleared from queues corresponding to the NIC. Send and receive queues are reset. Queue pairs corresponding to the NIC are recreated. Queue pairs are connected to corresponding links. Data transfer resumes over the links.

Abstract: The disclosure describes a system for managing a namespace move between nodes of a data storage environment. During a namespace move associated with a data storage system, the system receives requests at a first node (e.g., a source node) in the data storage system to perform input/output (I/O) operations associated with a namespace subject to the namespace move and stores the requests in a queue at the first node until completion of the namespace move. Upon completion of the namespace move, the system forwards the requests from the first node to a second node (e.g., a destination node) in the data storage system and performs the I/O operations at the second node.

Abstract: The disclosure describes a node management service that groups pods based on an impact of the available instance pool. The node management service identifies a request group associated with a scale-up request to scale up a cluster of compute nodes to host pods in the request group. The node management service iteratively determines to add pods to the request group until an impact of a next pod on a pool of available nodes exceeds a threshold. The node management service sends a request to a distributor to distribute the pods in the request group to one or more nodes obtained from the pool of available nodes.