Abstract: A method for verifying a consistency of snapshot metadata maintained in an ordered data structure for a plurality of snapshots in a snapshot hierarchy is provided. The method includes identifying a first plurality of nodes maintained in a first ordered data structure for a first snapshot that is a child of a second snapshot; for a first node of the first plurality of nodes, verifying the first node by checking for the first node in a second node map maintained in memory for the second snapshot, wherein the second node map includes a plurality of verified nodes in a second ordered data structure; and based on whether the first node is in the second node map: adding the first node to a first node map maintained in memory for the first snapshot, wherein the first node map includes verified nodes of the first plurality of nodes; or triggering an alarm.

Abstract: A method of processing data in a system having a host and a storage node may include performing a shuffle operation on data stored at the storage node, wherein the shuffle operation may include performing a shuffle write operation, and performing a shuffle read operation, wherein at least a portion of the shuffle operation is performed by an accelerator at the storage node. A method for partitioning data may include sampling, at a device, data from one or more partitions based on a number of samples, transferring the sampled data from the device to a host, determining, at the host, one or more splitters based on the sampled data, communicating the one or more splitters from the host to the device, and partitioning, at the device, data for the one or more partitions based on the one or more splitters.

Abstract: Snapshot lifecycle management systems and methods are disclosed herein. An example method includes establishing a repository for a user, determining indices for the user, generating a snapshot lifecycle policy for the indices of the cluster. The snapshot lifecycle policy comprises snapshot gathering parameters that dictate when and how often snapshots of indices of the cluster are obtained, as well as retention parameters that control how long the snapshots are stored and when the snapshots are to be deleted. The method includes storing the snapshots for the indices of the cluster in the repository according to the snapshot gathering parameters, and managing retention of the snapshots stored in the repository according to the retention parameters.

Abstract: A system, method, and machine-readable storage medium for determining an amount of unique data in a distributed storage system are provided. In some embodiments, a combined efficiency set for a first data set stored in the distributed storage system, such as at a volume, may be generated. The first data set may include a first subset of data and a second subset of data in the distributed storage system. Additionally, a set of efficiency sets for the first subset of data may be generated. A set difference based on the combined efficiency set and the set of efficiency sets may be computed. An amount of memory used for storing unique data of the second subset of data may be estimated based on the set difference. The unique data may be present in the second subset of data but absent from the first subset of data.

Abstract: A system and method for improving storage system performance by reducing or avoiding load spike amplification when performing garbage collection is disclosed. A storage controller in a storage system tracks system load including write load and read load, as well as available free segments. The storage controller uses these tracked values as inputs and, with these inputs, generates a garbage collection rate. Where read load is included, a scaled portion of the read load is taken into consideration so that, as the number of free segments nears the minimum amount desired and to prevent garbage collecting too slowly, the read load is gradually excluded from the garbage collection rate determination. The garbage collection rate is therefore responsive to system load so that, in times of high system load, the rate reduces as much as is safe so that the write load takes priority with computing resources of the storage controller.

Abstract: A system and method for updating a configuration of a host system so that the memory sub-system of the host system emulates performance characteristics of a target memory sub-system. An example system determining a configuration of the host system, the host system comprising a memory sub-system; receiving, by a processing device, a request to emulate a characteristic of a target memory sub-system; analyzing a plurality of candidate configurations for the host system, wherein the plurality of candidate configurations comprises a candidate configuration that generates a load on the memory sub-system to decrease characteristics of the memory sub-system; and updating the configuration of the host system based on the plurality of candidate configurations, wherein the updated configuration changes the memory sub-system to emulate the characteristic of the target memory sub-system.

Abstract: An information processing system includes a first storage system for supplying a primary site, and a second storage system for supplying a secondary site. The first storage system is allowed to execute replication by transferring a processing history of a data volume of the first storage system to the second storage system, and to transfer multiple processing histories collectively. The first storage system integrates histories of multiple write accesses included in the multiple processing histories to be collectively transferred, which are duplicatedly addressed on the volume for transfer.

Abstract: A data access system including a processor, multiple cache modules for the main memory, and a storage drive. The cache modules include a FLC controller and a main memory cache. The multiple cache modules function as main memory. The processor sends read/write requests (with physical address) to the cache module. The cache module includes two or more stages with each stage including a FLC controller and DRAM (with associated controller). If the first stage FLC module does not include the physical address, the request is forwarded to a second stage FLC module. If the second stage FLC module does not include the physical address, the request is forwarded to the storage drive, a partition reserved for main memory. The first stage FLC module has high speed, lower power operation while the second stage FLC is a low-cost implementation. Multiple FLC modules may connect to the processor in parallel.

Abstract: A method for ranking includes the steps of receiving, at least one model from a central server to form at least one model node; training, the at least one model node with at least one data node to generate a trained model; generating, a weight from each of the at least one data node for the trained model; transferring, the weight from the at least one data node to the central server; inferencing, an inference output using the trained model and the data node; determining, an edge between the at least one data node and the model node, wherein the edge is determined depending on the influence of the data node or the model node on each other in generating the trained model and the inference output; determining a score for the data node and the model node based on the edge formed.

Abstract: System and methods for using a discovery controller to establish network connections in a network that comprises Non-Volatile Memory Express™ over fabrics (NVMe-oF™) entities that support multicast change notifications comprise: in response to a change that affects at an NVMe-oF™ entity that has not yet established a connection with the discovery controller, generating a multicast change notification (MCN) that notifies the unconnected entity of the change and automatically communicating the MCN to the unconnected entity without requiring an explicit persistent connection to the discovery controller, e.g., to access storage ports in the network.

Abstract: A distributed storage system and a data synchronization method are used with a network. The system includes a first network host and a second network host. A first file system directly writes data generated by the first network host into a first software-simulated persistent memory. The data in the first software-simulated persistent memory is stored into a first remote block device and cached, respectively. The cached data is stored into a first persistent storage by asynchronous writeback mechanisms. The first remote block device transmits the received data to the second software-simulated persistent memory through the network. The data transmitted to the second software-simulated persistent memory is cached, and the cached data is stored into a second persistent storage by asynchronous writeback mechanisms. The second network host replaces the first network host to provide services when the first network host is out of service.

Abstract: A method including determining a to-be-verified data block among a plurality of data blocks corresponding to a predetermined file in a distributed storage system, wherein a storage node where the to-be-verified data block is located comprises a storage node that meets a load balancing strategy in the distributed storage system; and verifying the to-be-verified data block. The present disclosure solves the technical problem in the conventional technologies in which the read/write performance at a user front end is affected during a data verification process.

Abstract: Disclosed embodiments relate to remote atomic operations (RAO) in multi-socket systems. In one example, a method, performed by a cache control circuit of a requester socket, includes: receiving the RAO instruction from the requester CPU core, determining a home agent in a home socket for the addressed cache line, providing a request for ownership (RFO) of the addressed cache line to the home agent, waiting for the home agent to either invalidate and retrieve a latest copy of the addressed cache line from a cache, or to fetch the addressed cache line from memory, receiving an acknowledgement and the addressed cache line, executing the RAO instruction on the received cache line atomically, subsequently receiving multiple local RAO instructions to the addressed cache line from one or more requester CPU cores, and executing the multiple local RAO instructions on the received cache line independently of the home agent.

Abstract: A command is transmitted to a storage device to relocate first data that partially fills a first erase block of the storage device and second data that partially fills a second erase block of the storage device to a third erase block of the storage device, wherein the command causes the relocation of the first data and the second data while bypassing sending the data to the storage controller. An acknowledgement that the first data and the second data have been stored at the third erase block is received from the storage device.

Abstract: A computer-based system and method for providing a distributed write buffer in a storage system, including: obtaining a write request at a primary storage server to store data associated with the write request in a non-volatile storage of the primary storage server; and storing the data associated with the write request in a persistent memory of the primary storage server or in a persistent memory of an auxiliary storage server based on presence of persistent memory space in the primary storage server. The write request may be acknowledged by the primary storage server after storing the data associated with the write request in the persistent memory of the primary storage server or in the persistent memory of the auxiliary storage server.

Abstract: A computer-implemented method, according to one embodiment, includes: processing records by, for each of the records: shearing the key associated with the record from the payload data, normalizing the sheared key, and storing the normalized sheared key in a first target area of memory. A determination is made whether a size of the payload data is outside a first predetermined range. In response to determining that the size of the payload data is outside the first predetermined range, the payload data is stored in a second target area of memory, and a data locator is appended to the normalized sheared key. Furthermore, in response to determining that a storage capacity of the memory is outside a second predetermined range, some of the payload data is transferred to external physical storage. Moreover, an external list is integrated with each of the data locators that correspond to the transferred payload data.

Abstract: A storage system includes a host including a processor and a memory unit, and a storage device including a controller and a non-volatile memory unit. The processor is configured to output a write command, write data, and size information of the write data, to the storage device, the write command that is output not including a write address. The controller is configured to determine a physical write location of the non-volatile memory unit in which the write data are to be written, based on the write command and the size information, write the write data in the physical write location of the non-volatile memory unit, and output the physical write location to the host. The processor is further configured to generate, in the memory unit, mapping information between an identifier of the write data and the physical write location.

Abstract: A data storage manager may manage storage locations for blocks of a storage volume. The blocks of the storage volume may be assigned to a logical volume exposed to a computing instance supported by a host. Furthermore, the data storage manager may also generate and maintain a set of rules that specify the locations of blocks of the storage volume, and provides the set of rules to the host. The set of rules may be included in a data structure enabling the host to access the blocks based at least in part on the information included in the set of rules.

Abstract: Examples may forward an input/output (IO) request with use of kernel-level instructions. Examples may receive the IO request via a port of a standby controller, generate an alternate version of the IO request using at least kernel-level instructions of the standby controller, and provide the alternate version of the IO request to physical memory of the active controller by providing the alternate version of the IO request to a designated region of physical memory of the standby controller that is mapped to a designated region of the physical memory of the active controller.

Abstract: According one embodiment, a memory device controlling method includes: receiving, by a first semiconductor memory, a read command transmitted from a controller; receiving, by a second semiconductor memory, a write command transmitted from the controller; reading, by the first semiconductor, data from the first semiconductor memory based on the read command, and transmitting, from the first semiconductor memory to the second semiconductor memory, the data and a control signal indicating that the data is output; and receiving, by the second semiconductor memory, the data at a timing based on the control signal transmitted from the first semiconductor memory without intermediation of the controller based on the write command and writing the received data into the second semiconductor memory.