Abstract: In example implementations, a computer system includes first memory for file storage and second memory storing a first mapping table and a second mapping table. The first mapping table associates user addresses with fingerprints and the second mapping table associates the fingerprints with storage locations of the first memory. Instructions cause one or more processors to receive an encrypted data file associated with a received user address and a fingerprint associated with the received encrypted data file. Based on the received fingerprint, it is determined whether the received encrypted data file is a duplicate of a previously stored data file. If the received encrypted data file is not a duplicate, the received encrypted data file is stored in the first memory and the first and second mapping tables are updated. If the received file is a duplicate, the first mapping table is updated.

Abstract: Systems and methods are provided for automatically generating an embedding that is linked to a newly created data object by its primary key. For example, in response to entering a data object into an object data structure of the object storage system, the system may automatically generate an embedding comprising a primary key of the data object that links the embedding with the data object. In response to receiving an update of the data object, the system may automatically identify the primary key of the data object and synchronize, using a notification service of the object storage system, the embedding with the update of the data object.

Abstract: In certain examples, a method includes obtaining request metadata, corresponding to requests received at a storage device, to generate a local training dataset; performing training of a local ML model at the storage device for predicting future data accesses and classifying data to be stored in storage types accessible by the storage device; sharing local ML model updates; obtaining a global ML model that incorporates the local ML model updates and ML model updates from other storage devices; performing additional training using the global ML model until the global ML model converges as a trained global ML model; executing the trained global ML model at the storage device to determine storage types in which to store data units and to predict future data requests; and storing a data unit in one or more of the storage types based on an output of the trained global ML model.

Abstract: Example implementations relate to operations in a storage system. An example implementation includes receiving a write of a first data unit to a first address in a volume, and populating a first volume entry, in a base volume table, with a reference to a first version table. The example implementation also includes, in response to detecting a snapshot trigger event, generating a first snapshot table and a second version table, where the first snapshot table is a copy of the base volume table. The example implementation also includes, subsequent to generating the first snapshot table, receiving a write of a second data unit to a second address in the volume, and populating a second volume entry, in the base volume table, with a reference to a second version table.

Abstract: In some examples, a system applies an inline detection of a write of data in a storage, the inline detection to detect potential data encryption of the data. In response to an indication of the potential data encryption, the system creates a first object that represents a first version of the data, and applies a further analysis to determine whether the potential data encryption constitutes unauthorized data encryption, the further analysis based on the first object and a second object that represents a second version of the data that is prior to the first version of the data.

Abstract: Systems and methods are provided for lock-free thread scheduling. Threads may be placed in a ring buffer shared by all computer processing units (CPUs), e.g., in a node. A thread assigned to a CPU may be placed in the CPU's local run queue. However, when a CPU's local run queue is cleared, that CPU checks the shared ring buffer to determine if any threads are waiting to run on that CPU, and if so, the CPU pulls a batch of threads related to that ready-to-run thread to execute. If not, an idle CPU randomly selects another CPU to steak threads from, and the idle CPU attempts to dequeue a thread batch associated with the CPU from the shared ring buffer. Polling may be handled through the use of a shared poller array to dynamically distribute polling across multiple CPUs.

Abstract: In some examples, a storage system intercepts a write request communicated over a network from a requester in a host system, the write request to write data of a data volume. The storage system determines whether the data of the write request matches a specified pattern. In response to determining that the data of the write request does not match the specified pattern, the storage system indicates that the write request from the requester in the host system has been corrupted by malware that has performed an unauthorized encryption on the data.

Abstract: Example implementations relate to storing data in a storage system. An example includes accessing a first portion of a data stream to be stored in a storage system; selecting sample data blocks included in the first portion; determining entropy values based on the sample data blocks; selecting, based on the sample data blocks, a entropy threshold from multiple precalculated entropy thresholds; determining whether the generated set of entropy values matches the selected entropy threshold within a probability level; and in response to a determination that the generated set of entropy values matches the selected entropy threshold within the probability level, identifying the first portion of the data stream as potentially including encrypted data affected by a ransomware attack.

Abstract: Example implementations relate to storing data in a storage system. An example includes receiving, by a storage controller of a storage system, a data unit to be stored in persistent storage of the storage system. The storage controller calculates multiple entropy values for the data unit. The storage controller selects, based on the multiple entropy values, at least one reduction operation from multiple different reduction operations. The storage controller performs the selected at least one reduction operation on the received data unit.

Abstract: In some examples, a computer system computes a rate of operations that involves a first system, and classifies, using a classifier, a request for an operation. The computer system determines a relationship between the computed rate of operations and a dynamic threshold rate determined during a training phase, and based on the determined relationship and a classification of the request by the classifier, selectively activates or disables an operational feature of the first system.

Abstract: Example implementations relate to storing data in a storage system. An example includes receiving, by a storage controller of a storage system, a data unit to be stored in persistent storage of the storage system. The storage controller determines maximum and minimum entropy values for the received data unit. The storage controller determines, based on at least the minimum entropy value and the maximum entropy value, whether the received data unit is viable for data reduction. In response to a determination that the received data unit is viable for data reduction, The storage controller performs at least one reduction operation on the received data unit.

Abstract: In some examples, a storage system creates a first copy of a data volume, and receives write requests having a specified characteristic from a host system, the write requests to write data of the data volume, where the storage system is to reject the write requests having the specified characteristic and to accept write requests without the specified characteristic. The storage system maintains metadata for the first copy of the data volume, the metadata indicating blocks of the data volume that have changed since the first copy of the data volume was created. The storage system determines, using the metadata, whether an unauthorized data encryption of the data of the data volume has occurred.

Abstract: Example implementations relate to storing data in a storage system. An example includes accessing a first portion of a data stream to be stored in a storage system; selecting sample data blocks included in the first portion; determining entropy values based on the sample data blocks; selecting, based on the sample data blocks, a entropy threshold from multiple precalculated entropy thresholds; determining whether the generated set of entropy values matches the selected entropy threshold within a probability level; and in response to a determination that the generated set of entropy values matches the selected entropy threshold within the probability level, identifying the first portion of the data stream as potentially including encrypted data affected by a ransomware attack.

Abstract: Example implementations relate to storing data in a storage system. An example includes receiving, by a storage controller of a storage system, a data unit to be stored in persistent storage of the storage system. The storage controller determines maximum and minimum entropy values for the received data unit. The storage controller determines, based on at least the minimum entropy value and the maximum entropy value, whether the received data unit is viable for data reduction. In response to a determination that the received data unit is viable for data reduction, The storage controller performs at least one reduction operation on the received data unit.

Abstract: In some examples, a system dynamically adjusts a prefetching load with respect to a prefetch cache based on a measure of past utilizations of the prefetch cache, wherein the prefetching load is to prefetch data from storage into the prefetch cache.

Abstract: In some examples, a storage system intercepts a write request communicated over a network from a requester in a host system, the write request to write data of a data volume. The storage system determines whether the data of the write request matches a specified pattern. In response to determining that the data of the write request does not match the specified pattern, the storage system indicates that the write request from the requester in the host system has been corrupted by malware that has performed an unauthorized encryption on the data.

Abstract: In some examples, a storage system creates a first copy of a data volume, and receives write requests having a specified characteristic from a host system, the write requests to write data of the data volume, where the storage system is to reject the write requests having the specified characteristic and to accept write requests without the specified characteristic. The storage system maintains metadata for the first copy of the data volume, the metadata indicating blocks of the data volume that have changed since the first copy of the data volume was created. The storage system determines, using the metadata, whether an unauthorized data encryption of the data of the data volume has occurred.

Abstract: In some examples, a computer system computes a rate of operations that involves a first system, and classifies, using a classifier, a request for an operation. The computer system determines a relationship between the computed rate of operations and a dynamic threshold rate determined during a training phase, and based on the determined relationship and a classification of the request by the classifier, selectively activates or disables an operational feature of the first system.

Abstract: In some examples, a system samples a subset of input/output (I/O) accesses of a storage, the I/O accesses being part of a workload. The system determines, based on the sampled subset of the I/O accesses, a first reuse distance distribution for a first time interval, determines a similarity measure representing a similarity of the first reuse distance distribution and a second reuse distance distribution for a second time interval different from the first time interval, and based on a change in the similarity measure, triggers a workload placement process to determine a placement of the workload on a compute node of a plurality of compute nodes.

Abstract: Systems and methods are provided for lock-free thread scheduling. Threads may be placed in a ring buffer shared by all computer processing units (CPUs), e.g., in a node. A thread assigned to a CPU may be placed in the CPU's local run queue. However, when a CPU's local run queue is cleared, that CPU checks the shared ring buffer to determine if any threads are waiting to run on that CPU, and if so, the CPU pulls a batch of threads related to that ready-to-run thread to execute. If not, an idle CPU randomly selects another CPU to steak threads from, and the idle CPU attempts to dequeue a thread batch associated with the CPU from the shared ring buffer. Polling may be handled through the use of a shared poller array to dynamically distribute polling across multiple CPUs.