Abstract: A storage system distributes data across multiple nodes, where the data is stored with partial redundancy to meet a predetermined reliability level. The system monitors a number of nodes in the storage system to determine whether a number of nodes changes. Responsive to determining that the number of nodes changes, the system determines whether reliability level of the storage system exceeds the predetermined reliability level by a threshold. Responsive to determining that the reliability level of the storage system exceeds the predetermined reliability level by the threshold, the system adjusts a data storing scheme of the storage system to reduce redundancy and increase available storage capacity in the storage system, while maintaining the predetermined reliability level.

Abstract: A file system stores data in a hierarchical storage system comprising a plurality of levels of storage units. If an indication of failure of a storage unit at a lower level is received, the storage system indicates a storage unit at the higher level as failed. The storage system initiates reconstruction of data at the storage unit at the higher level based on redundant data stored other storage units at the higher level. If the number of failed storage units at the higher level exceeds a threshold value, the storage system indicates that the storage unit at the higher level is available. The storage system initiates reconstruction of data at the storage unit at the lower level using redundant data stored in other storage units at the lower level. The storage system may perform data reconstruction at the higher level and lower level concurrently.

Abstract: A method or system for managing data placement in multi-tier storage devices. The system monitors usage of data items stored across storage devices, each associated with a performance tier. The system collects usage metrics for the data items, including access frequency, recency, file size, growth rate, sequentiality, or temporal access patterns. Multiple tiering methods are executed in parallel, with each method analyzing at least a subset of the usage metrics to generate tiering recommendations. These recommendations indicate whether each data item should be promoted to a higher-performance tier, demoted to a lower-performance tier, or retained in its current tier. The system aggregates the outputs of the tiering methods to compute a consensus decision for each data item. Based on the consensus, the system relocates data items to appropriate storage tiers.

Abstract: A storage system stores data on a persistent storage device and intent log on a high performance persistent storage device. The storage system receives requests to write data and in response, writes the data to the persistent storage device. The storage system receives a request to write data and a request to read data before the write request is processed. The storage system determines whether the data being read by the read request overlaps the data being written by the write request. If the data being read overlaps the data being written, the storage system writes the data being written in the intent log stored on the high performance persistent storage device. Subsequent to storing the data in the intent log on the high performance persistent storage device, the storage system sends the data being read to the client that sent the read request.

Abstract: A storage system distributes data across multiple nodes, where the data is stored with partial redundancy to meet a predetermined reliability level. The system monitors a number of nodes in the storage system to determine whether a number of nodes changes. Responsive to determining that the number of nodes changes, the system determines whether reliability level of the storage system exceeds the predetermined reliability level by a threshold. Responsive to determining that the reliability level of the storage system exceeds the predetermined reliability level by the threshold, the system adjusts a data storing scheme of the storage system to reduce redundancy and increase available storage capacity in the storage system, while maintaining the predetermined reliability level.

Abstract: A system monitors application-level performance metrics associated with a high-performance computing (HPC) application and resource-level performance metrics associated with resources provisioned for the HPC application. The system accesses an HPC deployment map representing placement and interconnection of the resources provisioned for the HPC application, and analyzes the application-level performance metrics, resource-level performance metrics, and the HPC deployment map to determine one or more recommended actions to improve the application-level and resource-level performance metrics. The system then executes at least one of the one or more recommended actions.

Abstract: A storage system receives an initial failure rate of a storage device based on manufacturer specifications and determining a first erasure coding configuration to meet minimum reliability metrics. Data is encoded in the storage device using the first configuration. Operational data is then collected to monitor failures, and a second failure rate is determined if deviations from the initial rate exceed a predetermined threshold. Based on the updated failure rate and reliability metrics, a second erasure coding configuration, different from the first erasure coding configuration, is determined. The system then re-encodes data using the second configuration.

Abstract: A method for mitigating the effects of malware is provided. The method includes determining a compressibility of a portion of data, determining a data corruption condition is satisfied based on the determined compressibility, and modifying a retention policy for retention of stored snapshots associated with the portion of data based on the satisfaction of the data corruption condition. The modifying of the retention policy includes generating a first snapshot associated with the portion of the data, prior to writing cached data associated with the portion of the data, writing the cached data associated with the portion of the data, and generating a second snapshot associated with the portion of the data, responsive to the deletion.

Abstract: A file system stores data in a hierarchical storage system comprising a plurality of levels of storage units. If an indication of failure of a storage unit at a lower level is received, the storage system indicates a storage unit at the higher level as failed. The storage system initiates reconstruction of data at the storage unit at the higher level based on redundant data stored other storage units at the higher level. If the number of failed storage units at the higher level exceeds a threshold value, the storage system indicates that the storage unit at the higher level is available. The storage system initiates reconstruction of data at the storage unit at the lower level using redundant data stored in other storage units at the lower level. The storage system may perform data reconstruction at the higher level and lower level concurrently.

Abstract: A data storage system can have one or more hosts connected to a data storage subsystem with the host having a host processor and the data storage subsystem having a controller. Write back data generated at the host triggers the host processor to allocate a cache location in the data storage subsystem where the generated data is subsequently stored. The generated write back data is written in a non-volatile destination address as directed by the controller prior to waiting for a secondary event with the generated data stored in both the cache location and the non-volatile destination address. Detection of the secondary event prompts the controller to signal the host processor that the cache location is free for new data.

Abstract: The presently disclosed technology is directed to combatting data theft, particularly of verified authentication data (or hashes thereof) such as login information, thumbprint data, digital signatures, identification numbers, and any other data that should be known to an accessor of stored data. The verified authentication data is initially saved for later comparison in a new type of memory, write-and-compare-only memory, where the data may be queried as to whether a to-be-verified value matches the stored verified value, but the stored and verified data is not read from the write-and-compare-only memory. This prevents a data breach by preventing the verified authentication data from being read by anyone, including those with access (whether legitimate or illegitimate) to any system connected to the write-and-compare-only memory.

Abstract: Systems and processes are disclosed to preserve data integrity during a storage controller failure. In some examples, a storage controller of an active-active controller configuration can back-up data and corresponding cache elements to allow a surviving controller to construct a correct state of a failed controller's write cache. To accomplish this, the systems and processes can implement a relative time stamp for the cache elements that allow the backed-up data to be merged on a block-by-block basis.

Abstract: Systems and methods are disclosed for multiple duration deduplication entries. A processor may be configured to perform data deduplication operations, including setting a first indicator for first metadata stored to a hash database identifying a first time interval to maintain the first metadata in the database, and setting a second indicator for second metadata stored to the database identifying a second time interval to maintain the second metadata in the database longer than the first time interval. The processor may remove the first metadata from the database when no duplicative data is detected for the first metadata within the first time interval, and remove the second metadata from the database when no duplicative data is detected for the second metadata within the second time interval.

Abstract: Systems and processes are disclosed to preserve data integrity during a storage controller failure. In some examples, a storage controller of an active-active controller configuration can back-up data and corresponding cache elements to allow a surviving controller to construct a correct state of a failed controller's write cache. To accomplish this, the systems and processes can implement a relative time stamp for the cache elements that allow the backed-up data to be merged on a block-by-block basis.

Abstract: Systems and processes are disclosed to preserve data integrity during a storage controller failure. In some examples, a storage controller of an active-active controller configuration can back-up data and corresponding cache elements to allow a surviving controller to construct a correct state of a failed controller's write cache. To accomplish this, the systems and processes can implement a relative time stamp for the cache elements that allow the backed-up data to be merged on a block-by-block basis.

Abstract: Use of a deduplication module remote from a storage system for storage of data subject to deduplication by the remote deduplication module. The storage system may generate a hash of target data and send the hash of the target data to the remote deduplication module. The remote deduplication module may access a hash database corresponding to the storage system and apply a deduplication algorithm on the hash. In turn, information regarding a deduplication result may be returned to the storage system which may either store the unique target data or update a mapping between the target data and duplicate existing data in a persistent storage of the storage device. The remote deduplication module may be executed by scalable cloud resources such that the computational resources dedicated to the deduplication module may be scaled.

Abstract: A data storage network may have multiple data storage devices that each consist of a device buffer. A network buffer and buffer circuit can be found in a network controller with the buffer circuit arranged to divide and store data associated with a data access request in the network buffer and the device buffer of the first data storage device.

Abstract: Systems and processes are disclosed to preserve data integrity during a storage controller failure. In some examples, a storage controller of an active-active controller configuration can back-up data and corresponding cache elements to allow a surviving controller to construct a correct state of a failed controller's write cache. To accomplish this, the systems and processes can implement a relative time stamp for the cache elements that allow the backed-up data to be merged on a block-by-block basis.

Abstract: The presently disclosed technology is directed to combatting data theft, particularly of verified authentication data (or hashes thereof) such as login information, thumbprint data, digital signatures, identification numbers, and any other data that should be known to an accessor of stored data. The verified authentication data is initially saved for later comparison in a new type of memory, write-and-compare-only memory, where the data may be queried as to whether a to-be-verified value matches the stored verified value, but the stored and verified data is not read from the write-and-compare-only memory. This prevents a data breach by preventing the verified authentication data from being read by anyone, including those with access (whether legitimate or illegitimate) to any system connected to the write-and-compare-only memory.

Abstract: A data storage system includes a host system configured to provide an I/O request stream and a data storage subsystem. The data storage subsystem has a data storage device and a processor. The processor is configured to provide streaming data in response to the I/O request stream, determine a current streaming rate of the I/O request stream, and determine a time-based read ahead size based on a time ahead duration and the current streaming rate of the I/O request stream. The time-based read ahead size may be updated while streaming, for example, after a delay period.