Abstract: A technique for managing a datapath of a data storage system includes receiving a request to access target data and creating a transaction that includes multiple datapath elements in a cache, where the datapath elements are used for accessing the target data. In response to detecting that one of the datapath elements is invalid, the technique further includes processing the transaction in a rescue mode. The rescue mode attempts to replace each invalid datapath element of the transaction with a valid version thereof obtained from elsewhere in the data storage system. The technique further includes committing the transaction as processed in the rescue mode.

Abstract: A technique for managing a mapping structure in a data storage system includes identifying levels in the mapping structure of pages to be removed and estimating the workload required for removing the pages based at least in part on the identified levels. The technique further includes initiating processing of the pages based at least in part on the estimated workload.

Abstract: A method is used for managing overlapping communications regarding downtiering and invalidating cached data among nodes in a storage system. The method includes identifying, by a node, data to downtier. The node adds the data to an in progress table, and sends a request for permission to downtier the data to a peer node. The node receives permission from the peer node to downtier the data. In response to the permission to downtier the data, the node checks the in progress table for the data and takes an action based on a result of checking the in progress table.

Abstract: Techniques for detecting state changes in a system may include receiving a first neural network that is trained to detect when the system transitions into a first resulting state, wherein the system transitions into at least a first intermediate state prior to transitioning into the final resulting state; training the first neural network using a first plurality of inputs denoting the system in the first intermediate state; obtaining a plurality of sets of internal state information of the first neural network, each set of the plurality of sets denoting an internal state of the first neural network at a different point in time after the first neural network has processed at least a portion of the first plurality of inputs; and training a second neural network, using the plurality of sets of internal state information, to detect the first intermediate state.

Abstract: A method of validating metadata pages that map to user data in a data storage system is provided. The method includes (a) obtaining first information stored for a first metadata page and second information stored for a second metadata page, the first and second metadata pages having a relationship to each other within a hierarchy of metadata pages for accessing user data; (b) performing a consistency check between the first information and the second information, the consistency check producing a first result in response to the relationship being verified and a second result otherwise; and (c) in response to the consistency check yielding the second result, performing a corrective action to restore consistency between the first and second information. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A technique of protecting data from ransomware attacks identifies a set of blocks written to a data object between first and second points in time, determines a set of attributes of the set of blocks, and, in response to the set of attributes indicating a likelihood of a ransomware attack, secures a state of the data object as of the first point in time.

Abstract: A method, computer program product, and computing system for allocating a first number of tokens from a plurality of tokens for processing read IO requests from a read IO queue, thus defining a number of allocated read tokens. A second number of tokens may be allocated from the plurality of tokens for processing write IO requests from a write IO queue, thus defining a number of allocated write tokens. It may be determined that the processing of the write IO requests is throttled. In response to determining that the processing of the write IO requests from the write IO queue is throttled, a maximum allowable number of write tokens may be defined. Additional tokens may be allocated for processing the read IO requests from the read IO queue based upon, at least in part, the maximum allowable number of write tokens and the number of allocated write tokens.

Abstract: A method, computer program product, and computing system for maintaining a first cursor pointer for a memory system, wherein the first cursor pointer concerns a first search criteria and identifies an initial first cursor starting location; receiving a first search request for the memory system that satisfies the first search criteria; executing the first search request within an initial first search window starting at the initial first cursor starting location; and updating the first cursor pointer to identify an updated first cursor starting location based, at least in part, upon the initial first search window.

Abstract: Rotating disk drives can be partitioned into performance partitions (PPs) each including slices having similar performance in accordance with an allowable difference or tolerance for any of data throughput, bandwidth and data transfer rate. Each subdrive can correspond to slices of the rotating disk drives included in a single PP. Each PP can have a performance level denoting a relative performance ranking of PPs. Each PP tier can include PPs across multiple rotating disk drives have the same performance level. Resiliency sets (RSs) can be configured from the PP tiers where each RS can include slices of subdrives from only one of the PP tiers. Physical ubers or units (PUs) can be allocated from the RSs. Each PU can denote a mapped RAID group configured from slices of different subdrives of one RS. Spare slices can be reserved in each RS or a lowest performance PP tier.

Abstract: A method system, and computer program product for improving data reduction using aggregate machine learning systems comprising receiving, by an aggregating machine learning system from one or more machine learning systems associated with a set of one or more storage arrays, a first set of output parameters indicative of performance metrics for the set of the one or more storage arrays, aggregating, by the aggregating machine learning system, the first set of output parameters, resulting in a second set of output parameters, and sending, from the aggregating machine learning system, at least one member of the second set of output parameters as an input to at least one of the one or more machine learning systems associated with the set of the one or more storage arrays.

Abstract: A method performed by a block-storage server, of storing data is described. The method includes (1) receiving, from a remote file server, data blocks to be written to persistent block storage managed by the block-storage server; (2) receiving, from the remote file server, metadata describing files to which the data blocks belong in a set of filesystems managed by the remote file server; and (3) selectively applying data reduction when storing the data blocks in the persistent block storage based, at least in part, on the received metadata. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A method is used in managing deduplication characteristics in a storage system. Deduplication entries stored in a deduplication cache are categorized into a set of deduplication groups based on a data deduplication probability associated with the deduplication entries. A machine learning system is used to dynamically adjust deduplication characteristics associated with the set of deduplication groups based on an I/O workload associated with the storage system.

Abstract: Techniques are used for identifying allocated blocks in a storage system. The techniques may be used to provide, among other things, retrieval of a value associated with a most recent reinitialization of the storage system. For a tier of user data storage in the storage system, a page of storage with the value and a marker indicating that the page marks an end of storage that has been allocated for the tier of user data storage may be identified. Storage may be allocated from a page immediately preceding the identified page.

Abstract: A method for file system reorganization in the presence of inline compression includes obtaining a virtual block pointer for an original compressed segment to be reorganized, the original compressed segment comprising compressed allocation units of data stored in a storage system, wherein the virtual block pointer comprises an extent list identifying the compressed allocation units in the original compressed segment and a pointer to where the original compressed segment is stored; copying only the referenced compressed allocation units in the original compressed segment to a new compressed segment in a substantially contiguous manner; updating the extent list to identify the referenced compressed allocation units in the new compressed segment, and the pointer to where the new compressed segment is stored; and freeing the original compressed segment.

Abstract: A method is used for efficient storage allocation in a storage system. An infrastructure manager receives a request for storage from a first storage operating entity. The infrastructure manager identifies, from entries in a pagebin, a pagelist with sufficient available storage to satisfy the request. The infrastructure manager locks the pagebin while assigning the identified pagelist to the first storage operating entity. Upon a commitment of the identified pagelist by the first storage operating entity, the infrastructure manager unlocks the pagebin and identifies, from the entries in the pagebin, a second pagelist with sufficient available storage to satisfy a request from a second storage operating entity.

Abstract: A method performed by a block-storage server, of storing data is described. The method includes (1) receiving, from a remote file server, data blocks to be written to persistent block storage managed by the block-storage server; (2) receiving, from the remote file server, metadata describing a placement of the data blocks in a filesystem managed by the remote file server; and (3) organizing the data blocks within the persistent block storage based, at least in part, on the received metadata. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A data storage system in which data storage space for storing data storage metadata is selectively allocated from non-volatile data storage devices contained in a first storage enclosure. The first storage enclosure is directly connected to at least one storage processor of the data storage system, and the data storage system further includes at least one secondary storage enclosure that is indirectly connected to the storage processor. Data storage metadata is generated within the data storage system, and the data storage metadata generated within the data storage system is stored into the data storage space that was selectively allocated for storing data storage metadata from the non-volatile data storage devices contained in the first storage enclosure.

Abstract: Techniques for CPU cache efficiency may include performing concurrent processing, such as for first and second data operations, in a synchronized manner that prevents loading the same data chunk into the CPU cache more than once. Processing may include synchronizing the first and second data operations with respect to a first data chunk to ensure that both the first and second data operation processing has completed prior to proceeding with performing such processing on a second data chunk. The first and second data operations may be any two of deduplication, encryption, and compression, performed inline as part of the data path. In one embodiment, the first and second data operations for the first data chunk may be performed in parallel or sequentially where neither data operation proceeds with another data chunk until processing of the first and second data operations is complete for the first data chunk.

Abstract: A method, computer program product, and computer system for extending, by a computing device, transaction log page-buffers for Non-Volatile Random Access Memory (NVRAM) onto a solid state drive (SSD). It may be determined whether a bandwidth limit of the NVRAM has reached a threshold bandwidth. An IO may be processed on one of the NVRAM and the SSD based upon, at least in part, whether the bandwidth limit of the NVRAM has reached the threshold bandwidth.

Abstract: Techniques for managing storage may comprise: receiving a request for a first amount of free capacity, wherein the request includes a first priority denoting a purpose for which the first amount of free capacity is requested; determining whether a current utilization of storage exceeds a first threshold associated with the first priority; responsive to determining the current utilization of storage is less than the first threshold associated with the first priority, performing first processing including: determining whether there is a sufficient amount of existing free capacity to grant the first amount; and responsive to determining there is a sufficient amount of existing free capacity to grant the first amount, granting the first amount of free capacity; and responsive to determining the current utilization of storage is not less than the first threshold associated with the first priority, rejecting the request and not granting the first amount of free capacity.