Abstract: A technique of protecting data in a data storage system includes, in response to receiving a write request that specifies data to be written in the data storage system, calculating an error detection code (EDC) of the specified data. The technique further includes persisting the specified data and the EDC in persistent cache and, in response to receiving a read request for reading the specified data, validating the specified data using the EDC before returning the specified data in satisfaction of the read request.

Abstract: A technique for managing data storage includes generating entropy of blocks on a per-block basis and selectively performing inline compression on blocks based at least in part on their entropy. Entropy of a block provides a rough measure of the block's compressibility. Thus, using per-block entropy enables a storage system to steer compression decisions, e.g., whether to compress and/or how much to compress, flexibly and with high granularity, striking a balance between throughput and storage efficiency.

Abstract: In response to failure of a data storage drive, data previously stored on the failed drive is rebuilt on a replacement data storage drive by executing drive rebuild jobs on a first set of processor cores. While the data previously stored on the failed data storage drive is being rebuilt, a second set of processor cores is used to process host I/O (Input/Output) requests. The host I/O requests processed by the second set of processor cores may be a first set of host I/O requests processed during the data rebuild. When a current total number of outstanding host I/O requests exceeds a threshold maximum number of outstanding host I/O requests, at least one processor core in the first set of processor cores may be used to process a second set of host I/O requests while the data previously stored on the failed data storage drive is being rebuilt.

Abstract: A method is used in ensuring data integrity in data storage. A set of physical pages that include data for a virtual page is identified on a storage device. A set of reconstructed pages is generated from other storage devices, and each reconstructed page corresponds to a page in the set of physical pages. One or more pages from the set of physical pages and the set of reconstructed pages with data yielding a checksum that matches a checksum for the virtual page is identified, and data corresponding to the virtual page from the identified one or more pages is obtained.

Abstract: A method, system, and computer program product for multi-generational virtual block compaction comprising identifying a first virtual block, the first virtual block being associated with a first generation number, determining a second virtual block as an appropriate target for live information in the first virtual block, creating an association between the second virtual block and the first virtual block, updating the live information in the first virtual block to be associated with the second virtual block, updating a generation information mapping associated with the first virtual block; and associating a second generation number with the first virtual block.

Abstract: Techniques for providing a direct IO path to compressed data on storage media of a storage system. The techniques include triggering a transaction cache to perform a flush operation for updating mapping metadata for a storage object containing the compressed data. Having updated the mapping metadata for the storage object, the techniques further include issuing, by a copier module, an IO read request for the compressed data of the storage object to a namespace layer, which issues the IO read request to a mapping layer. The techniques further include forwarding the IO read request to a logical layer of the mapping layer, bypassing the transaction cache. The techniques further include reading, by the logical layer, the compressed data of the storage object from the storage media, and providing, via the mapping layer and the namespace layer, the compressed data to the copier module for transfer to a destination storage system.

Abstract: A method, computer program product, and computing system for receiving content on a high-availability storage system. The content is compared to one or more entries in a static database associated with a cache memory system of the high-availability storage system. If the content does not match the one or more entries in the static database, the content is compared to one or more entries in a dynamic database associated with the cache memory system. If the content does not match the one or more entries in the dynamic database: the content is written to the cache memory system and a representation of the content is written to a temporal database associated with the cache memory system and maintained for a defined period of time.

Abstract: A method, computer program product, and computer system for identifying a bit for an allocation unit. It may be determined if data has been modified on the allocation unit while degraded. A rebuild of the allocation unit may be executed when the bit is a first value. The rebuild of the allocation unit may be skipped when the bit is a second value.

Abstract: A method of managing CPU cores in a data storage apparatus (DSA) configured to perform both host I/O (Input/Output) processing and background storage processing is provided. The method includes (a) selectively classifying background storage tasks in one of a first classification for longer-running tasks and a second classification for shorter-running tasks; (b) selecting CPU cores that are running fewer than a threshold number of first-classification background tasks to process host I/O requests; and (c) processing the host I/O requests on their selected CPU cores. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A method is used for efficient storage allocation in a storage system. An infrastructure manager receives a request for storage from a first storage operating entity. The infrastructure manager identifies, from entries in a pagebin, a pagelist with sufficient available storage to satisfy the request. The infrastructure manager locks the pagebin while assigning the identified pagelist to the first storage operating entity. Upon a commitment of the identified pagelist by the first storage operating entity, the infrastructure manager unlocks the pagebin and identifies, from the entries in the pagebin, a second pagelist with sufficient available storage to satisfy a request from a second storage operating entity.

Abstract: A technique of protecting data in a data storage system includes, in response to receiving a write request that specifies data to be written in the data storage system, calculating an error detection code (EDC) of the specified data. The technique further includes persisting the specified data and the EDC in persistent cache and, in response to receiving a read request for reading the specified data, validating the specified data using the EDC before returning the specified data in satisfaction of the read request.

Abstract: In response to detecting the failure of a data storage drive, a drive rebuild operation is performed. The drive rebuild operation is performed by periodically i) calculating a target rebuild rate that enables data previously stored on the failed data storage drive to be completely rebuilt on at least one other data storage drive within an expected drive rebuild time window, ii) calculating a new value for at least one drive rebuild parameter based on a value for the drive rebuild parameter corresponding to the target rebuild rate indicated by a static rebuild rate data structure and a current level of host I/O (Input/Output) activity, and iii) rebuilding the data previously stored on the failed data storage drive on the at least one other data storage drive for a predetermined time period with the at least one drive rebuild parameter set to the new value for the drive rebuild parameter.

Abstract: A method of validating metadata pages that map to user data in a data storage system is provided. The method includes (a) obtaining first information stored for a first metadata page and second information stored for a second metadata page, the first and second metadata pages having a relationship to each other within a hierarchy of metadata pages for accessing user data; (b) performing a consistency check between the first information and the second information, the consistency check producing a first result in response to the relationship being verified and a second result otherwise; and (c) in response to the consistency check yielding the second result, performing a corrective action to restore consistency between the first and second information. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: Host data stored in one or more source physical extents of non-volatile data storage is identified as valid and determined to be infrequently written by host I/O requests, and is therefore compressed to generate a highly compressed version of the valid host data. The highly compressed version is then stored into at least one target physical extent. The valid host data may be initially compressed before it is stored in the source physical extent(s), and may be re-compressed to generate the highly compressed version. If the valid host data is also infrequently read, it may be recompressed using larger blocks of host data than were used to perform the initial compression. The performance tier of the target physical extent may be different from (e.g. lower than) the performance tier of the source physical extent. The technology may be embodied in a background process such as a garbage collector.

Abstract: Coordinating a reclaiming of data storage space among processing nodes of a data storage system includes, by a first node in response to an event, performing local cleanup of first-node mapping data and issuing a request message to a second node, the request message identifying the data storage space to be reclaimed. The event may be a notification from a RAID component rebuilding a RAID volume. In response to the request message, the second node performs local cleanup of second-node mapping data and issues a completion message to the first node indicating that the second node has completed its part of the reclaiming of the data storage space. The first node responds to the completion message by marking the data storage space as being reclaimed and signaling to a source of the event that the data storage space is available for new use.

Abstract: Techniques for data processing may include: receiving a data chunk of the data set; determining, in accordance with criteria including a compressibility ratio for the data set and a cost ratio of compression computation cost and entropy computation cost, whether to activate or deactivate entropy computation for the data set, wherein the compressibility ratio is ratio of a number of compressible data chunks of the data set and a number of uncompressible data chunks of the data set; and responsive to determining to activate entropy computation for the data set, performing first processing comprising: determining an entropy value for the data chunk; and determining, in accordance with the entropy value for the data chunk, whether to compress the data chunk.

Abstract: An allocation history is maintained for each node in a multi-node data storage system. The allocation history for each node indicates sets of physical blocks previously allocated to the node to store dirty pages of user data flushed from a cache of the node. The allocation history indicates non-quarantined sets of physical blocks that are currently being used to provide non-volatile data storage to cache flush operations, and quarantined sets of physical blocks not currently being used to provide non-volatile data storage to cache flush operations. Allocation is prevented to any node of any set of physical blocks that is indicated as non-quarantined by any of the allocation histories, and of any set of physical blocks that is indicated as quarantined by any of the allocation histories.

Abstract: A technique for managing write requests in a data storage system checks whether newly-arriving data match previously-stored data that have been recorded in a deduplication database. If a match is found, the technique compares mapping metadata for the newly-arriving data with mapping metadata for the matching data. If both sets of metadata point to the same storage location, then the newly-arriving data is a same-data write and a new write to disk is avoided.

Abstract: A method is used in managing inline data compression in storage systems. A request is received to flush data of a file system to a storage device of a storage system. A compression ratio is determined for writing the data to the storage device in a compressed format. Based on the determination, a number of allocation units is determined in which a segment is apportioned. The segment comprises of a set of allocation units in the compressed format. Based on the determination, the data is written to the segment in the compressed format.

Abstract: A data storage system performs updating of a set of global counters stored in persistent storage accessible to a plurality of processing nodes of a data storage system, the global counters storing respective global count values describing associated units of data storage shared by the processing nodes for performing data storage operations. The updating includes, by each processing node, using a respective set of in-memory delta counters storing respective delta count values to track changes to respective global count values due to data storage operations performed by the processing node with respect to the units of data storage. A summarization process is regularly executed to merge the delta count values of the delta counters of all processing nodes into the respective global counters.