Abstract: Techniques for data processing may include: receiving a page P having a binary representation; determining a plurality of features for the page P; determining a plurality of feature hashes for the plurality of features, wherein each of the plurality of feature hashes corresponds to a different one of the plurality of features, wherein said each feature hash is determined using a hash function and in accordance with said different one of the plurality of features and an index associated with said different one of the plurality of features; and determining, in accordance with the plurality of feature hashes for the plurality of features, a similarity hash for the page P.

Abstract: Techniques are used for enabling deduplication for asynchronous replication in a storage system. The techniques may be used to provide, among other things, retrieving, on a source system, a virtual index key corresponding to a logical address, and determining whether a deduplication index table for a target system includes an entry with the same virtual index key.

Abstract: A method, computer program product, and computer system for identifying, by a computing device, an attribute of a lock object associated with data. It may be determined that the lock object is acquired by a first task performing a transaction with the data based upon, at least in part, the attribute of the lock object. The lock object may be acquired by a second task by releasing the lock object from the first task prior to the transaction with the data being completed, wherein the second task is a higher priority task than the first task.

Abstract: An aspect of determining per volume exclusive capacity in a deduplication system includes setting a percentage of a population of pages for selection. For each of the pages, an aspect includes selecting a page in the population, providing a data segment facilitating multiple references of the segment by at least one storage entity, maintaining counts corresponding with each segment in the page, and determining exclusive ownership of the page based on the counts and a key value of one of a plurality of storage entities.

Abstract: Techniques for processing operations may include: receiving, at a data storage system from a host, an operation requesting to perform an offload copy operation internally within the data storage system, wherein the operation indicates that the offload copy operation copies data from a source interval to a destination interval; inserting a record in a log file logging the offload copy operation; inserting a source node representing the source interval in a source interval tree, wherein each node in the source interval tree represents a source interval of an offload copy operation having a corresponding record in the log file; inserting a destination node representing the destination interval in a destination interval tree, wherein each node in the destination interval tree represents a destination interval of an offload copy operation having a corresponding record in the log file; and returning an acknowledgement to the host regarding the offload copy operation.

Abstract: A method, computer program product, and computer system for creating, by a computing device, a snapshot related to a snapshot family, wherein an inheritance tree may be maintained for each snapshot family. A write to the snapshot may be received. A hash-based overwrite mapping data structure may be updated based upon, at least in part, receiving the write to the snapshot.

Abstract: A technique for scheduling access to a resource arranges tasks into multiple classes, where each class has a respective share and a respective priority. The share of a class sets an amount of access allocated to the class, and the priority sets an order in which the class can use its share, with higher priority classes getting access before lower-priority classes. The technique assigns latency-critical tasks, such as synchronous I/O tasks, to a first class having the highest priority and assigns bandwidth-critical tasks, such as background I/O processing, to a second class having a lower priority.

Abstract: A technique for validating metadata includes creating log entries for virtualization structures pointed to by mapping pointers in a mapping tree and processing the log entries in multiple passes. A current pass validates a current level of redirection and creates new log entries to be processed during a next pass. The new log entries represent a next level of redirection, and as many next passes are processed in sequence as there are next levels of redirection.

Abstract: A method, computer program product, and computer system for partitioning, by a computing device, resources between a plurality of central processing unit (CPU) cores. A logical block address (LBA) space of a user may be divided into a slice with an affinity to a CPU core of the plurality of CPU cores. IO flow processing may be processed by the CPU core of the plurality of CPU cores associated with the LBA space divided into the slice.

Abstract: A technique for managing data storage includes generating digests of chunks of data. The digests are arranged to provide similar results for similar chunks but to weight contributions from different regions of chunks differently based on their entropy. Chunks that differ in low-entropy regions tend to produce more similar digests than do chunks that differ in high-entropy regions. The technique further includes identifying similar chunks based on similarity of their digests and preforming data reduction on one chunk by reference to another, storing any differences between the chunks separately.

Abstract: Determining and using deduplication estimates may include: determining two deduplication sample indexes (DSIs) for two logical device sets each including one or more logical devices, determining a Jaccard Similarity for the two DSIs, wherein the Jaccard Similarity denotes a measurement of similarity and mutual deduplication between the two logical device sets; determining, in accordance with one or more criteria, whether the two logical device sets should be located in different data storage systems or a same data storage system that performs data deduplication, wherein the one or more criteria uses the Jaccard Similarity in determining whether to locate the two logical device sets in the same data storage system or the different data storage systems; and responsive to determining that the two logical device sets should be located in the same data storage system, locating the two logical device sets in the same data storage system.

Abstract: A method, computer program product, and computing system for remotely storing first content received on a first processing node of a clustered computing environment onto a storage platform, wherein the clustered computing environment includes a plurality of processing nodes; locally storing metadata that identifies the location of the first content within the storage platform on the first processing node, thus defining first locally-stored differential metadata; and after the occurrence of a storage trigger event, instructing the first processing node write the first locally-stored differential metadata to a cluster metadata pool within the storage platform.

Abstract: Write operation may be persistently recorded in a log using PDESC (page descriptor)-PB (page block) pairs. The PDESC-PB pairs of the log may be flushed. Flushing the log may include: determining a working set of PDESC-PB pairs; partitioning the working set into buckets by mapping each PDESC-PB pair of the working set to a bucket using a function; flushing a portion of the PDESC-PB pairs of a first bucket of the working set; updating, at a point in time, a first BHFS (bucket highest flushed sequence ID) value for the first bucket, wherein the first BHFS denotes a first sequence ID and each sequence ID associated with a PDESC-PB pair of the portion flushed prior to the point in time is less than the first sequence ID; and reclaiming PBs of the portion. As part of recovery processing, BHFSs for the buckets may be used to detect invalid PDESCs.

Abstract: Processing requests may include: receiving a write request from a host at a first node of a system; and servicing the write comprising assigning, by the first node, a sequence identifier to the write request, wherein the sequence identifier is included in a subsequence of identifiers only assignable by the first node, performing in parallel a first operation that stores first data written by the write request in a cache, a second operation that stores a descriptor for the write request in the cache, and a third operation that sends the descriptor (including the sequence identifier) to a peer node of the system; determining by the first node that the first, second and third operations have successfully completed; and responsive to determining the first, second and third operations have successfully completed, sending an acknowledgement from the first node to a host indicating successful completion of the write request.

Abstract: A technique for performing deduplication extends a match found in a digest database by looking forward and/or backward to adjacent data elements. The technique performs data comparisons between data blocks adjacent to a candidate block and corresponding data blocks adjacent to an identified target block. If the data comparisons indicate that an adjacent candidate block matches an adjacent target block, then the adjacent candidate block may be deduplicated to the adjacent target block without having to compute a hash value of the adjacent candidate block or to perform a separate lookup into the digest database.

Abstract: A technique for managing data storage includes generating digests of chunks of data. The digests are arranged to provide similar results for similar chunks but to weight contributions from different regions of chunks differently based on their entropy. Chunks that differ in low-entropy regions tend to produce more similar digests than do chunks that differ in high-entropy regions. The technique further includes identifying similar chunks based on similarity of their digests and preforming data reduction on one chunk by reference to another, storing any differences between the chunks separately.

Abstract: A technique for managing deduplication performs partial-block matching opportunistically by leveraging information acquired during times when a storage system has available resources. The information identifies anchor blocks that are likely targets for partial-block matches, based on discovering that the anchor blocks belong to populations of blocks that have high similarity. When processing write requests, inline activities access anchor blocks that closely match newly arriving candidate blocks and perform partial-block deduplication against those anchor blocks.

Abstract: Techniques are used for enabling deduplication for asynchronous replication in a storage system. The techniques may be used to provide, among other things, retrieving, on a source system, a virtual index key corresponding to a logical address, and determining whether a deduplication index table for a target system includes an entry with the same virtual index key.

Abstract: A method, computer program product, and computing system for identifying one or more metadata pages stored in a storage array, thus defining a primary set of metadata pages. An alternative set of metadata pages may be generated from the primary set of metadata pages. A log of changes associated with the primary set of metadata pages may be generated. A copy of at least a portion of the primary set of metadata pages may be generated based upon, at least in part, the alternative set of metadata pages and the log of changes associated with the primary set of metadata pages. The alternative set of metadata pages may lag behind in time from the primary set of metadata pages by a predefined amount of time.

Abstract: A method of data reduction in a block-based data storage system includes selecting a starting position in a block based on a deterministic function of block data content. Then for an unaligned block beginning at the selected starting position, a block digest (e.g., block hash) is generated and compared with stored block digests of stored data blocks. If there is a match, and the stored block matches the unaligned block, then a reference to the stored block is stored in place of the unaligned block, and otherwise the unaligned block and a corresponding digest are stored. The storing of references to already stored blocks, without the constraint of observing aligned-block boundaries, realizes increased savings of physical storage space.