Abstract: An aspect includes implementing capacity reduction in a storage system includes for each of a candidate page and a target page in the storage system, identifying a subset of sectors having identical data or a minimum amount of non-identical data, performing a bit-wise exclusive OR (XOR) operation on sectors of the candidate page and the target page, determining entropy from results of the XOR operation. Upon determining the entropy is less than or equal to a threshold value, an aspect includes building a reference page from an XOR sector containing results of the bit-wise XOR operation, and performing a compression operation on the reference page.

Abstract: Techniques for performing cache management may include: partitioning entries of a hash table into buckets, wherein each of the buckets includes a portion of the entries of the hash table; configuring a cache, wherein said configuring includes allocating a section of the cache for exclusive use by each bucket; and performing first processing that stores a data block in the cache. The first processing may include: determining a hash value for a data block; selecting, in accordance with the hash value, a first bucket of the plurality of buckets, wherein a first section of the cache is used exclusively for storing cached data blocks of the first bucket; storing metadata used in connection with caching the data block in a first entry of the first bucket; and storing the data block in a first cache location of the first section of the cache.

Abstract: An aspect of managing data block compression in a storage system includes performing, for each block written to the storage system: bit-wise traversing the block, searching the block for a pattern indicating a repeating sequence of bits and, upon determining the pattern exists in the block and the repeating sequence of bits in the pattern exceeds a threshold value, removing the repeating sequence of bits from the block thereby yielding a reduced-size block.

Abstract: An aspect of implementing globally optimized partial deduplication of storage objects includes gathering pages that share a common feature, dividing the pages into groups based on commonality with corresponding representative pages, where each is assigned as a representative dedupe page for the corresponding groups. For each group in the groups of pages, an aspect also includes writing the pages to a corresponding physical container.

Abstract: An aspect of determining per volume exclusive capacity in a deduplication system includes setting a percentage of a population of pages for selection. For each of the pages, an aspect includes selecting a page in the population, providing a data segment facilitating multiple references of the segment by at least one storage entity, maintaining counts corresponding with each segment in the page, and determining exclusive ownership of the page based on the counts and a key value of one of a plurality of storage entities.

Abstract: An aspect of performance optimization in a storage system environment includes providing a metadata structure that describes a storage space of a storage system. For each storage object of a plurality of storage objects, an aspect includes monitoring input/output (IO) patterns for IO operations conducted, monitoring deduplication characteristics of the storage object, and configuring local mapping layer parameters. An aspect further includes modifying local object metadata in the metadata structure as a function of the monitoring and the configuring.

Abstract: A method, computer program product, and computer system for identifying, by a computing device, a plurality of blocks. A maximum number of blocks of the plurality of blocks capable of being copied to a new block may be identified. Data from the maximum number of blocks of the plurality of blocks may be copied to the new block.

Abstract: A method comprising: storing, in a memory, a mapping tree that is implemented by using an array of mapping pages, the mapping tree having a depth of D, wherein D is an integer greater than or equal to 0; receiving a write request that is associated with a first type-1 address; storing, in a storage device, data associated with the write request, the data associated with the write request being stored in the storage device based on a first type-2 address; generating a map entry that maps the first type-1 address to the first type-2 address; calculating a first hash digest of the first type-1 address; and storing the map entry in a first mapping page.

Abstract: A method, computer program product, and computer system for storing data in a bucket of a plurality of buckets. A spare bucket may be reserved in the plurality of buckets. A copy of the data may be stored in the spare bucket. A pointer to the data in the bucket and a pointer to the copy of the data in the spare bucket may be updated based upon, at least in part, storing the data in the bucket and storing the copy of the data in the spare bucket.

Abstract: Techniques for determining unique ownership of data blocks includes selecting a first data block referenced by a first logical address of a first logical device, wherein first metadata for the first data block includes a reference count and a result used in determining sole unique ownership of the first data block; determining C1, an expected value of the result of the first metadata when the first logical device is a sole unique owner of the first data block; and performing first processing that determines whether the first logical device is the sole unique owner of the first data block. The first processing includes: determining whether C1 equals a current value of the result of the first metadata; and determining that the first logical device is the sole unique owner of the first data block if C1 equals the current value of the result of the first metadata.

Abstract: A method, computer program product, and computer system for identifying, by a computing device, a list of objects containing a plurality of physical layer blocks (PLBs). One or more next PLBs of the plurality of PLBs may be allocated from a selected free object of the list of objects. One or more additional free objects from the list of objects may be generated. Garbage collection may be performed between an inactive object of the plurality of objects and the selected free object.

Abstract: Virtual block redirection clean-up is accomplished for a virtual block in a mapper tree for a logical volume that is redirected to a destination virtual block in the mapper tree. In response to redirection of the virtual block, a redirection notification is sent to each component in a set of dependent components. Each dependent component may store one or more pointers to pointers located within virtual blocks of the mapper tree. The redirected virtual block is cleaned up in response to determining that each dependent component has expressly indicated, in response to the redirection notification, that it does not store any pointer to any pointer that is located within the redirected virtual block.

Abstract: A storage system comprises a plurality of storage devices and an associated storage controller. The storage controller is configured to receive a request to copy a first range of logical addresses to a second range of logical addresses, determine at least one physical block of the storage devices to which the first range of logical addresses is mapped, map the second range of logical addresses to the determined at least one physical block, and add at least one content-based signature associated with the determined at least one physical block to a pending increment data structure that includes content-based signatures corresponding to physical blocks for which an increment of an associated reference count is pending. The storage controller is further configured to execute a pending increment of a reference count associated with a given physical block corresponding to at least one of the content-based signatures in the pending increment data structure.

Abstract: An apparatus in one embodiment comprises at least one processing device. The processing device is configured to generate a hash entry of a hash table implementing a key-value store that comprises a plurality of buckets associated with corresponding keys where a given bucket comprises a bucket index having two or more entries for storage of values that hash to a key associated with the given bucket. The processing device is also configured to allocate blocks in non-volatile memory, to split the given bucket into chunks each comprising at least one entry of the bucket index, and to persist the hash table to the non-volatile memory utilizing a direct mapping of the chunks of the given bucket to respective designated block offset positions in the allocated blocks. Persisting the hash table comprises storing first and second chunks of the given bucket in first and second allocated blocks.

Abstract: A method of performing deduplication is provided. The method includes (a) applying an ownership model in assigning digest values to processing nodes configured for active-active writing to a storage object by performing an operation that distinguishes a first class of digest values from a second class of digest values, the first class of digest values assigned to a first processing node and the second class of digest values assigned to a second processing node; (b) performing deduplication lookups by the first processing node for digest values belonging to the first class; and (c) directing the second processing node to perform deduplication lookups for digest values belonging to the second class. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: Techniques for providing metadata (and/or data) protection in a data storage system. The techniques can include storing, for a specific data page, two (2) instances of a main metadata page and one (1) instance of a corresponding alternative metadata page in a protected storage object referred to herein as the “2+1-way mirror”. For the specific data page, two (2) instances of a main metadata page can be stored on a first drive and a second drive, respectively, of the 2+1-way mirror such that the first and second drives each have a copy of the same main metadata page. Further, an instance of a corresponding alternative metadata page can be stored on a third drive of the 2+1-way mirror. The 2+1-way mirror provides protection against metadata loss due to the concurrent failure of two (2) physical drives, and further provides protection against catastrophic metadata errors and/or inconsistencies due to software failures.

Abstract: A technique is directed to managing a redundant array of independent disks (RAID) group. The technique involves forming the RAID group from storage devices of different types that provide different data storage characteristics. The technique further involves receiving input/output (I/O) requests, and performing I/O operations on the RAID group in response to the I/O requests.

Abstract: Techniques for providing an efficient virtualization layer structure in a data storage system. The techniques include implementing multiple layers of indirection for accessing host data in the data storage system, including a mapping layer, a virtualization layer, and a physical layer. The virtualization layer includes virtual layer blocks (VLBs), each VLB including virtual pointers. Each virtual pointer is pointed to by one or more leaf pointers in the mapping layer, and points to a data block in the physical layer. The techniques include generating, for each virtual pointer in the VLB, reference count metadata to keep track of the number of leaf pointers pointing to the virtual pointer, and maintaining, in a metadata page, the reference count metadata for the virtual pointers in a three (3)-way mirror. The techniques include maintaining each VLB of the virtualization layer in a RAID stripe across multiple physical drives in the data storage system.

Abstract: A method of performing deduplication is provided. The method includes (a) selecting, by applying a deterministic selection criterion, a sub-block of a block of data that contains multiple sub-blocks; (b) performing a lookup, into a deduplication table, of a digest generated by hashing the selected sub-block, the lookup matching an entry indexed by the digest in the deduplication table, the entry identifying a previously processed block; and (c) effecting storage of the block, including pointing to the previously processed block. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A mapper tree for a logical volume is provided by storing, in each leaf node of the mapper tree, pointers to pages of non-volatile storage that store host data written to corresponding pages within a segment of the logical address space of the logical volume that corresponds to the leaf node. In response to receiving an initial write operation directed to a segment of the logical address space of the logical volume for which no leaf node currently exists in the mapper tree, a representation of a new leaf node is added to a super leaf node in the mapper tree that efficiently stores representations of multiple leaf nodes.