Abstract: A method, computer program product, and computing system for storing data in one or more buckets of a plurality of buckets in a first level. The data of the one or more buckets may be de-staged to a plurality of buckets in a second level. The one or more buckets may be invalidated in the first level. A range of valid buckets in the first level may be defined based upon, at least in part, invalidating the one or more buckets in the first level.

Abstract: A method, computer program product, and computing system for identifying one or more metadata pages stored in a storage array, thus defining a primary set of metadata pages. An alternative set of metadata pages may be generated from the primary set of metadata pages. A log of changes associated with the primary set of metadata pages may be generated. A copy of at least a portion of the primary set of metadata pages may be generated based upon, at least in part, the alternative set of metadata pages and the log of changes associated with the primary set of metadata pages. The alternative set of metadata pages may lag behind in time from the primary set of metadata pages by a predefined amount of time.

Abstract: A method of data reduction in a block-based data storage system includes selecting a starting position in a block based on a deterministic function of block data content. Then for an unaligned block beginning at the selected starting position, a block digest (e.g., block hash) is generated and compared with stored block digests of stored data blocks. If there is a match, and the stored block matches the unaligned block, then a reference to the stored block is stored in place of the unaligned block, and otherwise the unaligned block and a corresponding digest are stored. The storing of references to already stored blocks, without the constraint of observing aligned-block boundaries, realizes increased savings of physical storage space.

Abstract: Techniques for caching may include: determining an update to a first data page of a first cache on a first node, wherein a second node includes a second cache and wherein the second cache includes a copy of the first data page; determining, in accordance with one or more criteria, whether to send the update from the first node to the second node; responsive to determining, in accordance with the one or more criteria, to send the update, sending the update from the first node to the second node; and responsive to determining not to send the update, sending an invalidate request from the first node to the second node, wherein the invalidate request instructs the second node to invalidate the copy of the first data page stored in the second cache of the second node.

Abstract: A technique for managing data storage begins at a predetermined offset relative to a chunk of data received for writing, and identifies a span of contiguous regions of the chunk that contain identical data. The technique replaces the span of contiguous regions of the chunk with a single instance of a region of the contiguous regions. The technique persistently stores a shortened version of the chunk with the single instance replacing the span of contiguous regions.

Abstract: A method, computer program product, and computer system for storing data in a bucket of a plurality of buckets. A spare bucket may be reserved in the plurality of buckets. A copy of the data may be stored in the spare bucket. A pointer to the data in the bucket and a pointer to the copy of the data in the spare bucket may be updated based upon, at least in part, storing the data in the bucket and storing the copy of the data in the spare bucket.

Abstract: A method, computer program product, and computer system for identifying, by a computing device, content in a first bucket in a first cache. It may be determined that a first portion of the content in the first bucket is a duplicate, wherein a second portion of the content in the first bucket may be unique. The first portion of the content in the first bucket may be deduplicated from the first cache. The second portion of the content may be stored in a second bucket in a second cache.

Abstract: A method of performing deduplication is provided. The method includes (a) selecting, by applying a deterministic selection criterion, a sub-block of a block of data that contains multiple sub-blocks; (b) performing a lookup, into a deduplication table, of a digest generated by hashing the selected sub-block, the lookup matching an entry indexed by the digest in the deduplication table, the entry identifying a previously processed block; and (c) effecting storage of the block, including pointing to the previously processed block. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A technique for managing data storage obtains a batch of chunks of data. The technique generates, using multiple pipelined instructions operating on the batch, a measure of data entropy for each of the chunks in the batch. The technique selectively compresses chunks in the batch based at least in part on the measures of data entropy generated for the respective chunks.

Abstract: Techniques for performing cache management includes partitioning entries of a hash table into buckets, wherein each of the buckets includes a portion of the entries of the hash table, configuring a cache, wherein the configuring includes allocating a section of the cache for exclusive use by each bucket, and performing first processing that stores a data block in the cache. The first processing includes determining a hash value for a data block, selecting, in accordance with the hash value, a first bucket of the plurality of buckets, wherein a first section of the cache is used exclusively for storing cached data blocks of the first bucket, storing metadata used in connection with caching the data block in a first entry of the first bucket, and storing the data block in a first cache location of the first section of the cache.

Abstract: A method, computer program product, and computing system for receiving data for storage in a storage system. The data may be written to a head entry of a log buffer, wherein the log buffer includes a plurality of data entries for flushing to the storage system. At least a portion of the plurality of data entries of the log buffer may be flushed, via a plurality of threads, to the storage system based upon, at least in part, a tail entry of the log buffer. A queue of committed data entries may be updated, via each thread of the plurality of threads, with one or more data entries of the log buffer flushed to the storage system by each thread. A new tail entry of the log buffer may be determined, via a thread of the plurality of threads, based upon, at least in part, the queue of committed data entries.

Abstract: At least one processing device is configured to select a virtual block corresponding to a physical block in a storage system, wherein the virtual block comprises a plurality of pages with one or more pages of the virtual block respectively pointing to one or more data portions in the physical block. The at least one processing device is also configured to identify a target physical block and one or more virtual blocks corresponding to the target physical block. The at least one processing device is further configured to move at least one data portion from the physical block to the target physical block. Still further, the at least one processing device is configured to redirect the at least one page of the virtual block corresponding to the at least one moved data portion to one of the one or more virtual blocks corresponding to the target physical block.

Abstract: A method comprising: storing, in a memory, a mapping tree that is implemented by using an array of mapping pages, the mapping tree having a depth of D, wherein D is an integer greater than or equal to 0; receiving a write request that is associated with a first type-1 address; storing, in a storage device, data associated with the write request, the data associated with the write request being stored in the storage device based on a first type-2 address; generating a map entry that maps the first type-1 address to the first type-2 address; calculating a first hash digest of the first type-1 address; and storing the map entry in a first mapping page.

Abstract: A method, computer program product, and computer system for identifying, by a computing device, a plurality of blocks. A maximum number of blocks of the plurality of blocks capable of being copied to a new block may be identified. Data from the maximum number of blocks of the plurality of blocks may be copied to the new block.

Abstract: Techniques for providing an efficient virtualization layer structure in a data storage system. The techniques include implementing multiple layers of indirection for accessing host data in the data storage system, including a mapping layer, a virtualization layer, and a physical layer. The virtualization layer includes virtual layer blocks (VLBs), each VLB including virtual pointers. Each virtual pointer is pointed to by one or more leaf pointers in the mapping layer, and points to a data block in the physical layer. The techniques include generating, for each virtual pointer in the VLB, reference count metadata to keep track of the number of leaf pointers pointing to the virtual pointer, and maintaining, in a metadata page, the reference count metadata for the virtual pointers in a three (3)-way mirror. The techniques include maintaining each VLB of the virtualization layer in a RAID stripe across multiple physical drives in the data storage system.

Abstract: A technique for performing deduplication traverses a deduplication database and assigns digest values in the database to buckets, where each bucket covers a respective range of digest values. To deduplicate a particular candidate block, the technique generates a digest from the candidate block and searches for the computed digest in a subset of the buckets, where the subset is selected based on the computed digest. If a target block providing an exact match or a suitably close partial match is found in the subset of buckets, the technique effects storage of the candidate block at least in part by providing a reference to the target block.

Abstract: An aspect of implementing globally optimized partial deduplication of storage objects includes gathering pages that share a common feature, dividing the pages into groups based on commonality with corresponding representative pages, where each is assigned as a representative dedupe page for the corresponding groups. For each group in the groups of pages, an aspect also includes writing the pages to a corresponding physical container.

Abstract: A technique for validating metadata includes creating log entries for virtualization structures pointed to by mapping pointers in a mapping tree and processing the log entries in multiple passes. A current pass validates a current level of redirection and creates new log entries to be processed during a next pass. The new log entries represent a next level of redirection, and as many next passes are processed in sequence as there are next levels of redirection.

Abstract: An aspect of managing data block compression in a storage system includes performing, for each block written to the storage system: bit-wise traversing the block, searching the block for a pattern indicating a repeating sequence of bits and, upon determining the pattern exists in the block and the repeating sequence of bits in the pattern exceeds a threshold value, removing the repeating sequence of bits from the block thereby yielding a reduced-size block.

Abstract: A method, computer program product, and computing system for associating each data container of a first set of data containers in volatile memory with a metadata page. One or more metadata changes associated with a metadata page may be written to the data container associated with the metadata page. The one or more metadata changes stored in the first set of data containers may be written to a storage array.