Abstract: In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein a block of the plurality of blocks comprises fingerprints computed based on content of respective data values. The system merges, in a merge operation, updates for the deduplication fingerprint index to the hash data structure stored in a persistent storage. As part of the merge operation, the system mirrors the updates to a cached copy of the hash data structure in a cache memory, and updates, in an indirect block, information regarding locations of blocks in the cached copy of the hash data structure.

Abstract: In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein a block of the plurality of blocks comprises fingerprints computed based on content of respective data values. The system merges, in a merge operation, updates for the deduplication fingerprint index to the hash data structure stored in a persistent storage. As part of the merge operation, the system mirrors the updates to a cached copy of the hash data structure in a cache memory, and updates, in an indirect block, information regarding locations of blocks in the cached copy of the hash data structure.

Abstract: In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein a block of the plurality of blocks comprises fingerprints computed based on content of respective data values. The system merges, in a merge operation, updates for the deduplication fingerprint index to the hash data structure stored in a persistent storage. As part of the merge operation, the system mirrors the updates to a cached copy of the hash data structure in a cache memory, and updates, in an indirect block, information regarding locations of blocks in the cached copy of the hash data structure.

Abstract: In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein the hash data structure is stored in persistent storage, and a block of the plurality of blocks comprises fingerprints computed based on content of respective data units. The system uses an indirect block in a memory to access a given block of the plurality of blocks in the hash data structure, the indirect block containing references to blocks of the hash data structure containing the deduplication fingerprint index, and the references indicating storage locations of the plurality of blocks in the persistent storage.

Abstract: In some examples, a system performs data deduplication using a fingerprint index comprising a plurality of buckets, each bucket of the plurality of buckets comprising entries associating fingerprints for data units to storage location indicators of the data units, wherein a storage location indicator of the storage location indicators provides an indication of a storage location of a data unit in persistent storage. For adding a new fingerprint to the fingerprint index, the system detects that a corresponding bucket of the plurality of buckets is full, in response to the detecting, adds space to the corresponding bucket by taking a respective amount of space from a further bucket of the plurality of buckets, and inserts the new fingerprint into the corresponding bucket after increasing the size of the corresponding bucket.

Abstract: In some examples, a system receives, in a synchronization buffer stored in a memory, fingerprint index entries for respective data units received by the system, each fingerprint index entry of the fingerprint index entries comprising a fingerprint computed for a corresponding data unit, and a storage location indicator for the corresponding data unit. The system merges the fingerprint index entries in the synchronization buffer with fingerprint index entries of a persistent fingerprint index in a hash data structure stored in persistent storage, the hash data structure comprising a plurality of buckets.

Abstract: In some examples, a system associates a plurality of buffers in a memory with respective multiple bins of a fingerprint index in persistent storage. The system computes fingerprints for incoming data units, and selects, based on an adaptive sampling indication, a subset of the fingerprints. The system adds fingerprint index entries corresponding to the selected subset of the fingerprints to a respective subset of the multiple bins, wherein adding a fingerprint index entry to a bin of the respective subset of the multiple bins comprises adding the fingerprint index entry to the buffer of the bin.

Abstract: In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein a block of the plurality of blocks comprises fingerprints computed based on content of respective data values. The system merges, in a merge operation, updates for the deduplication fingerprint index to the hash data structure stored in a persistent storage. As part of the merge operation, the system mirrors the updates to a cached copy of the hash data structure in a cache memory, and updates, in an indirect block, information regarding locations of blocks in the cached copy of the hash data structure.

Abstract: A system may include a persistent storage device, a low latency cache device, a volatile memory; and a processor. The processor is to store a data structure in the volatile memory that is usable to directly translate a block logical address for targeted data to a candidate physical location on the cache device, store a multilevel translation index in the volatile memory for translating the block logical address for the targeted data to an expected physical location of the targeted data on the cache device and attempt accessing the targeted data at the candidate physical location retrieved from the direct cache address translation data structure. In response to the targeted data not being at the candidate physical address, access the targeted data at the expected physical location retrieved from the multilevel translation index.

Abstract: A system may include a persistent storage device, a low latency cache device, a volatile memory; and a processor. The processor is to store a data structure in the volatile memory that is usable to directly translate a block logical address for targeted data to a candidate physical location on the cache device, store a multilevel translation index in the volatile memory for translating the block logical address for the targeted data to an expected physical location of the targeted data on the cache device and attempt accessing the targeted data at the candidate physical location retrieved from the direct cache address translation data structure. In response to the targeted data not being at the candidate physical address, access the targeted data at the expected physical location retrieved from the multilevel translation index.

Abstract: In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein the hash data structure is stored in persistent storage, and a block of the plurality of blocks comprises fingerprints computed based on content of respective data units. The system uses an indirect block in a memory to access a given block of the plurality of blocks in the hash data structure, the indirect block containing references to blocks of the hash data structure containing the deduplication fingerprint index, and the references indicating storage locations of the plurality of blocks in the persistent storage.

Abstract: In some examples, a system performs data deduplication using a fingerprint index comprising a plurality of buckets, each bucket of the plurality of buckets comprising entries associating fingerprints for data units to storage location indicators of the data units, wherein a storage location indicator of the storage location indicators provides an indication of a storage location of a data unit in persistent storage. For adding a new fingerprint to the fingerprint index, the system detects that a corresponding bucket of the plurality of buckets is full, in response to the detecting, adds space to the corresponding bucket by taking a respective amount of space from a further bucket of the plurality of buckets, and inserts the new fingerprint into the corresponding bucket after increasing the size of the corresponding bucket.

Abstract: In some examples, a system associates a plurality of buffers in a memory with respective multiple bins of a fingerprint index in persistent storage. The system computes fingerprints for incoming data units, and selects, based on an adaptive sampling indication, a subset of the fingerprints. The system adds fingerprint index entries corresponding to the selected subset of the fingerprints to a respective subset of the multiple bins, wherein adding a fingerprint index entry to a bin of the respective subset of the multiple bins comprises adding the fingerprint index entry to the buffer of the bin.

Abstract: In some examples, a system receives, in a synchronization buffer stored in a memory, fingerprint index entries for respective data units received by the system, each fingerprint index entry of the fingerprint index entries comprising a fingerprint computed for a corresponding data unit, and a storage location indicator for the corresponding data unit. The system merges the fingerprint index entries in the synchronization buffer with fingerprint index entries of a persistent fingerprint index in a hash data structure stored in persistent storage, the hash data structure comprising a plurality of buckets.

Abstract: In some examples, a system dynamically adjusts a proportion of incoming data blocks for which respective fingerprints are added to a fingerprint index, where the fingerprint index correlates fingerprints for data blocks to storage location indicators for the data blocks. The system inputs, into a buffer, fingerprint index delta updates for the incoming data blocks according to the dynamically adjusted proportion, and updates the fingerprint index based on the fingerprint index delta updates input into the buffer. The system retrieves, in response to a fingerprint of an incoming data block, a collection of block entries from a block location index that stores information for data blocks in temporal order, and deduplicates the incoming data blocks using the collection of block entries from the block location index.

Abstract: A system may include a persistent storage device, a low latency cache device, a volatile memory; and a processor. The processor is to store a data structure in the volatile memory that is usable to directly translate a block logical address for targeted data to a candidate physical location on the cache device, store a multilevel translation index in the volatile memory for translating the block logical address for the targeted data to an expected physical location of the targeted data on the cache device and attempt accessing the targeted data at the candidate physical location retrieved from the direct cache address translation data structure. In response to the targeted data not being at the candidate physical address, access the targeted data at the expected physical location retrieved from the multilevel translation index.

Abstract: In some examples, a system dynamically adjusts a proportion of incoming data blocks for which respective fingerprints are added to a fingerprint index, where the fingerprint index correlates fingerprints for data blocks to storage location indicators for the data blocks. The system inputs, into a buffer, fingerprint index delta updates for the incoming data blocks according to the dynamically adjusted proportion, and updates the fingerprint index based on the fingerprint index delta updates input into the buffer. The system retrieves, in response to a fingerprint of an incoming data block, a collection of block entries from a block location index that stores information for data blocks in temporal order, and deduplicates the incoming data blocks using the collection of block entries from the block location index.

Abstract: This disclosure is directed to a technique for memory management where physical memory areas may be partitions into a hierarchy of portions, the hierarchy may include a domain level that includes a page level that includes a slice level that includes an object level. Objects within a slice are a consistent size but may be different sized for different slices. A set of states reflecting memory usage status for each of the slices includes: a clean state for unused; a partial state; a full state; and a dirty state. Responses to allocation requests may be performed by selecting objects that are in a most preferred state based on a state allocation cost and a memory allocation cost either alone or in combination. A compact memory layout may be used to reduce run-time fragmentation of memory.

Abstract: This disclosure is directed to a technique for memory management where physical memory areas may be partitions into a hierarchy of portions, the hierarchy may include a domain level that includes a page level that includes a slice level that includes an object level. Objects within a slice are a consistent size but may be different sized for different slices. A set of states reflecting memory usage status for each of the slices includes: a clean state for unused; a partial state; a full state; and a dirty state. Responses to allocation requests may be performed by selecting objects that are in a most preferred state based on a state allocation cost and a memory allocation cost either alone or in combination. A compact memory layout may be used to reduce run-time fragmentation of memory.

Abstract: Processing and memory resources are optimized in a data storage system by reading a region of compressed data containing desired data is read from primary storage, writing the compressed data to a memory page, selectively decompressing the compressed data to retrieve the desired data, and the writing the decompressed data back to the same page. State information about the start of the compressed data and the end of the decompressed data on the page is maintained to enable decompression to be halted and resumed on demand.