Abstract: In some examples, a system creates a partition map that maps partitions of a data bucket to respective virtual processors executed in a cluster of computer nodes. Responsive to a request to access a data object in the data bucket, the system identifies which partition contains metadata for the data object based on a key associated with the data object, and identifies, based on the identified partition and using the partition map, a virtual processor that has the metadata for the data object. Responsive to a migration of a first virtual processor from a first to a second computer node, the system updates a virtual processor-computer node map that maps the respective virtual processors to corresponding computer nodes of the cluster of computer nodes, where the partition map remains unchanged in response to the migration of the first virtual processor from the first computer node to the second computer node.

Abstract: In some examples, a first computing node receives a write request, where the first computing node is part of a collection of multiple computing nodes, and a plurality of virtual processors are executable in the multiple computing nodes to manage access of data in a shared storage system. In response to the write request, a first virtual processor at the first computing node sends, to a second virtual processor, a request for metadata stored by the second virtual processor. The first virtual processor updates an intent structure in a nonvolatile memory with information indicating an intent to write data for the write request. In response to the metadata received at the first virtual processor from the second virtual processor, a write of the data is initiated to cause storage of the data in the shared storage system.

Abstract: In some examples, a system maintains mapping metadata for data in storage, the data divided into chunks, the mapping metadata comprising chunk entries mapping chunk identifiers of the chunks to storage locations, wherein the chunk identifiers are monotonically increasing as chunks are created. The system updates an epoch in response to triggering a creation of a snapshot of the data. In response to a deletion of a first chunk, the system updates a first chunk entry for the first chunk to include a creation epoch identifier and a deletion epoch identifier for the first chunk. In response to a deletion of a first snapshot of the multiple snapshots, the system uses the creation epoch identifier and the deletion epoch identifier to determine whether to remove the first chunk entry from the mapping metadata.

Abstract: Examples described herein relate to selection and variation of extent size thresholds in a storage volume. Examples may select an extent size threshold for a volume according to a type of application that is to store data on the volume. Examples may dynamically vary the extent size threshold based on data reduction metrics, such as deduplication ratio and/or compression ratio. Examples may dynamically vary the extent size threshold for the volume based on cache performance metrics, such as cache miss rate. Examples may dynamically vary the extent size threshold for the volume based on an amount of dead storage space corresponding to partially overwritten extents in the volume.

Abstract: Examples described herein relate to selection and variation of extent size thresholds in a storage volume. Examples may select an extent size threshold for a volume according to a type of application that is to store data on the volume. Examples may dynamically vary the extent size threshold based on data reduction metrics, such as deduplication ratio and/or compression ratio. Examples may dynamically vary the extent size threshold for the volume based on cache performance metrics, such as cache miss rate. Examples may dynamically vary the extent size threshold for the volume based on an amount of dead storage space corresponding to partially overwritten extents in the volume.

Abstract: In some examples, a system maintains mapping metadata for data in storage, the data divided into chunks, the mapping metadata comprising chunk entries mapping chunk identifiers of the chunks to storage locations, wherein the chunk identifiers are monotonically increasing as chunks are created. The system updates an epoch in response to triggering a creation of a snapshot of the data. In response to a deletion of a first chunk, the system updates a first chunk entry for the first chunk to include a creation epoch identifier and a deletion epoch identifier for the first chunk. In response to a deletion of a first snapshot of the multiple snapshots, the system uses the creation epoch identifier and the deletion epoch identifier to determine whether to remove the first chunk entry from the mapping metadata.

Abstract: In some examples, a system samples a subset of input/output (I/O) accesses of a storage, the I/O accesses being part of a workload. The system determines, based on the sampled subset of the I/O accesses, a first reuse distance distribution for a first time interval, determines a similarity measure representing a similarity of the first reuse distance distribution and a second reuse distance distribution for a second time interval different from the first time interval, and based on a change in the similarity measure, triggers a workload placement process to determine a placement of the workload on a compute node of a plurality of compute nodes.

Abstract: A technique for reducing recovery time in a computing apparatus includes: storing data in a key-value store residing in a persistent memory, the key-value store including a plurality of persistent nodes including key-value pairs, the key-value pairs of the persistent nodes being unsorted; caching selected persistent nodes in a volatile memory, the key-value pairs of the cached nodes being sorted, the caching including instantiating a dual pointer for each cached node, the dual pointer including a volatile pointer to a child node of the cached node and a persistent pointer to the child node of the corresponding persistent node in the key-value store; and rebuilding the cache of selected nodes upon demand using the dual pointers in the event of recovery.

Abstract: Exemplary methods, apparatuses, and systems maintain a plurality of summary data structures corresponding to a plurality of logical file system namespaces representing a plurality of hierarchies of one or more directories having one or more files, each file being stored in the storage system as a plurality of segments in a deduplicated manner. In response to a request to estimate a storage usage by a first of the file system namespace, identify a first of the summary data structures corresponding to the first file system namespace, wherein the first summary data structure stores information summarizing deduplicated segments referenced by one or more files of the first file system namespace. Estimate the storage usage of the first file system namespace based on the first summary data structure and a global summary data structure, wherein the global summary data structure stores information summarizing deduplicated segments referenced by all of the file system namespaces.

Abstract: A technique for reducing recovery time in a computing apparatus includes: storing data in a key-value store residing in a persistent memory, the key-value store including a plurality of persistent nodes including key-value pairs, the key-value pairs of the persistent nodes being unsorted; caching selected persistent nodes in a volatile memory, the key-value pairs of the cached nodes being sorted, the caching including instantiating a dual pointer for each cached node, the dual pointer including a volatile pointer to a child node of the cached node and a persistent pointer to the child node of the corresponding persistent node in the key-value store; and rebuilding the cache of selected nodes upon demand using the dual pointers in the event of recovery.

Abstract: When data blocks of a data seglet are compressed using a shared dictionary and when the requested data block (or blocks) do not include the last data block of the data seglet, an optimization in the read path may involve decompressing a certain portion of the data seglet from a starting position of the data seglet to a decompression endpoint of the data seglet, but not including the portion of the data seglet following the decompression endpoint. Such technique may involve the storing of a mapping that maps, for each data block within the data seglet, an identifier of the data block to a decompression endpoint that indicates a portion of the data seglet that includes the data block.

Abstract: When data blocks of a data seglet are compressed using a shared dictionary and when the requested data block (or blocks) do not include the last data block of the data seglet, an optimization in the read path may involve decompressing a certain portion of the data seglet from a starting position of the data seglet to a decompression endpoint of the data seglet, but not including the portion of the data seglet following the decompression endpoint. Such technique may involve the storing of a mapping that maps, for each data block within the data seglet, an identifier of the data block to a decompression endpoint that indicates a portion of the data seglet that includes the data block.

Abstract: Methods, systems, and computer programs are presented for deduplicating data. One method includes an operation for receiving a data block having a logical address. The storage system includes a logical address mapping index for mapping logical addresses to block numbers, a block index for mapping block numbers to physical locations, and a fingerprint index for mapping fingerprints of data blocks to block numbers. Additionally, the method includes an operation for inline processing the data block. Inline processing the data block includes determining a fingerprint of the data block, examining the fingerprint index to determine if the fingerprint is already mapped to an existing data block in the storage system, if the fingerprint is already mapped then adding a mapping of the logical address to the existing data block in the logical address mapping index, and if the fingerprint is not already mapped then creating the corresponding entries in the indices.

Abstract: Methods, systems, and computer programs are presented for deduplicating data. One method includes an operation for receiving a data block having a logical address. The storage system includes a logical address mapping index for mapping logical addresses to block numbers, a block index for mapping block numbers to physical locations, and a fingerprint index for mapping fingerprints of data blocks to block numbers. Additionally, the method includes an operation for inline processing the data block. Inline processing the data block includes determining a fingerprint of the data block, examining the fingerprint index to determine if the fingerprint is already mapped to an existing data block in the storage system, if the fingerprint is already mapped then adding a mapping of the logical address to the existing data block in the logical address mapping index, and if the fingerprint is not already mapped then creating the corresponding entries in the indices.

Abstract: A garbage collector of a storage system traverses a namespace of a file system of the storage system to identify segments that are alive in a breadth-first manner. The namespace includes information identifying files that are represented by segments arranged in a plurality of levels in a hierarchy, where an upper level segment includes one or more references to one or more lower level segments, and at least one segment is referenced by multiple files. All live segments of an upper level are identified before any of live segments of a lower level are identified. Upon all live segments of all levels have been identified, the live segments are copied from their original storage locations to a new storage location, and a storage space associated with the original storage locations is reclaimed.

Abstract: Scaling Asynchronous reclamation of free space in de-duplicate multi-controller storage systems is disclosed. In one example, a set of de-duplicated data segments/blocks organized in a hierarchical data structure is received, the set of de-duplicated data segments/blocks are allocated across a plurality of controllers of the file system, where each de-duplicated data segment is restricted to be co-allocated to the same controller as the reference(s) referencing the de-duplicated data segment/block in the hierarchical data structure. In various embodiments, the file system is traversed, each controller of the file system updates its own local live vector and reclaim storage space that are not marked as alive.

Abstract: Exemplary methods, apparatuses, and systems maintain a plurality of summary data structures corresponding to a plurality of logical file system namespaces representing a plurality of hierarchies of one or more directories having one or more files, each file being stored in the storage system as a plurality of segments in a deduplicated manner. In response to a request to estimate a storage usage by a first of the file system namespace, identify a first of the summary data structures corresponding to the first file system namespace, wherein the first summary data structure stores information summarizing deduplicated segments referenced by one or more files of the first file system namespace. Estimate the storage usage of the first file system namespace based on the first summary data structure and a global summary data structure, wherein the global summary data structure stores information summarizing deduplicated segments referenced by all of the file system namespaces.