Abstract: A method for reverse range lookup in an ordered data structure of keys, wherein each key comprises a logical block address (LBA) and a snapshot identifier (ID) of one of one or more snapshots in a snapshot hierarchy, is provided. The keys in the ordered data structure are in an order from smallest to largest LBA, wherein in the order, keys having a same LBA are ordered from smallest to largest snapshot ID. The method includes determining a range of LBAs and traversing the ordered data structure in reverse order until a key is found that: has an LBA and a snapshot ID that is less than or equal to the last LBA and the largest snapshot ID, respectively. The method further includes adding an extent corresponding to the located key to an extent result list if the snapshot ID of the located key is not within a deny list.

Abstract: A method for resumeable snapshot deletion is provided. A method for deletion of nodes maintained in an ordered data structure for a first snapshot includes processing the nodes maintained in the ordered data structure according to a defined order, setting a node path cursor with a pointer to a node and an indication of the deletion of the node; storing the node path cursor in a persistent storage; and during processing of the nodes: detecting a failure; after the failure, checking the pointer of the node path cursor; and resuming processing of the nodes starting from the first node indicated by the pointer.

Abstract: The present disclosure relates to bootstrapping an encrypted single node VSAN cluster. One method includes receiving a request to create an encrypted VSAN cluster from a single host in a software-defined datacenter, deploying a virtual server on a VSAN datastore of the software-defined datacenter, registering a native key provider (NKP) in the virtual server, creating an empty VSAN cluster encrypted by the NKP, adding the single host to the encrypted empty cluster to create a one-host encrypted cluster, registering a KMIP KMS in the virtual server, switching encryption of the one-host encrypted cluster from the NKP to the KMIP KMS, and adding another host to the one-host encrypted cluster to create the encrypted cluster.

Abstract: Techniques for efficiently exporting snapshot changes are provided. In some embodiments, a system may receive a first snapshot of a data set in a storage system and a second snapshot the data set in the storage system. The system may further generate actions based on differences between the first snapshot and the second snapshot to produce a list of actions, wherein a modification to a file or directory path having a first directory location includes a first action to rename a file from the first directory location to a temporary storage location and a second action to rename the file from the temporary storage location to a second directory location; and provide the generated actions to a backup system. The backup system may apply the generated actions to a first backup associated with the first snapshot to produce a second backup associated with the second snapshot.

Abstract: System and method for managing distributed storage objects for host unavailability in a distributed storage system uses at least one of a crash indicator in a specific on-disk block and a paused object indicator for a distributed storage object to determine whether to perform data recovery for the distributed storage object. When the crash indicator is set or the paused object indicator implies that the distributed storage object is a paused object, the distributed storage object is left as a paused object without perform the data recovery for the distributed storage object. When the crash indicator is unset and the paused object indicator implies that the distributed storage object is not a paused object, the data recovery for the distributed storage object is performed.

Abstract: Techniques for efficiently managing a file clone from a filesystem which supports efficient volume snapshots are provided. In some embodiments, a system may receive an instruction to remove the file clone from the filesystem. The file clone may be a point-in-time copy of metadata of an original file. The system may further—for a file map entry in a filesystem tree associated with the file clone, the file map entry indicating a data block—decrement a reference count in a reference count entry associated with the file map entry. The reference count entry may be stored in the filesystem tree according to a key and the key may comprise an identification of the original file. The system may further reclaim the data block in a storage system when the reference count is zero.

Abstract: Exemplary methods, apparatuses, and systems include a file system process obtaining locks on a first node and a second node in a tree structure, with the second node being a child node of the first node. The file system process determines a quantity of child nodes of the second. While holding the locks on the first and second nodes, the file system determines whether to proactively split or merge the second node. In response to determining that the quantity of child nodes is within a first range, the file system process splits the second node. If the file system process determines that the quantity of child nodes is within a second range, the file system process merges the second node.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for cloneless snapshot reversion. A request can be received to revert to a past snapshot of a virtual computing instance in a snapshot chain of a snapshot tree provided by a software defined data center. A live snapshot can be created at an end of the snapshot chain comprising the past snapshot. An intervening snapshot in the snapshot chain can be indicated as abandoned in a snapshot map associated with the snapshot tree based on the reversion.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for managing extent sharing between snapshots using mapping addresses. A first mapping address can be assigned to a first extent responsive to a request to write the first extent. A second mapping address can be assigned to a second extent responsive to a request to write the second extent. A snapshot can be created. A snapshot mapping address, that is monotonically increased from the second mapping address, can be assigned to the snapshot. A third mapping address, that is monotonically increased from the second mapping address, can be assigned to a third extent of the snapshot responsive to a request to write the third extent.

Abstract: The disclosure herein describes storing data using a capacity data storage tier and a smaller performance data storage tier. The capacity data storage tier includes capacity data storage hardware configured to store log-structured leaf pages (LLPs), and the performance data storage tier includes performance data storage hardware. A virtual address table (VAT) includes a set of virtual address entries referencing the LLPs. A tree-structured index includes index nodes referencing the set of virtual address entries of the VAT. Data to be stored is received, and at least a first portion of metadata associated with the received data is stored in the LLPs using the VAT, and at least a second portion of metadata associated with the received data is stored in the performance data storage tier. The architecture reduces space usage of the performance data storage tier.

Abstract: A method for deleting one or more snapshots using micro-batch processing is provided. The method includes receiving a request to delete the one or more snapshots, identifying one or more middle map extents exclusively owned by the one or more snapshots requested to be deleted, wherein metadata for the one or more snapshots is stored in one or more logical maps having logical map extents mapping logical block addresses (LBAs) to middle block addresses (MBAs) and a middle map having middle map extents mapping MBAs to physical block addresses (PBAs) of physical locations where data blocks are written, adding MBAs of the identified one or more middle map extents in a batch, determining a first micro-batch including a first subset of the MBAs in the batch, the first subset of MBAs being MBAs less than a first upper bound MBA, and using a first transaction to delete the middle map extents corresponding to the first subset of MBAs included in the first micro-batch.

Abstract: Example methods and systems for creating a plurality of snapshots of a storage object backed by a plurality of copy-on-write (COW) B+ tree data structure including a first COW B+ tree data structure having a first root node and leaf nodes maintaining mappings of LBAs to PBAs associated with a first snapshot of the storage object are disclosed. One example method includes creating a first root node of a first B+ tree data structure, maintaining a delta mapping table between a set of LBAs to a set of PBAs in the first leaf node, in response to receiving a request to create a second snapshot of the storage object: creating a second root node of a second COW B+ tree data structure and creating leaf nodes of the second COW B+ tree data structure in batches based on an order of the set of LBAs.

Abstract: A method for metadata updating is provided. The method generally includes identifying a first segment containing a first physical block corresponding to a first PBA where content of a first data block was previously stored, determining a first key associated with the first data block, wherein the first key comprises a block address in a first key-value pair that maps the block address to the first PBA, traversing a B+ tree to locate a node storing a second key-value pair that maps the first key to a second PBA, determining the second PBA and the first PBA match, and based on the determination: updating, in the second key-value pair, the second PBA to a third PBA corresponding to a second physical block where the content of the first data block is currently stored or removing, in the second key-value pair, the second PBA.

Abstract: Example methods and systems to perform a migration of a virtualized computing instance and its first snapshot hierarchy from a first object store to a second object store have been disclosed. One example method includes identifying a first disk chain of the first snapshot hierarchy having an object running point, identifying a second disk chain of the first snapshot hierarchy different from the first disk chain, and migrating the second disk chain from the first object store to the second object store to form a first branch of a second snapshot hierarchy in the second object store. After the migrating, the example method includes instructing to take a first native snapshot on the object running point in the second object store, instructing to revert the object running point along the first branch and migrating the first disk chain from the first object store to the second object store.

Abstract: The disclosure provides techniques for deduplicating files. The techniques include, upon creating or modifying a file, placing a logical timestamp of the current logical time, within a queue associated with the directory of the file. The techniques further include placing the logical timestamp within a queue of each parent directory of the directory of the file. To determine a set of files for deduplication, the techniques disclosed herein identify files that have been modified within a logical time range. The set of files modified within a logical time is identified by traversing directories of a storage system, the directories being organized within a tree structure. If a directory's queue does not contain a timestamp that is within the logical time range, then all child directories can be skipped over for further processing, such that no files within the child directories end up being within the set of files for deduplication.

Abstract: Optimizing file access includes a process for identifying a file access event for a first accessed file, and incrementing a first access counter in an access list in a memory, which also includes access counters for other accessed files. The process further includes exporting the first access counter to a performance monitoring dashboard, or exporting to a storage allocator and, based on the value, moving the first accessed file between a first storage and a second storage. The process also includes determining whether the value of the first access counter meets a first threshold, or a sum of values of the access counters for the other accessed files meets a second threshold. Based on meeting the first threshold or meeting the second threshold, the process includes persisting the access counters on a storage media. The access counters also provide security monitoring (e.g., identifying excessive file access).

Abstract: A method for modifying key-value pairs of a B+ tree is provided. The method receives a request to modify a particular key-value pair. Each node of the tree has a modification number. The method traverses a path on the tree from the root node toward the particular node. The traversing includes upon reaching a parent node of the path, acquiring a shared lock on both the parent node and a child node one level below the parent node. Upon determining that the child node is the particular node, the method stores the modification number of the particular node, releases the shared lock on the particular node, compares a current modification number of the node with its stored number, and acquires an exclusive lock on the node if the numbers are the same. The method increments the current modification number of the node and modifies it while in the exclusive lock.

Abstract: System and method for deleting parent snapshots of running points of storage objects stored in a storage system, in response to a request to delete a parent snapshot of a running point of a storage object stored in the storage system, traverses a subtree of a B tree that corresponds to a logical map of the parent snapshot to find nodes of the subtree that are exclusively owned by the parent snapshot, which are added to an exclusive node list of the parent snapshot. The minimum node ownership value of the running point is then changed to the minimum node ownership value of the parent snapshot so that any node of the subtree of the B tree with a node ownership value equal to or greater than the changed minimum node ownership value is deemed to be owned by the running point. The nodes of the subtree of the B tree that are found in the exclusive node list of the parent snapshot are then deleted.

Abstract: A method for the reverse deletion of a plurality of snapshots in a chain of snapshots is provided. The method includes in reverse order, starting from a latest snapshot in time to an earliest snapshot in time of the plurality of snapshots: identifying at least one of a first set of one or more data blocks of a snapshot that are shared with an earlier snapshot in time in the chain of snapshots or a second set of one or more data blocks of the snapshot that are owned by the snapshot and processing the second set of one or more data blocks and skipping processing the first set of one or more data blocks, wherein processing the second set of one or more data blocks comprises performing one or more actions to maintain and/or delete data blocks of the second set of one or more data blocks.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for log-structured file system management operations. An aggregate amount of over-provisioned computing resources assigned to a plurality of log-structured file system (LFS) objects that are allocated for performance of memory management operations associated with a virtual storage area network (vSAN) can be determined. A subset of LFS objects that are candidates for performance of a particular memory management operation based on an amount of over-provisioned computing resources consumed by one or more LFS objects among the plurality of LFS objects exceeding a resource consumption threshold associated with the determined amount of over-provisioned computing resources assigned to the plurality of LFS objects can be selected. The particular memory management operation using one or more of the candidate LFS objects can be performed.