Abstract: Techniques for efficiently managing a file clone from a filesystem which supports efficient volume snapshots are provided. In some embodiments, a system may receive an instruction to remove the file clone from the filesystem. The file clone may be a point-in-time copy of metadata of an original file. The system may further—for a file map entry in a filesystem tree associated with the file clone, the file map entry indicating a data block—decrement a reference count in a reference count entry associated with the file map entry. The reference count entry may be stored in the filesystem tree according to a key and the key may comprise an identification of the original file. The system may further reclaim the data block in a storage system when the reference count is zero.

Abstract: Exemplary methods, apparatuses, and systems include a file system process obtaining locks on a first node and a second node in a tree structure, with the second node being a child node of the first node. The file system process determines a quantity of child nodes of the second. While holding the locks on the first and second nodes, the file system determines whether to proactively split or merge the second node. In response to determining that the quantity of child nodes is within a first range, the file system process splits the second node. If the file system process determines that the quantity of child nodes is within a second range, the file system process merges the second node.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for cloneless snapshot reversion. A request can be received to revert to a past snapshot of a virtual computing instance in a snapshot chain of a snapshot tree provided by a software defined data center. A live snapshot can be created at an end of the snapshot chain comprising the past snapshot. An intervening snapshot in the snapshot chain can be indicated as abandoned in a snapshot map associated with the snapshot tree based on the reversion.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for managing extent sharing between snapshots using mapping addresses. A first mapping address can be assigned to a first extent responsive to a request to write the first extent. A second mapping address can be assigned to a second extent responsive to a request to write the second extent. A snapshot can be created. A snapshot mapping address, that is monotonically increased from the second mapping address, can be assigned to the snapshot. A third mapping address, that is monotonically increased from the second mapping address, can be assigned to a third extent of the snapshot responsive to a request to write the third extent.

Abstract: The disclosure herein describes storing data using a capacity data storage tier and a smaller performance data storage tier. The capacity data storage tier includes capacity data storage hardware configured to store log-structured leaf pages (LLPs), and the performance data storage tier includes performance data storage hardware. A virtual address table (VAT) includes a set of virtual address entries referencing the LLPs. A tree-structured index includes index nodes referencing the set of virtual address entries of the VAT. Data to be stored is received, and at least a first portion of metadata associated with the received data is stored in the LLPs using the VAT, and at least a second portion of metadata associated with the received data is stored in the performance data storage tier. The architecture reduces space usage of the performance data storage tier.

Abstract: A method for deleting one or more snapshots using micro-batch processing is provided. The method includes receiving a request to delete the one or more snapshots, identifying one or more middle map extents exclusively owned by the one or more snapshots requested to be deleted, wherein metadata for the one or more snapshots is stored in one or more logical maps having logical map extents mapping logical block addresses (LBAs) to middle block addresses (MBAs) and a middle map having middle map extents mapping MBAs to physical block addresses (PBAs) of physical locations where data blocks are written, adding MBAs of the identified one or more middle map extents in a batch, determining a first micro-batch including a first subset of the MBAs in the batch, the first subset of MBAs being MBAs less than a first upper bound MBA, and using a first transaction to delete the middle map extents corresponding to the first subset of MBAs included in the first micro-batch.

Abstract: Example methods and systems for creating a plurality of snapshots of a storage object backed by a plurality of copy-on-write (COW) B+ tree data structure including a first COW B+ tree data structure having a first root node and leaf nodes maintaining mappings of LBAs to PBAs associated with a first snapshot of the storage object are disclosed. One example method includes creating a first root node of a first B+ tree data structure, maintaining a delta mapping table between a set of LBAs to a set of PBAs in the first leaf node, in response to receiving a request to create a second snapshot of the storage object: creating a second root node of a second COW B+ tree data structure and creating leaf nodes of the second COW B+ tree data structure in batches based on an order of the set of LBAs.

Abstract: A method for metadata updating is provided. The method generally includes identifying a first segment containing a first physical block corresponding to a first PBA where content of a first data block was previously stored, determining a first key associated with the first data block, wherein the first key comprises a block address in a first key-value pair that maps the block address to the first PBA, traversing a B+ tree to locate a node storing a second key-value pair that maps the first key to a second PBA, determining the second PBA and the first PBA match, and based on the determination: updating, in the second key-value pair, the second PBA to a third PBA corresponding to a second physical block where the content of the first data block is currently stored or removing, in the second key-value pair, the second PBA.

Abstract: Example methods and systems to perform a migration of a virtualized computing instance and its first snapshot hierarchy from a first object store to a second object store have been disclosed. One example method includes identifying a first disk chain of the first snapshot hierarchy having an object running point, identifying a second disk chain of the first snapshot hierarchy different from the first disk chain, and migrating the second disk chain from the first object store to the second object store to form a first branch of a second snapshot hierarchy in the second object store. After the migrating, the example method includes instructing to take a first native snapshot on the object running point in the second object store, instructing to revert the object running point along the first branch and migrating the first disk chain from the first object store to the second object store.

Abstract: The disclosure provides techniques for deduplicating files. The techniques include, upon creating or modifying a file, placing a logical timestamp of the current logical time, within a queue associated with the directory of the file. The techniques further include placing the logical timestamp within a queue of each parent directory of the directory of the file. To determine a set of files for deduplication, the techniques disclosed herein identify files that have been modified within a logical time range. The set of files modified within a logical time is identified by traversing directories of a storage system, the directories being organized within a tree structure. If a directory's queue does not contain a timestamp that is within the logical time range, then all child directories can be skipped over for further processing, such that no files within the child directories end up being within the set of files for deduplication.

Abstract: Optimizing file access includes a process for identifying a file access event for a first accessed file, and incrementing a first access counter in an access list in a memory, which also includes access counters for other accessed files. The process further includes exporting the first access counter to a performance monitoring dashboard, or exporting to a storage allocator and, based on the value, moving the first accessed file between a first storage and a second storage. The process also includes determining whether the value of the first access counter meets a first threshold, or a sum of values of the access counters for the other accessed files meets a second threshold. Based on meeting the first threshold or meeting the second threshold, the process includes persisting the access counters on a storage media. The access counters also provide security monitoring (e.g., identifying excessive file access).

Abstract: System and method for deleting parent snapshots of running points of storage objects stored in a storage system, in response to a request to delete a parent snapshot of a running point of a storage object stored in the storage system, traverses a subtree of a B tree that corresponds to a logical map of the parent snapshot to find nodes of the subtree that are exclusively owned by the parent snapshot, which are added to an exclusive node list of the parent snapshot. The minimum node ownership value of the running point is then changed to the minimum node ownership value of the parent snapshot so that any node of the subtree of the B tree with a node ownership value equal to or greater than the changed minimum node ownership value is deemed to be owned by the running point. The nodes of the subtree of the B tree that are found in the exclusive node list of the parent snapshot are then deleted.

Abstract: A method for modifying key-value pairs of a B+ tree is provided. The method receives a request to modify a particular key-value pair. Each node of the tree has a modification number. The method traverses a path on the tree from the root node toward the particular node. The traversing includes upon reaching a parent node of the path, acquiring a shared lock on both the parent node and a child node one level below the parent node. Upon determining that the child node is the particular node, the method stores the modification number of the particular node, releases the shared lock on the particular node, compares a current modification number of the node with its stored number, and acquires an exclusive lock on the node if the numbers are the same. The method increments the current modification number of the node and modifies it while in the exclusive lock.

Abstract: A method for the reverse deletion of a plurality of snapshots in a chain of snapshots is provided. The method includes in reverse order, starting from a latest snapshot in time to an earliest snapshot in time of the plurality of snapshots: identifying at least one of a first set of one or more data blocks of a snapshot that are shared with an earlier snapshot in time in the chain of snapshots or a second set of one or more data blocks of the snapshot that are owned by the snapshot and processing the second set of one or more data blocks and skipping processing the first set of one or more data blocks, wherein processing the second set of one or more data blocks comprises performing one or more actions to maintain and/or delete data blocks of the second set of one or more data blocks.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for log-structured file system management operations. An aggregate amount of over-provisioned computing resources assigned to a plurality of log-structured file system (LFS) objects that are allocated for performance of memory management operations associated with a virtual storage area network (vSAN) can be determined. A subset of LFS objects that are candidates for performance of a particular memory management operation based on an amount of over-provisioned computing resources consumed by one or more LFS objects among the plurality of LFS objects exceeding a resource consumption threshold associated with the determined amount of over-provisioned computing resources assigned to the plurality of LFS objects can be selected. The particular memory management operation using one or more of the candidate LFS objects can be performed.

Abstract: Techniques for using data mirroring across regions to reduce the likelihood of losing objects in a cloud object storage platform are provided. In one set of embodiments, a computer system can upload first and second copies of a data object to first and second regions of the cloud object storage platform respectively, where the first and second copies are identical. The computer system can then attempt to read the first copy of the data object from the first region. If the read attempt fails, the computer system can retrieve the second copy of the data object from the second region.

Abstract: Techniques for recovering metadata associated with data backed up in cloud object storage are provided. In one set of embodiments, a computer system can create a snapshot of a data set, where the snapshot includes a plurality of data blocks of the data set that have been modified since the creation of a prior snapshot of the data set. The computer system can further upload the snapshot to a cloud object storage platform of a cloud infrastructure, where the snapshot is uploaded as a plurality of log segments conforming to an object format of the cloud object storage platform, and where each log segment includes one or more data blocks in the plurality of data blocks, and a set of metadata comprising, for each of the one or more data blocks, an identifier of the data set, an identifier of the snapshot, and a logical block address (LBA) of the data block.

Abstract: Exemplary methods, apparatuses, and systems include a file system process reading a first node in a tree data structure from a first memory. The first node includes a first approximate membership query data structure (“AMQ”), a first plurality of child pointers, a first plurality of pivot values, and a first buffer. The file system process determines that the first plurality of child pointers exceeds a maximum size. Using a pivot value in the first plurality of pivot values, the file system process splits the first node into a second node and a third node. The file system process uses the pivot value to split the first buffer into a second buffer and a third buffer. Using the pivot value and the first AMQ, the file system process generates a second AMQ and a third AMQ.

Abstract: Solutions for supporting storage using a multi-writer log-structured file system (LFS) are disclosed that include receiving incoming data from an object of a plurality of objects that are configured to simultaneously write to the LFS from different nodes; based at least on receiving the incoming data, determining whether sufficient free segments are available in a local segment usage table (SUT) for writing the incoming data; based at least on determining that insufficient free segments are available, requesting allocation of new free segments; writing the incoming data to a log; acknowledging the writing to the object; determining whether the log has accumulated a full segment of data; based at least on determining that the log has accumulated a full segment of data, writing the full segment of data to a first segment of the free segments; and updating the local SUT to mark the first segment as no longer free.

Abstract: Example methods and systems to perform a migration of a virtualized computing instance and its first snapshot hierarchy from a first object store to a second object store have been disclosed. One example method includes identifying a first disk chain of the first snapshot hierarchy having an object running point, identifying a second disk chain of the first snapshot hierarchy different from the first disk chain, and migrating the second disk chain from the first object store to the second object store to form a first branch of a second snapshot hierarchy in the second object store. After the migrating, the example method includes instructing to take a first native snapshot on the object running point in the second object store, instructing to revert the object running point along the first branch and migrating the first disk chain from the first object store to the second object store.