Abstract: The present disclosure is related to methods, systems, and machine-readable media for deleting snapshot pages using sequence numbers and page lookups. A monotonically-increasing sequence number (SN) can be assigned to each created page of a first snapshot of a storage volume. A first snapshot sequence number (snapSN) can be assigned to the first snapshot responsive to a creation of a second snapshot, wherein the first snapSN is equal to a largest SN of the first snapshot. An SN can be assigned to each created page of the second snapshot, wherein a first page of the second snapshot is assigned an SN monotonically increased from the first snapSN. A second snapSN can be assigned to the second snapshot responsive to a creation of a third snapshot, wherein the second snapSN is equal to a largest SN of the second snapshot. An SN can be assigned to each created page of the third snapshot, wherein a first page of the third snapshot is assigned an SN monotonically increased from the second snapSN.

Abstract: A method for encryption in a distributed datastore is provided. The method generally includes receiving random data from a virtualization management platform as a wrapped data encryption key (DEK), retrieving a key encryption key (KEK) from a key management server (KMS), decrypting the wrapped DEK using the KEK to determine a blank DEK, where the blank DEK is common to a plurality of hosts of a host cluster accessing a distributed datastore; encrypting first data using the blank DEK, and storing the encrypted first data in one or more disks of the distributed datastore, the one or more disks belonging to the plurality of hosts.

Abstract: System and method for managing copy-on-write (COW) B tree structures for metadata of storage objects stored in a storage system determine, when a request to modify a target storage object stored in the storage system that requires a modification of a target leaf node in a B tree structure for metadata of the target storage object is received, whether an operation sequence number of the target leaf node is greater than a snapshot sequence number of a parent snapshot of a running point of the B tree structure. When the operation sequence number is greater than the snapshot sequence number, the target leaf mode is modified in place without copying the target leaf node. When the operation sequence number is not greater than the snapshot sequence number, the target leaf node is copied as a new leaf node for the B tree structure and the new leaf node is modified.

Abstract: Linked clone read performance when retrieving data from a clone is improved at least by aggregating block mapping metadata efficiently. Primary metadata for a child clone maps a logical block address (LBA) for data in a data region of the child clone to a physical sector address (PSA) for data in the data region of the child clone. At least a portion of primary metadata for a parent clone of the child clone is copied into archival metadata for the child clone. In response to a read request, data is returned from the child clone, parent clone, or another ancestor of the child clone based on whether or not a read request LBA is within the primary metadata for the child clone, or within the archival metadata.

Abstract: A method for reverse range lookup in an ordered data structure of keys, wherein each key comprises a logical block address (LBA) and a snapshot identifier (ID) of one of one or more snapshots in a snapshot hierarchy, is provided. The keys in the ordered data structure are in an order from smallest to largest LBA, wherein in the order, keys having a same LBA are ordered from smallest to largest snapshot ID. The method includes determining a range of LBAs and traversing the ordered data structure in reverse order until a key is found that: has an LBA and a snapshot ID that is less than or equal to the last LBA and the largest snapshot ID, respectively. The method further includes adding an extent corresponding to the located key to an extent result list if the snapshot ID of the located key is not within a deny list.

Abstract: The disclosure herein describes management of distribution of resources between a global pool and an associated plurality of local pools using a flush threshold. A request for resources is received at the global pool from a local pool, the request indicating a requested quantity of resources. Based on the received request, it is determined that available resources in the global pool are below a flush threshold of the global pool. Based on this determination, flush instructions are sent to the local pools, wherein the flush instructions instruct each local pool to release unused resources (e.g., available to be released) to the global pool. Based on the available resources of the global pool then exceeding the requested quantity of resources and/or the flush threshold, resources of the global pool are allocated to the requesting local pool, whereby the local pool is enabled to use the allocated resources.

Abstract: Examples provide a method of communication between a client driver and a filesystem server. The client driver executes in a virtual machine (VM) and the filesystem server executes in a hypervisor. The method includes: allocating, by the client driver, shared memory in an address space of the VM for the communication; sending identification information for the shared memory from the client driver to the filesystem server through an inter-process communication channel between the client driver and the filesystem server; identifying, by the filesystem server in cooperation with a kernel of the hypervisor, the shared memory within an address space of the hypervisor, based on the identification information, to create a shared memory channel; sending commands from the client driver to the filesystem server through the shared memory channel; and receiving completion messages for the commands from the filesystem server to the client driver through the shared memory channel.

Abstract: Techniques for efficiently storing client data blocks on a distributed-computing system are provided. The system includes a fast performance tier and a large capacity tier. The capacity tier stores the client data blocks in erasure encoded data stripes. The performance tier stores logical map data including an address map indicating a correspondence between logical addresses associated with a first layer of the system and physical addresses associated with a second layer. A method includes receiving a request to include additional client data blocks in the client blocks. The request indicates logical addresses for additional blocks. Corresponding physical addresses for additional block are determined. Each additional block is stored at the physical address. Additional logical map data is stored in the performance tier. Storing the additional logical map data includes updating the address map to indicate the correspondence between the logical addresses and the physical addresses for the additional blocks.

Abstract: Exemplary methods, apparatuses, and systems include a file system process inserting a first key/value pair and a second key/value pair into a first tree. The second key is a duplicate of the first key and the value of the second key/value pair is an operation changing the value. In response to a request for a range of key/value pairs, the process reads the second key/value pair and inserts it in a second tree. The process reads the first pair and determines, while inserting the first pair in the second tree, that the second key is a duplicate of the first key. The file system process determines an updated value of the first value by applying the operation in the second value to first value. The file system operation updates the second key/value pair in the second tree with the updated value and returns the requested range of key/value pairs.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for supporting deduplication in file storage using file chunk hashes. A hash of a chunk of a log segment can be received from a software defined data center. A chunk identifier can be associated with the hash in a hash map that stores associations between sequentially-allocated chunk identifiers and hashes. The chunk identifier can be associated with a logical address corresponding to the chunk of the log segment in a logical map that stores associations between the sequentially-allocated chunk identifiers and logical addresses. A search of the hash map can be performed to determine if the chunk is a duplicate, and the chunk can be deduplicated responsive to a determination that the chunk is a duplicate.

Abstract: An example method of memory management in a computing system having a plurality of processors includes: receiving a first memory allocation request at a memory manager from a process executing on a processor of the plurality of processors in the computing system; allocating a local memory pool for the processor from a global memory pool for the plurality of processors in response to the first memory allocation request; and allocating memory from the local memory pool for the processor in response to the first memory allocation request without locking the local memory pool.

Abstract: Techniques for efficiently managing a file clone from a filesystem which supports efficient volume snapshots are provided. In some embodiments, a system may receive an instruction to remove the file clone from the filesystem. The file clone may be a point-in-time copy of metadata of an original file. The system may further—for a file map entry in a filesystem tree associated with the file clone, the file map entry indicating a data block—decrement a reference count in a reference count entry associated with the file map entry. The reference count entry may be stored in the filesystem tree according to a key and the key may comprise an identification of the original file. The system may further reclaim the data block in a storage system when the reference count is zero.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for maintaining fault tolerance for new writes in a storage system when one or more components of the storage system are unavailable. One of the methods includes determining that one or more first disks of a capacity object of a storage system are unavailable, wherein the storage system comprises a segment usage table identifying the plurality of segments of the capacity object; in response: identifying a plurality of available second disks, adding a plurality of new segments corresponding to the second disks to the capacity object, and adding data identifying the plurality of new segments to the segment usage table; and for each of one or more new write requests to the capacity object: identifying an available segment from the plurality of new segments, and writing data associated with the new write request to the identified available segment.

Abstract: Scalable segment cleaning for log-structured file systems (LFSs) includes determining counts of segment cleaners and virtual nodes, with each virtual node being associated with a plurality of objects. Each virtual node is assigned to a selected segment cleaner. Based at least on the assignments, performing, for each virtual node, segment cleaning of the objects by the assigned segment cleaner. A portion, less than all, of the virtual nodes are reassigned to a newly selected segment cleaner based on a change of the count of the segment cleaners and/or a change of the count of the virtual nodes. Based at least on the reassignments, segment cleaning of the objects is performed, for each reassigned virtual node, by the reassigned segment cleaner. In some examples, the objects comprise virtual machine disks (VMDKs) and the segment cleaning uses a segment usage table (SUT) to track segment usage and identify segment cleaning candidates.

Abstract: Processing write requests from clients includes logging the associated data into logs corresponding to the data objects targeted by the write requests. The logs are persisted by combining log entries from each of the logs into one ore more fixed-size data blocks. The fixed-size data blocks are inserted into a data tree stored on a block-based storage device.

Abstract: A method for updating block addresses is provided. The method includes overwriting content of a first data block referenced by a first logical block address (LBA) with updated content. Prior to overwriting, the content of the first data block is stored in a first physical block corresponding to a first physical block address (PBA), a logical map maps the first LBA to a first middle block address (MBA), and a middle map maps the first MBA to the first PBA. After overwriting, the updated content of the first data block is stored in a second physical block corresponding to a second PBA and, in response to the overwriting, the middle map is updated to map the first MBA to the second PBA instead of the first PBA.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for file system event monitoring using metadata snapshots. A traditional snapshot of a virtual computing instance (VCI) can be created in a file system. The snapshot can correspond to an extent. An indication can be made that the extent is owned by a single snapshot. A metadata snapshot, corresponding to the extent, can be created without changing the indication that the extent is owned. The extent can be modified, wherein the indication that the extent is owned causes the extent to be modified without allocating a new extent.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for deleting snapshots. A deletion process can be performed responsive to receiving a request to delete a snapshot of a virtual computing instance (VCI) in a file system. The deletion process can include performing a first file comparison between the snapshot and a previous snapshot to determine first extents exclusive to the snapshot, performing a second file comparison between the snapshot and a subsequent snapshot to determine second extents exclusive to the snapshot, performing a third file comparison between the first extents and the second extents to determine common extents, wherein the common extents are common to the first extents and the second extents, and deleting the common extents from the file system.

Abstract: A method for block addressing is provided. The method includes moving content of a data block referenced by a logical block address (LBA) from a first physical block corresponding to a first physical block address (PBA) to a second physical block corresponding to a second PBA, wherein prior to the moving a logical map maps the LBA to a middle block address (MBA) and a middle map maps the MBA to the first PBA and in response to the moving, updating the middle map to map the MBA to the second PBA instead of the first PBA.

Abstract: A method for updating block addresses is provided. The method includes overwriting content of a first data block referenced by a first logical block address (LBA) with updated content. Prior to overwriting, the content of the first data block is stored in a first physical block corresponding to a first physical block address (PBA), a logical map maps the first LBA to a first middle block address (MBA), and a middle map maps the first MBA to the first PBA. After overwriting, the updated content of the first data block is stored in a second physical block corresponding to a second PBA and, in response to the overwriting, the middle map is updated to map the first MBA to the second PBA instead of the first PBA.