Abstract: The present disclosure is related to methods, systems, and machine-readable media for deleting snapshots. A deletion process can be performed responsive to receiving a request to delete a snapshot of a virtual computing instance (VCI) in a file system. The deletion process can include performing a first file comparison between the snapshot and a previous snapshot to determine first extents exclusive to the snapshot, performing a second file comparison between the snapshot and a subsequent snapshot to determine second extents exclusive to the snapshot, performing a third file comparison between the first extents and the second extents to determine common extents, wherein the common extents are common to the first extents and the second extents, and deleting the common extents from the file system.

Abstract: A method for block addressing is provided. The method includes moving content of a data block referenced by a logical block address (LBA) from a first physical block corresponding to a first physical block address (PBA) to a second physical block corresponding to a second PBA, wherein prior to the moving a logical map maps the LBA to a middle block address (MBA) and a middle map maps the MBA to the first PBA and in response to the moving, updating the middle map to map the MBA to the second PBA instead of the first PBA.

Abstract: A method for updating block addresses is provided. The method includes overwriting content of a first data block referenced by a first logical block address (LBA) with updated content. Prior to overwriting, the content of the first data block is stored in a first physical block corresponding to a first physical block address (PBA), a logical map maps the first LBA to a first middle block address (MBA), and a middle map maps the first MBA to the first PBA. After overwriting, the updated content of the first data block is stored in a second physical block corresponding to a second PBA and, in response to the overwriting, the middle map is updated to map the first MBA to the second PBA instead of the first PBA.

Abstract: The present disclosure is related to methods, systems, and machine-readable media for snapshot space reporting. A first probabilistic data structure can be created for a first snapshot of a virtual computing instance (VCI) in a file system based on a hash of physical block numbers of a plurality of blocks of the first snapshot. A second probabilistic data structure can be created for a second snapshot of the VCI based on a hash of physical block numbers of a plurality of blocks of the second snapshot. A space report can be determined for the first and second snapshots based on the first probabilistic data structure and the second probabilistic data structure, wherein the space report is indicative of the storage space occupied by the first and second snapshots. A file system function can be performed by reference to the space report.

Abstract: Techniques for using data mirroring across regions to reduce the likelihood of losing objects in a cloud object storage platform are provided. In one set of embodiments, a computer system can upload first and second copies of a data object to first and second regions of the cloud object storage platform respectively, where the first and second copies are identical. The computer system can then attempt to read the first copy of the data object from the first region. If the read attempt fails, the computer system can retrieve the second copy of the data object from the second region.

Abstract: Techniques for implementing dynamic base disk mirroring for linked clones are provided. In one set of embodiments, a first node in a distributed storage system can monitor a congestion level of a base disk residing on the first node, where the base disk is shared by a plurality of linked clones. Upon determining that the congestion level exceeds a threshold, the first node can send, to a second node, a request to create a mirror of the base disk on that second node. Upon receiving an acknowledgement from the second node that the mirror has been successfully created, the first node can update a mirror set associated with the base disk to include an entry identifying the mirror. The first node can then communicate the updated mirror set to one or more other nodes.

Abstract: Techniques for combining the metadata and data address spaces of a distributed storage object are provided. In one set of embodiments, a distributed storage system can receive a request to provision a storage object. In response, the distributed storage system can create, in accordance with an erasure coding scheme, one or more capacity components for holding data of the storage object; create, in accordance with a mirroring scheme having an equivalent level of fault tolerance as the erasure coding scheme, one or more metadata components for holding metadata of the storage object; and create a composite object configuration tree for the storage object that includes first and second subtrees, where the first subtree comprises an indication of the mirroring scheme and references to the one or more metadata components, and where the second subtree comprises an indication of the erasure coding scheme and references to the one or more capacity components.

Abstract: A method includes obtaining a plurality of representations corresponding respectively to a plurality of blocks of data stored on a source node. A plurality of data pairs are sent to a destination node, where each data pair includes a logical address associated with a block of data from the plurality of blocks of data and the corresponding representation of the block of data. A determination is made whether the blocks of data associated with the respective logical addresses are duplicates of data stored on the destination node. In accordance with an affirmative determination, a reference to a physical address of the block of data stored on the destination node is stored. In accordance with a negative determination, an indication that the data corresponding to the respective logical address is not a duplicate is stored. The data indicated as not being a duplicate is written to the destination node.

Abstract: Example methods are provided for performing fast building and testing a software suite with multiple software components. In one example, the method may include obtaining a changed code file, identifying a software component of the software suite impacted by the changed code file, and instructing to generate a software component build based on the software component but excluding other software components of the software suite. Before completing generating the software component build, the method may also include selecting a software suite build. The method further includes instructing to prepare a testbed based on the software suite build and instructing to test the software component build on the testbed.

Abstract: Techniques for issuing efficient writes to an erasure coded storage object in a distributed storage system via adaptive logging are provided. In one set of embodiments, a node of the system can receive a write request for updating one or more logical data blocks of the storage object and determine whether a size of the one or more logical data blocks meets or exceeds a threshold size. Upon determining that the size of the one or more logical data blocks meets or exceeds the threshold size, the node can allocate a segment in a capacity object of the storage object, write the one or more logical data blocks via a full stripe write to the segment, and write metadata for the one or more logical data blocks to a log record in a log of a metadata object of the storage object. The metadata written to the log record can include mappings between logical block addresses (LBAs) of the one or more logical data blocks and physical block addresses (PBAs) where the one or more logical data blocks reside in the segment.

Abstract: Techniques for implementing dynamic base disk mirroring for linked clones are provided. In one set of embodiments, a first node in a distributed storage system can monitor a congestion level of a base disk residing on the first node, where the base disk is shared by a plurality of linked clones. Upon determining that the congestion level exceeds a threshold, the first node can send, to a second node, a request to create a mirror of the base disk on that second node. Upon receiving an acknowledgement from the second node that the mirror has been successfully created, the first node can update a mirror set associated with the base disk to include an entry identifying the mirror. The first node can then communicate the updated mirror set to one or more other nodes.

Abstract: The present disclosure provides techniques for deallocating previously allocated storage blocks. The techniques include obtaining a list of chunk IDs to analyze, choosing a chunk ID, and determining the storage blocks spanned by the chunk corresponding to the chosen chunk ID. The technique further includes determining whether any file references any storage blocks spanned by the chunk. The determining may be performed by comparing an internal reference count to a total reference count, where the internal reference count is the number of reference to the storage block by a chunk ID data structure. If no files reference any of the storage blocks spanned by the chunk, then all the storage blocks of the chunk can be deallocated.

Abstract: Example methods relating to maintain a quality of service of a cloud application are described. One example method includes monitoring a workload metric of a virtual machines which is running the cloud application, creating additional one or more virtual machines, isolating the additional one or more virtual machines, and determining whether the workload metric exceeds a threshold. In response to determining that the workload metric exceeds the threshold, the method includes deisolating the additional one or more virtual machines to support the cloud application. Otherwise, the method includes maintaining one or more virtual machines being isolated.

Abstract: Techniques for improving the read performance of an LFS-based storage system that supports COW snapshotting are provided. In one set of embodiments, the storage system can implement an intermediate map for each storage object in the system that is keyed by a composite key consisting of snapshot identifier (major key) and LBA (minor key). With this approach, contiguous logical block addresses (LBAs) of a storage object or its snapshots will map to contiguous <Snapshot ID, LBA>-to-PBA mappings in the storage object's intermediate map, resulting in good spatial locality for those LBAs and robust read performance.

Abstract: Optimizing file access includes a process for identifying a file access event for a first accessed file, and incrementing a first access counter in an access list in a memory, which also includes access counters for other accessed files. The process further includes exporting the first access counter to a performance monitoring dashboard, or exporting to a storage allocator and, based on the value, moving the first accessed file between a first storage and a second storage. The process also includes determining whether the value of the first access counter meets a first threshold, or a sum of values of the access counters for the other accessed files meets a second threshold. Based on meeting the first threshold or meeting the second threshold, the process includes persisting the access counters on a storage media. The access counters also provide security monitoring (e.g., identifying excessive file access).

Abstract: Embodiments of the present disclosure relate to techniques for performing a merge update for a database. In particular, certain embodiments of a method include generating a message comprising a first key and a first transaction associated with the first key, the first transaction indicating a transaction to perform other than for key-value pairs comprising the first key. The method further includes storing the message in a database. The method further includes merging the message with a first key-value pair stored in the database, the first-key value pair comprising the first key. The method further includes performing the first transaction based on merging the message with the first key-value pair.

Abstract: Solutions for managing archived storage include receiving, at a first node, a snapshot comprising object data (e.g., a virtual machine disk snapshot) from a second node (e.g., a software defined data center), and storing the snapshot in a tiered structure that includes a data tier and a metadata tier. Snapshots may be used for fail-over operations and/or backups, to support disaster recovery. The data tier comprises a log-structured file system (LFS), and the metadata tier comprises a content addressable storage (CAS) identifying addresses within the LFS. The metadata tier also comprises a logical layer indicating content in the CAS. Segment cleaning of the data tier is performed using a segment usage table (SUT). Some examples include performing a fail-over operation from the second node to a third node using at least the stored snapshot for workload recovery. In some examples, the CAS comprises a log-structured merge-tree (LSM-tree).

Abstract: A method for cleaning an object storage having a plurality of segments is provided. Each segment includes an identifier through which the segment is accessed. The method identifies a first segment in the plurality of segments. The first segment includes a first identifier and a first size. The method determines that a utilization ratio for the first segment is below a threshold. As a result, the method generates a second segment from the first segment, such that the second segment includes a second identifier that is the same as the first identifier and a second size that is smaller than the first size. The method then writes the second segment to the object storage.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for resynchronizing data in a storage system. One of the methods includes determining that a particular disk of a capacity object of a storage system was offline for an interval of time, wherein the capacity object comprises a plurality of segments, and wherein the storage system comprises a segment usage table identifying a linked list of particular segments of the capacity object that are in use; determining a time point at which the particular disk went offline; determining one or more first segments of the capacity object that were modified after the time point, wherein determining one or more first segments comprises determining each segment of the segment usage table having a transaction ID that is larger than the time point; and resynchronizing, for each first segment, a portion of the particular disk corresponding to the first segment.

Abstract: A method for compressing is provided. The method including receiving a block of data to store on at least one physical disk; determining whether to store the data in a data log as uncompressed or compressed data based on a determined size of resulting compressed data. When the method determines to store the data as compressed, compressing the data and storing the compressed data in at least one sector in the data log. Otherwise, the method stores the data, uncompressed, in a plurality of sectors in the data log. The method generates a one or more state bits indicating (i) whether the data is stored as uncompressed or compressed, and (ii) if the data is stored as compressed, a size of the compressed data. The method then stores the one or more state bits in an entry of a logical map table associated with an LBA that corresponds to the data block.