Abstract: The present disclosure relates to bootstrapping an encrypted single node VSAN cluster. One method includes receiving a request to create an encrypted VSAN cluster from a single host in a software-defined datacenter, deploying a virtual server on a VSAN datastore of the software-defined datacenter, registering a native key provider (NKP) in the virtual server, creating an empty VSAN cluster encrypted by the NKP, adding the single host to the encrypted empty cluster to create a one-host encrypted cluster, registering a KMIP KMS in the virtual server, switching encryption of the one-host encrypted cluster from the NKP to the KMIP KMS, and adding another host to the one-host encrypted cluster to create the encrypted cluster.

Abstract: An example method of providing a common volume (cVol) datastore for virtual machines (VMs) managed by a hypervisor in a cloud computing system includes: mounting, by the hypervisor in cooperation with a network file system server, a network file system share of a common volume (cVol), the network file system share storing metadata for the VMs; creating a file system container backed by the network file system share; routing file operations targeting the metadata to the file system container; attaching cloud volumes as devices on a host of the hypervisor, the cloud volumes referenced by descriptors in the metadata; and routing file operations targeting virtual disks of the VMs to the devices.

Abstract: An example method for a first host, being an owner of an object stored in a virtual storage area network (vSAN) cluster, to perform encryption and decryption operations during a rekey in the vSAN cluster is disclosed. The method includes obtaining a first encryption key and a first key identifier (ID) of the first encryption key; transmitting the first key ID and an active key index to a second host; using the first encryption key to perform encryption and decryption operations; and in response to a determination of receiving a key change notification from a master node of the vSAN cluster, terminating a connection with the second host.

Abstract: A method for encryption in a distributed datastore is provided. The method generally includes receiving random data from a virtualization management platform as a wrapped data encryption key (DEK), retrieving a key encryption key (KEK) from a key management server (KMS), decrypting the wrapped DEK using the KEK to determine a blank DEK, where the blank DEK is common to a plurality of hosts of a host cluster accessing a distributed datastore; encrypting first data using the blank DEK, and storing the encrypted first data in one or more disks of the distributed datastore, the one or more disks belonging to the plurality of hosts.

Abstract: A technique for managing storage space includes establishing an MSR (minimum space reservation) of a thin file system built upon a storage pool. In response to a set of storage requests to the file system to store new data, the method further includes obtaining an additional space guarantee from the storage pool. After deleting data from the file system such that the size of the file system falls below the MSR, the technique further includes performing a space reclaim operation, the space reclaim operation (i) compacting the file system to a size less than the MSR, (ii) canceling the additional space guarantee such that the storage pool no longer guarantees the additional space for the file system, and (iii) continuing to reserve the full MSR of the file system, even though the size of the file system is smaller than the MSR.

Abstract: A data storage system manages use of a pool of secondary storage by internal file systems hosting logical data storage objects accessed by clients. A choose-and-mark-slices operation scans the file system to identify a subset of the slices as evacuatable slices to be de-provisioned and returned to the pool, the subset having a size exceeding a per-iteration limit observed by a separate evacuate-slices operation. The subset is recorded in an in-memory structure. Each iteration of the evacuate-slices operation (1) examines the in-memory structure to select slices for evacuation, no more than the per-iteration limit being selected, (2) moves data from the selected slices to the remaining slices of the set, and (3) subsequently (i) updates metadata of the file system to reflect the moving of the data and (ii) de-provisions the selected slices from the file-system to return them to the pool for subsequent re-provisioning.

Abstract: A method includes (a) while processing data storage requests from users directed to filesystems of a plurality of filesystems served from a data storage system, receiving a request to execute a filesystem reorganization task on a filesystem of the plurality of filesystems, (b) in response to receiving the request, checking whether the filesystem reorganization task has both a global reorganization token and a filesystem-specific reorganization token, the filesystem-specific reorganization token being specific to the filesystem on which the filesystem reorganization task has been requested to execute, and (c) in response to determining that the filesystem reorganization task does have both the global reorganization token and the filesystem-specific reorganization token, executing the filesystem reorganization task to move a data block from one location within the filesystem to another location within the filesystem.

Abstract: A technique for performing storage tiering accesses allocation metadata in a data storage system and applies that allocation metadata when relocating data from a selected extent to a target extent. The selected extent includes a range of contiguous blocks. The allocation metadata may designate each of these blocks as either allocated or free. When relocating data from the selected extent to the target extent, the technique copies data of the selected extent on a per-block basis, checking whether that block is allocated or free before copying it to the target extent.

Abstract: Techniques for improving storage efficiency of a data storage system are provided. The techniques include (a) performing a testing operation to evaluate whether a test condition is satisfied for a filesystem having a plurality of subspaces, each subspace being provisioned with a respective amount of storage, each subspace of the filesystem being dedicated to storing different respective types of filesystem content, wherein the test condition indicates that a first subspace has available space in excess of a threshold percentage, (b) in response to determining that the test condition is satisfied for the filesystem, performing a reduction operation on the first subspace to remove a portion of storage provisioned to that subspace while retaining the removed portion for use by the filesystem, and (c) upon a second subspace reaching a fullness threshold, provisioning the removed portion to the second subspace and storing content of the respective type of the second subspace thereon.

Abstract: Techniques for improving storage efficiency of a data storage system are provided. The techniques include (a) performing a testing operation to evaluate whether a test condition is satisfied for a filesystem having a plurality of subspaces, each subspace being provisioned with a respective amount of storage, each subspace of the filesystem being dedicated to storing different respective types of filesystem content, wherein the test condition indicates that a first subspace has available space in excess of a threshold percentage, (b) in response to determining that the test condition is satisfied for the filesystem, performing a reduction operation on the first subspace to remove a portion of storage provisioned to that subspace while retaining the removed portion for use by the filesystem, and (c) upon a second subspace reaching a fullness threshold, provisioning the removed portion to the second subspace and storing content of the respective type of the second subspace thereon.

Abstract: A data storage system manages use of a pool of secondary storage by internal file systems hosting logical data storage objects accessed by clients. A choose-and-mark-slices operation scans the file system to identify a subset of the slices as evacuatable slices to be de-provisioned and returned to the pool, the subset having a size exceeding a per-iteration limit observed by a separate evacuate-slices operation. The subset is recorded in an in-memory structure. Each iteration of the evacuate-slices operation (1) examines the in-memory structure to select slices for evacuation, no more than the per-iteration limit being selected, (2) moves data from the selected slices to the remaining slices of the set, and (3) subsequently (i) updates metadata of the file system to reflect the moving of the data and (ii) de-provisions the selected slices from the file-system to return them to the pool for subsequent re-provisioning.

Abstract: A data storage system manages a pool of secondary storage by internal file systems hosting logical data storage objects accessed by clients, the pool organized into fixed-size, dynamically provisioned slices. A first choose-and-mark-slices operation scans a set of slices to identify a first subset as evacuatable, and records them as a recorded subset in an in-memory structure. Iterations of an evacuate-slices operation (1) select slices from among the recorded subset, observing a per-iteration limit, (2) move data from the selected slices to remaining slices of the set, and (3) subsequently (i) updates file system metadata to reflect the data movement and (ii) de-provisions the selected slices and returns them to the pool. Between iterations, a change in the composition of evacuatable slices is detected, and a second choose-and-mark-slices operation re-scans the slices and records a second subset as the recorded subset for a subsequent iteration of the evacuate-slices operation.

Abstract: A computer-implemented technique performs block deduplication. The technique involves generating a list of candidate blocks and adding classifications to the list of candidate blocks. The classifications classify the candidate blocks on the list into different categories based on a set of block sequence matching criteria. The technique further involves performing, based on the classifications added to the list of candidate blocks, further block deduplication processing on first candidate blocks having a first classification while not performing further block deduplication processing on second candidate blocks having a second classification. Examples of the categories include an “exact long match” category, a “disrupted long match” category, a “loosely long match” category, and a “short match” category.

Abstract: Techniques for shrinking a filesystem backed by a volume identify slices in the volume to be evacuated in order to reach a target size, identify a target endpoint in the volume, and evacuate identified slices to available locations prior to the target endpoint. The same data is typically not moved from slice to slice multiple times.

Abstract: Improved techniques of implementing range lock involve granting a sleeping thread access to an address range when no conflicts exist with earlier pending requests as well as with already granted requests. Along these lines, a current child thread that has a conflict with a parent thread that currently holds a range lock on a range of bytes during a read/write operation will be awoken from waiting state when the parent thread has completed its access and releases its lock. In response, the processor compares the range to which the current child thread request access with the ranges to which other sleeping threads request access. However, the comparisons are only performed in connection with (i) requests that arrived prior to the current child thread and (ii) other granted requests.