Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for reclaiming one or more portions of storage resources in a computer system serving one or more virtual computing instances, where the storage resources in the computer system are organized in clusters of storage blocks. In one aspect, a method includes maintaining a respective block tracking value for each storage block that indicates whether a call to reclaim the storage block is outstanding; determining, from the block tracking values, a respective cluster priority value for each of the clusters based on a count of storage blocks in the respective cluster for which a call to reclaim is outstanding; and reclaiming a first portion of storage resources in the computer system in accordance with the cluster priority values.

Abstract: Exemplary methods, apparatuses, and systems determine whether a skip optimization process can be used to store a file in a storage space. When it is determined that the skip optimization can be performed, a file stored in the storage space can be referenced in a file metadata data structure using direct addressing of file blocks storing the file instead of through indirect addressing (e.g., pointer addresses stored in pointer blocks).

Abstract: The systems described herein are configured to enhance the efficiency of memory usage and access in a VM file system data store with respect to allocating memory in large and small file block clusters using affinity metadata and propagating and maintaining the affinity metadata in support of the described allocation. During storage of file data, an affinity identifier of the file data is determined. The affinity identifier is used to identify a large file block cluster and a small file block cluster within the identified large file block cluster. The file data is stored in the selected small file block cluster and affinity metadata of the selected small file block cluster is updated to reflect the storage of the file data.

Abstract: The systems described herein are configured to enhance the efficiency of memory in a host file system with respect to hosted virtual file systems. In situations when the hosted virtual file systems use smaller file block sizes than the file block sizes of the host file system. During storage of a file, a file block is assigned a block address and unmapping bits. The block address and unmapping bits are stored in a pointer block or other similar data structure associated with the file. Particularly, the block address is stored in a first address block and the unmapping bits are stored in at least one additional address block located in proximity to the block address, such that the unmap granularity of the file is not limited by the fixed size of address blocks in the system.

Abstract: The systems described herein are configured to enhance the efficiency of memory usage and access in a VM file system data store with respect to allocating memory in large and small file block clusters using affinity metadata and propagating and maintaining the affinity metadata in support of the described allocation. In order to maintain affinity metadata of the large file block cluster, affinity generation values stored on the large file block cluster are read and cached affinity generation values for each small file block cluster are read from an in-memory cache associated with the large file block cluster. When the stored affinity generation values and the cached affinity generation values do not match, affinity metadata from all the small file block clusters associated with the large file block cluster is used to update the affinity metadata of the large file block cluster and the associated cache.

Abstract: System and method for executing a file system operation for a computer system utilize a computed hash value of a file system object to access a hash block of a file system directory stored in a storage system to locate a hash slot corresponding to the computed hash value. Using at least one of a hash pointer in the hash slot and an allocation block of the file system directory, a dirent slot in a dirent block of the file system directory is located to perform an operational task on the particular dirent slot to execute the file system operation.

Abstract: Techniques for decoupling the commit and replay of file system metadata updates in a clustered file system (CFS) are provided. In one embodiment, a CFS layer of a computer system can receive a file I/O operation from a client application, where the file I/O operation involves an update to a file system metadata resource maintained on persistent storage. In response, a journaling component of the CFS layer can execute a commit phase for committing the update to a journal on the persistent storage. The CFS layer can then return an acknowledgment to the client application indicating that the file I/O operation is complete, where the acknowledgement is returned prior to completion of a replay phase configured to propagate the update from the journal to one or more locations on the persistent storage where the file system metadata resource is actually stored.

Abstract: A distributed file system may be configured with file blocks of a first type and file blocks of a second type, from allocation units that comprise a logical volume containing the file system. File blocks of the second type may be defined from one or more file blocks of the first type. A thick file may be instantiated with a number of allocation units totaling a size greater than or equal to a specified file size of the thick file. The allocation units may be allocated to the thick file in units of file blocks of the first type or file blocks of the second type, depending on the specified file size of the thick file.

Abstract: Techniques for enabling end-to-end compression when performing host-based replication (HBR) are provided. In one set of embodiments, a hypervisor running on a host system at a primary site can intercept I/O write requests that originate from a virtual machine (VM) configured to be replicated via HBR, the I/O write requests being destined for a virtual disk (VMDK) of the VM. The hypervisor can further track VMDK file blocks that are modified by the I/O write requests and can retrieve the VMDK file blocks from a storage tier at the primary site. The hypervisor can then compress the retrieved VMDK file blocks and transmit the compressed blocks to a secondary site.

Abstract: Examples provide input and output request block size compatibility. A storage filter converts input and output (IO) requests associated with a first data block size into modified IO requests compatible with a data storage organized in a second data block size where the first data block size is different than the first data block size. The storage filter translates read IO requests for a smaller block size into modified read requests for a data storage organized with a larger data block size. Write IO requests for smaller block size are converted into modified write IO requests for larger data block size data storage. The storage filter also converts read IO requests generated for larger block size into smaller block size read IO requests. Likewise, the storage filter also translates write IO requests corresponding to larger data block size into modified write IO requests of smaller block size.

Abstract: Techniques for enabling end-to-end compression when performing host-based replication (HBR) are provided. In one set of embodiments, a hypervisor running on a host system at a primary site can intercept I/O write requests that originate from a virtual machine (VM) configured to be replicated via HBR, the I/O write requests being destined for a virtual disk (VMDK) of the VM. The hypervisor can further track VMDK file blocks that are modified by the I/O write requests and can retrieve the VMDK file blocks from a storage tier at the primary site. The hypervisor can then compress the retrieved VMDK file blocks and transmit the compressed blocks to a secondary site.

Abstract: Controlling backup data access is disclosed. A request is received to access backup data in a requested mode, wherein backup data is accessed based at least in part on a restore point. It is determined if the requested mode is allowed given a currently executing mode. In the event that the mode is allowed, access to backup data is allowed.

Abstract: In order to synchronize copies of a virtual disk, a virtualization layer maintains a first record of file system blocks of a first copy of the virtual disk that are modified during an access session by a virtual machine using the first copy of the virtual disk. The file system blocks correspond to a file system of the virtual disk. During an attempt to synchronize the first copy with a second copy of the virtual disk, (i) a second record of file system blocks that are currently used by the file system is obtained from the guest operating system, and (ii) file system blocks in the first copy of the virtual disk that are present in both the first record and the second record are copied into the second copy of the virtual disk.

Abstract: A backup method for a computer system network avoids generating hashes from data that may be inaccurate due to network errors affords verification of source data written to backup media includes reading at a network client a portion of the source data from a source storage volume and generating a hash of the source data at the client. The source data and hash are transmitted to a backup server and written to the backup media. The hash is written to the backup media is associated with the backup data corresponding to the source data from which the hash was generated. Verification of the backup data is performed by reading the backup data from the backup media, generating a hash from the backup data read, and comparing that hash with the hash originally stored on the backup media.

Abstract: A method for grooming a backup data set is disclosed. The method comprises receiving data associated with one or more restore points associated with the backup data set. The method further comprises automatically selecting one or more of said one or more restore points for retention, removal, or transfer.

Abstract: Computer implemented methods, computer program products and computer systems synchronize copies of a virtual disk. A record of blocks that are modified during an access session of a copy of the virtual disk is maintained. For each partition of the virtual disk, a file system level indication of currently relevant blocks is obtained. Only those blocks that were modified during the access session and are currently relevant are copied to at least one additional copy of the virtual disk.

Abstract: A backup method for a computer system network avoids generating hashes from data that may be inaccurate due to network errors affords verification of source data written to backup media includes reading at a network client a portion of the source data from a source storage volume and generating a hash of the source data at the client. The source data and hash are transmitted to a backup server and written to the backup media. The hash is written to the backup media is associated with the backup data corresponding to the source data from which the hash was generated. Verification of the backup data is performed by reading the backup data from the backup media, generating a hash from the backup data read, and comparing that hash with the hash originally stored on the backup media.

Abstract: A backup method for a computer system network avoids generating hashes from data that may be inaccurate due to network errors affords verification of source data written to backup media includes reading at a network client a portion of the source data from a source storage volume and generating a hash of the source data at the client. The source data and hash are transmitted to a backup server and written to the backup media. The hash is written to the backup media is associated with the backup data corresponding to the source data from which the hash was generated. Verification of the backup data is performed by reading the backup data from the backup media, generating a hash from the backup data read, and comparing that hash with the hash originally stored on the backup media.

Abstract: A method of automatically classifying clients into different backup policy groups according to pre-selected criteria, such as type of client, whether the client is a server, a desktop or a mobile computer, the department or business unit, or even the user. The method uses information obtained from the client itself to classify the client. This information includes the kind of processor platform and operating system in the client, the type and version of software being run, and whether the client has a battery, indicating that it is a mobile computer. Other information that can be used to classify the client for backup may be obtained from information that identifies the user, the department to which the client is assigned, and the location in which it is used. The process enables a mobile client traveling to another site of an enterprise to connect to a local LAN of the enterprise and be authenticated for backup, and to be automatically assigned an appropriate backup classification.

Abstract: Efficient incremental backups using a change database is disclosed. Based at least in part on a file attribute it is determined that a file has changed since a previous backup. Change data is stored in a data structure additional to a file system with which a file is associated. Change data indicates the file has changed since the previous backup. The file attribute is configured to be reset, each time a file is backed up, to a value indicating the file has not changed. Storing the change data enables a backup operation performed subsequent to the file attribute being reset to determine that the file has changed since the previous backup.