Abstract: Techniques for tuning the key space of an instance of a tree data structure are provided. In one embodiment, a computer system can receive a request to create the instance, where the request includes an expected key space value indicating an expected range of key values to be addressed by the instance. The computer system can further calculate a number of bits to allocate to each key of each node of the instance based on the expected key space value. The computer system can then, at a time of instantiating each node of the instance, allocate the keys for the node in accordance with the calculated number of bits.

Abstract: Techniques for dynamically allocating keys in an instance of a tree data structure are provided. In one embodiment, a computer system can, at a time of instantiating each non-root node in the instance, determine a key space to be addressed by the non-root node, where the key space is based on a key subinterval in a parent node of the non-root node that is associated with a pointer to the non-root node. The computer system can further calculate a number of bits to allocate to each key of the non-root node in view of the determined key space. The computer system can then allocate the keys of the non-root node in accordance with the calculated number of bits.

Abstract: A method for restoring a data volume using incremental snapshots of the data volume includes creating a first series of incremental snapshots according to a first predefined interval. The method further includes creating a second series of incremental snapshots according to a second predefined interval that is an integer multiple of the first predefined interval. The method also includes receiving a request to restore the data volume to a point-in-time. The method further includes restoring the data volume to the point-in-time using none or some of the snapshots in the first series that were created at or prior to the point-in-time, and all of the snapshots in the second series that were created at or prior to the point-in-time.

Abstract: User data of different snapshots for the same virtual disk are stored in the same storage object. Similarly, metadata of different snapshots for the same virtual disk are stored in the same storage object, and log data of different snapshots for the same virtual disk are stored in the same storage object. As a result, the number of different storage objects that are managed for snapshots do not increase proportionally with the number of snapshots taken. In addition, any one of the multitude of persistent storage back-ends can be selected as the storage back-end for the storage objects according to user preference, system requirement, snapshot policy, or any other criteria. Another advantage is that the storage location of the read data can be obtained with a single read of the metadata storage object, instead of traversing metadata files of multiple snapshots.

Abstract: In a computer system with a disk array that has physical storage devices arranged as logical storage units and is capable of carrying out hardware storage operations on a per logical storage unit basis, data movement operations can be carried out on a per-file basis. A data mover software component for use in a computer or storage system enables cloning and initialization of data to provide high data throughput without moving the data between the kernel and application levels.

Abstract: Techniques for using a host-side cache to accelerate virtual machine (VM) I/O are provided. In one embodiment, the hypervisor of a host system can intercept an I/O request from a VM running on the host system, where the I/O request is directed to a virtual disk residing on a shared storage device. The hypervisor can then process the I/O request by accessing a host-side cache that resides one or more cache devices distinct from the shared storage device, where the accessing of the host-side cache is transparent to the VM.

Abstract: In a computer system with a disk array that has physical storage devices arranged as logical storage units and is capable of carrying out hardware storage operations on a per logical storage unit basis, data movement operations can be carried out on a per-file basis. A data mover software component for use in a computer or storage system enables cloning and initialization of data to provide high data throughput without moving the data between the kernel and application levels.

Abstract: Techniques for using a cache to accelerate virtual machine (VM) I/O are provided. In one embodiment, a host system can intercept an I/O request from a VM running on the host system, where the I/O request is directed to a virtual disk residing on a shared storage device. The host system can then process the I/O request by accessing a cache that resides on one or more cache devices directly attached to the host system, where the accessing of the cache is transparent to the VM.

Abstract: Multiple servers sharing a distributed file system are used to perform copies of regions of a source file in parallel from a source storage unit to corresponding temporary files at a destination storage unit. These temporary files are then merged or combined into a single file at the destination storage unit in a way that preserves the inode structure and attributes of the source file. A substantial speedup is obtained by copying regions of the file in parallel.

Abstract: Techniques for dynamically managing the placement of blocks of a logical file between a flash storage tier and an HDD storage tier are provided. In one embodiment, a computer system can collect I/O statistics pertaining to the logical file, where a first subset of blocks of the logical file are stored on the flash storage tier and where a second subset of blocks of the logical file are stored on the HDD storage tier. The computer system can further generate a heat map for the logical file based on the I/O statistics, where the heat map indicates, for each block of the logical file, the number of times the block has been accessed. The computer system can then identify, using the heat map, one or more blocks of the logical file as being performance-critical blocks, and can move data between the flash and HDD storage tiers such that the performance-critical blocks are placed on the flash storage tier.

Abstract: Techniques for utilizing flash storage as an extension of hard disk (HDD) storage are provided. In one embodiment, a computer system stores a subset of blocks of a logical file in a first physical file, associated with a first data structure that represents a filesystem object, on flash storage and a subset of blocks, associated with a second data structure that represents a filesystem object comprising tiering configuration information that includes an identifier of the first physical file, in a second physical file on HDD storage. The computer system processes an I/O request directed to the logical file by directing it to either the physical file on the flash storage or the HDD storage by verifying that the tiering configuration information exists in the data structure and determining whether the one or more blocks are part of the first subset of blocks or the second subset of blocks.

Abstract: Techniques for surfacing host-side flash storage capacity to a plurality of VMs running on a host system are provided. In one embodiment, the host system creates, for each VM in the plurality of VMs, a flash storage space allocation in a flash storage device that is locally attached to the host system. The host system then causes the flash storage space allocation to be readable and writable by the VM as a virtual flash memory device.

Abstract: Techniques for surfacing host-side flash storage capacity to a plurality of VMs running on a host system are provided. In one embodiment, the host system creates, for each VM in the plurality of VMs, a flash storage space allocation in a flash storage device that is locally attached to the host system. The host system then causes the flash storage space allocation to be readable and writable by the VM as a virtual flash memory device.

Abstract: Techniques for utilizing flash storage as an extension of hard disk (HDD) based storage are provided. In one embodiment, a computer system can store a first subset of blocks of a logical file in a first physical file residing on a flash storage tier, and a second subset of blocks of the logical file in a second physical file residing on an HDD storage tier. The computer system can then receive an I/O request directed to one or more blocks of the logical file and process the I/O request by accessing the flash storage tier or the HDD storage tier, the accessing being based on whether the one or more blocks are part of the first subset of blocks stored in the first physical file.

Abstract: A method of acquiring a lock by a node, on a shared resource in a system of a plurality of interconnected nodes, is disclosed. Each node that competes for a lock on the shared resource maintains a list of locks currently owned by the node. A lock metadata is maintained on a shared storage that is accessible to all nodes that may compete for locks on shared resources. A heartbeat region is maintained on a shared resource corresponding to each node so nodes can register their liveness. A lock state is maintained in the lock metadata in the shared storage. A lock state may indicate lock held exclusively, lock free or lock in managed mode. If the lock is held in the managed mode, the ownership of the lock can be transferred to another node without a use of a mutual exclusion primitive such as the SCSI reservation.

Abstract: A network-based method for managing locks in a shared file system (SFS) for a group of hosts that does not require any configuration to identify a server for managing locks for the SFS. Each host in the group carries out the steps of checking a predetermined storage location to determine whether there is a host ID written in the predetermined location. If there is no host ID written in the predetermined location, the first host to notice this condition writes its host ID in the predetermined location to identify itself as the server for managing locks. If there is a host ID written in the predetermined location, the host ID of the server for managing locks is maintained in local memory. When the host needs to perform IO operations on a file of the SFS, it communicates with the server for managing locks over the network using the host ID of the server for managing locks to obtain a lock to the file.

Abstract: A network-based method for managing locks in a shared file system (SFS) for a group of hosts that does not require any configuration to identify a server for managing locks for the SFS. Each host in the group carries out the steps of checking a predetermined storage location to determine whether there is a host ID written in the predetermined location. If there is no host ID written in the predetermined location, the first host to notice this condition writes its host ID in the predetermined location to identify itself as the server for managing locks. If there is a host ID written in the predetermined location, the host ID of the server for managing locks is maintained in local memory. When the host needs to perform IO operations on a file of the SFS, it communicates with the server for managing locks over the network using the host ID of the server for managing locks to obtain a lock to the file.

Abstract: Techniques for utilizing flash storage as an extension of hard disk (HDD) based storage are provided. In one embodiment, a computer system can store a first subset of blocks of a logical file in a first physical file residing on a flash storage tier, and a second subset of blocks of the logical file in a second physical file residing on an HDD storage tier. The computer system can then receive an I/O request directed to one or more blocks of the logical file and process the I/O request by accessing the flash storage tier or the HDD storage tier, the accessing being based on whether the one or more blocks are part of the first subset of blocks stored in the first physical file.

Abstract: Techniques for dynamically managing the placement of blocks of a logical file between a flash storage tier and an HDD storage tier are provided. In one embodiment, a computer system can collect I/O statistics pertaining to the logical file, where a first subset of blocks of the logical file are stored on the flash storage tier and where a second subset of blocks of the logical file are stored on the HDD storage tier. The computer system can further generate a heat map for the logical file based on the I/O statistics, where the heat map indicates, for each block of the logical file, the number of times the block has been accessed. The computer system can then identify, using the heat map, one or more blocks of the logical file as being performance-critical blocks, and can move data between the flash and HDD storage tiers such that the performance-critical blocks are placed on the flash storage tier.

Abstract: Multiple servers sharing a distributed file system are used to perform copies of regions of a source file in parallel from a source storage unit to corresponding temporary files at a destination storage unit. These temporary files are then merged or combined into a single file at the destination storage unit in a way that preserves the inode structure and attributes of the source file. A substantial speedup is obtained by copying regions of the file in parallel.