Abstract: Techniques for adaptive replication factors for a computing cluster include one or more non-transitory computer-readable media storing program instructions that, when executed by one or more processors associated with a first node in an N-node cluster, cause the one or more processors to perform a method comprising setting a replication factor for the cluster to N, and in response to detecting a failure of a second node in the cluster, setting the replication factor for the cluster to N-1.

Abstract: Techniques for pre-processing metadata for faster background decisions include, in response to a metadata entry associated with storage of data in a cluster being modified, generating a summary entry corresponding to the metadata entry. The summary entry includes a subset of the information from the metadata entry. A data management action is performed based upon a scanning process that reads a plurality of summary entries corresponding to a plurality of metadata entries for data in the cluster.

Abstract: Techniques for storing data in a cluster include receiving, at a first node, a write transaction directed to a first data block of a first extent in an extent group and logging, at the first node, a tentative update in a tentative update journal for a first replica of the first data block. The techniques further include subsequent to logging the tentative update, forwarding the write transaction to a second node, determining that the first node restarted before the write transaction was committed on the first node, and rolling back the write transaction for the first replica.

Abstract: Techniques for storing metadata include receiving a write operation directed to at least one data block in a first location on a storage device, and storing write data associated with the write operation in a key-value store associated with the first location of the storage device. The techniques also include storing a first pending transaction record associated with the write operation in a pending transaction journal, wherein the pending transaction journal comprises a single storage page that further includes all other pending transaction records associated with other write operations associated with the storage device.

Abstract: One embodiment of the present invention provides a system that can manage access to a service from a cluster of computing nodes. An instance of the system can operate on a respective computing node. During operation, the system instance can identify an Internet Protocol (IP) address, which provides access to the service from the cluster of computing nodes to a client device at a client site. The system instance can select the computing node for hosting the IP address in the cluster of computing nodes based on a set of selection criteria. The selection is performed independently at the computing node. The system instance can then assign the IP address to the computing node. The assignment allows a request for the service to be directed to the computing node. Subsequently, the system instance can facilitate the service from the cluster of computing nodes based on the request.

Abstract: Methods, systems and computer program products for hibernating a computing cluster. The present disclosure describes techniques for hibernating and resuming nodes of a computing cluster and entire computing clusters including movement of data and metadata to and from a cloud-tier storage facility (e.g., a cloud disk(s)) in an efficient manner.

Abstract: One embodiment of the present invention provides a system that can manage access to a service from a cluster of computing nodes. An instance of the system can operate on a respective computing node. During operation, the system instance can identify an Internet Protocol (IP) address, which provides access to the service from the cluster of computing nodes to a client device at a client site. The system instance can select the computing node for hosting the IP address in the cluster of computing nodes based on a set of selection criteria. The selection is performed independently at the computing node. The system instance can then assign the IP address to the computing node. The assignment allows a request for the service to be directed to the computing node. Subsequently, the system instance can facilitate the service from the cluster of computing nodes based on the request.

Abstract: A technique improves implementation of an index for an operations log (oplog) that coalesces random write operations directed to a virtual disk (vdisk) stored on an extent store. The oplog temporarily caches data associated with the random write operations (i.e., write data) as well as metadata describing the write data. The metadata includes descriptors to the write data stored on virtual address regions, i.e., offset ranges, of the vdisk and are used to identify the offset ranges of write data for the vdisk that are cached in the oplog. To facilitate fast lookup operations of the offset ranges when determining whether write data is cached in the oplog, an oplog index provides a state of the latest data for offset ranges of the vdisk. The technique improves implementation of the oplog index by storing the oplog index in storage class memory, such as persistent memory, to obviate failure and subsequent recovery of the oplog index.

Abstract: Various embodiments set forth techniques for maintaining replicas of a data set. The techniques include storing, by a first node, a first replica of the data set on the first node; storing, by a second node, a second replica of the data set on the second node; and performing, by the first node and based on first metadata associated with the first replica, a first data management activity on the first replica that changes a first format of the first replica so that the first format of the first replica is different from a second format of the second replica.

Abstract: One embodiment of the present invention provides a system that can manage access to a service from a cluster of computing nodes. An instance of the system can operate on a respective computing node. During operation, the system instance can identify an Internet Protocol (IP) address, which provides access to the service from the cluster of computing nodes to a client device at a client site. The system instance can select the computing node for hosting the IP address in the cluster of computing nodes based on a set of selection criteria. The selection is performed independently at the computing node. The system instance can then assign the IP address to the computing node. The assignment allows a request for the service to be directed to the computing node. Subsequently, the system instance can facilitate the service from the cluster of computing nodes based on the request.

Abstract: Systems for high-performance distributed computing. The systems include techniques for managing data and metadata across multiple nodes. A method embodiment commences by storing data at a node using a first storage mechanism that is local to the node. A first set of metadata is configured to identify a storage location for the stored data. The first set of metadata is stored using the same first storage mechanism that is local to the node. For accessing the first set of metadata, a second set of metadata is configured to identify a storage location for the first set of metadata. The second set of metadata is stored using a second storage mechanism that comprises a distributed metadata storage facility that stores metadata across multiple storage locations having at least one of the multiple storage locations that is not local to the node that stores data and metadata using the first storage mechanism.

Abstract: Methods, systems, and computer program products for importing a workload from an external system into a virtualization system. A virtual disk (vDisk) is created based on analysis of the data layout of the workload. The vDisk is sharded into a plurality of non-overlapping shards. Independently executing shard controllers are assigned to the non-overlapping shards. On an ongoing basis, the plurality of shard controllers manage respective shards of the plurality of shards. I/O activity on at least some individual ones of the plurality of shard controllers is monitored on an ongoing basis. Two or more shard controllers can be merged into a single shard controller when the I/O activity on the two or more shard controllers is measured to be below a threshold. A single shard controller can be split into two or more shard controllers when I/O activity on the single shard controller is measured to be above a threshold.

Abstract: Various embodiments set forth techniques for transactional allocation and deallocation of blocks in a block store. A first technique includes sending a first request that causes a non-persistent allocation of a block. The first technique also includes adding a first entry in a log recording the allocation as tentative, sending a second request that causes persistence of the allocation, and adding a second entry in a log recording the allocation as finalized. A second technique includes adding a first entry in a log recording a deallocation of a block, sending a first request that causes the deallocation of the block and causes the block to be unavailable for reallocation in a non-persistent manner, adding a second entry in the log recording that the deallocation is finalized, and sending a second request that causes the block to be made available for reallocation.

Abstract: A technique replicates an index of an operations log (oplog) from a primary node to a secondary node of a cluster in the event of failure. The oplog functions as a staging area to coalesce random write operations directed to a virtual disk (vdisk) stored on a backend storage tier. The oplog temporarily caches write data as well as metadata describing the write data. The metadata includes descriptors to the write data corresponding to offset ranges of the vdisk and are used to identify ranges of write data for the vdisk that are cached in the oplog. To facilitate fast lookup operations of whether write data is cached in the oplog, an oplog index provides a state of the latest data for offset ranges of the vdisk that enables fast failover of metadata used to construct the oplog index in memory without downtime or significant metadata replay.

Abstract: Various embodiments set forth techniques for maintaining replicas of a data set. The techniques include storing, by a first node, a first replica of the data set on the first node; storing, by a second node, a second replica of the data set on the second node; and performing, by the first node and based on first metadata associated with the first replica, a first data management activity on the first replica that changes a first format of the first replica so that the first format of the first replica is different from a second format of the second replica.

Abstract: A technique improves implementation of an index for an operations log (oplog) that coalesces random write operations directed to a virtual disk (vdisk) stored on an extent store. The oplog temporarily caches data associated with the random write operations (i.e., write data) as well as metadata describing the write data. The metadata includes descriptors to the write data stored on virtual address regions, i.e., offset ranges, of the vdisk and are used to identify the offset ranges of write data for the vdisk that are cached in the oplog. To facilitate fast lookup operations of the offset ranges when determining whether write data is cached in the oplog, an oplog index provides a state of the latest data for offset ranges of the vdisk. The technique improves implementation of the oplog index by storing the oplog index in storage class memory, such as persistent memory, to obviate failure and subsequent recovery of the oplog index.

Abstract: A technique replicates an index of an operations log (oplog) from a primary node to a secondary node of a cluster in the event of a failure of the primary node. The oplog functions as a staging area to coalesce random write operations directed to a virtual disk (vdisk) stored on a backend storage tier organized as an extent store. The oplog temporarily caches data associated with the random write operations (i.e., write data) as well as metadata describing the write data. The metadata includes descriptors to the write data corresponding to virtual address regions, i.e., offset ranges, of the vdisk and are used to identify the offset ranges of write data for the vdisk that are cached in the oplog. To facilitate fast lookup operations of the offset ranges when determining whether write data io is cached in the oplog, an oplog index provides a state of the latest data for offset ranges of the vdisk.

Abstract: Various embodiments set forth techniques for transactional allocation and deallocation of blocks in a block store. A first technique includes sending a first request that causes a non-persistent allocation of a block. The first technique also includes adding a first entry in a log recording the allocation as tentative, sending a second request that causes persistence of the allocation, and adding a second entry in a log recording the allocation as finalized. A second technique includes adding a first entry in a log recording a deallocation of a block, sending a first request that causes the deallocation of the block and causes the block to be unavailable for reallocation in a non-persistent manner, adding a second entry in the log recording that the deallocation is finalized, and sending a second request that causes the block to be made available for reallocation.

Abstract: One embodiment of the present invention provides a system that can manage access to a service from a cluster of computing nodes. An instance of the system can operate on a respective computing node. During operation, the system instance can identify an Internet Protocol (IP) address, which provides access to the service from the cluster of computing nodes to a client device at a client site. The system instance can select the computing node for hosting the IP address in the cluster of computing nodes based on a set of selection criteria. The selection is performed independently at the computing node. The system instance can then assign the IP address to the computing node. The assignment allows a request for the service to be directed to the computing node. Subsequently, the system instance can facilitate the service from the cluster of computing nodes based on the request.

Abstract: Communication between VMs running in a clustered virtualization environment and a distributed storage system is managed. The distributed storage system consists of multiple storage/compute nodes. A hypervisor communicates with a controller in a local node on behalf of a VM. The controller will redirect a connection to a second controller on different node. The controller to which the connection is redirected is selected so as to distribute the load evenly across the cluster.