Abstract: Various embodiments set forth techniques for maintaining replicas of a data set. The techniques include storing, by a first node, a first replica of the data set on the first node; storing, by a second node, a second replica of the data set on the second node; and performing, by the first node and based on first metadata associated with the first replica, a first data management activity on the first replica that changes a first format of the first replica so that the first format of the first replica is different from a second format of the second replica.

Abstract: Various embodiments set forth techniques for managing metadata for a vblock include dynamically normalizing and denormalizing vblock metadata associated with an extent. Vblock metadata associated with an extent is normalized when the extent is migrated to a different extent group by having the vblock metadata to a mapping between the extent identifier and an extent group identifier in a metadata map separate from the vblock metadata. Vblock metadata associated with an extent is denormalized whenever the number of vblock metadata associated with an extent drops below a threshold. Vblock metadata is denormalized by updating the vblock metadata to include a mapping to an extent group, based on a mapping of the extent to the extent group in a separate metadata map, and then removing the mapping in the separate metadata map.

Abstract: Methods, systems, and computer program products for importing a workload from an external system into a virtualization system. A virtual disk (vDisk) is created based on analysis of the data layout of the workload. The vDisk is sharded into a plurality of non-overlapping shards. Independently executing shard controllers are assigned to the non-overlapping shards. On an ongoing basis, the plurality of shard controllers manage respective shards of the plurality of shards. I/O activity on at least some individual ones of the plurality of shard controllers is monitored on an ongoing basis. Two or more shard controllers can be merged into a single shard controller when the I/O activity on the two or more shard controllers is measured to be below a threshold. A single shard controller can be split into two or more shard controllers when I/O activity on the single shard controller is measured to be above a threshold.

Abstract: Systems for low-latency data access in distributed computing systems. A method embodiment commences upon generating a first storage area in local storage of a first computing node. Access to the first storage area is provided through the first computing node. A second storage area is generated wherein the second storage area comprises a first set of metadata that comprises local storage device locations of at least some of the local storage areas of the first storage area. A set of physical access locations of the second storage area is stored to a database that manages updates to the second set of metadata pertaining to the second storage area. Accesses to the first storage area are accomplished by querying the database to retrieve a location of the second set of metadata, and then accessing the first storage area through one or more additional levels of metadata that are node-wise collocated.

Abstract: Various embodiments set forth techniques for maintaining replicas of a data set. The techniques include storing, by a first node, a first replica of the data set on the first node; storing, by a second node, a second replica of the data set on the second node; and performing, by the first node and based on first metadata associated with the first replica, a first data management activity on the first replica that changes a first format of the first replica so that the first format of the first replica is different from a second format of the second replica.

Abstract: Systems for low-latency data access in distributed computing systems. A method embodiment commences upon generating a first storage area in local storage of a first computing node. Access to the first storage area is provided through the first computing node. A second storage area is generated wherein the second storage area comprises a first set of metadata that comprises local storage device locations of at least some of the local storage areas of the first storage area. A set of physical access locations of the second storage area is stored to a database that manages updates to the second set of metadata pertaining to the second storage area. Accesses to the first storage area are accomplished by querying the database to retrieve a location of the second set of metadata, and then accessing the first storage area through one or more additional levels of metadata that are node-wise collocated.

Abstract: Examples of a distributed message service include a virtualized file system including a virtual disk configured to store messages for a message topic, and a broker logically allocated to the message topic. The broker is configured to cause a message directed to the message topic provided from a publisher to be stored at the virtual disk, and to route the message to a subscriber of the message topic that is registered with the broker. The distributed message service further includes an operating system configured to manage the virtualized file system and includes a message service configured to manage logical allocation of the broker and manage allocation of the virtual disk to the message topic.

Abstract: Systems for low-latency data access in distributed computing systems. A method embodiment commences upon generating a first storage area in local storage of a first computing node. Access to the first storage area is provided through the first computing node. A second storage area is generated wherein the second storage area comprises a first set of metadata that comprises local storage device locations of at least some of the local storage areas of the first storage area. A set of physical access locations of the second storage area is stored to a database that manages updates to the second set of metadata pertaining to the second storage area. Accesses to the first storage area are accomplished by querying the database retrieve a location of the second set of metadata, and then accessing the first storage area through one or more additional levels of metadata that are node-wise collocated.

Abstract: Systems for low-latency data access in distributed computing systems. A method embodiment commences upon generating a first storage area in local storage of a first computing node. Access to the first storage area is provided through the first computing node. A second storage area is generated wherein the second storage area comprises a first set of metadata that comprises local storage device locations of at least some of the local storage areas of the first storage area. A set of physical access locations of the second storage area is stored to a database that manages updates to the second set of metadata pertaining to the second storage area. Accesses to the first storage area are accomplished by querying the database retrieve a location of the second set of metadata, and then accessing the first storage area through one or more additional levels of metadata that are node-wise collocated.