Abstract: A technique improves storage efficiency of an object store configured to maintain numerous snapshots for long-term storage in an archival storage system by efficiently determining data that is exclusively owned by an expiring snapshot to allow deletion of the expiring snapshot from the object store. The technique involves managing index data structures to enable efficient garbage collection across a very large number of data objects. When a snapshot expires, the technique obviates the need to scan the numerous snapshot data objects to determine which index structures are no longer needed and can be reclaimed (garbage collected). The technique is directed to management of underlying storage based on different sets of policies. When certain snapshots expire and are ready for deletion, the technique is directed to finding those data blocks that are no longer referenced (used) by any valid snapshots.

Abstract: A technique improves storage efficiency of an object store configured to maintain numerous snapshots for long-term storage in an archival storage system by efficiently determining data that is exclusively owned by an expiring snapshot to allow deletion of the expiring snapshot from the object store. The technique involves managing index data structures to enable efficient garbage collection across a very large number of data objects. When a snapshot expires, the technique obviates the need to scan the numerous snapshot data objects to determine which index structures are no longer needed and can be reclaimed (garbage collected). The technique is directed to management of underlying storage based on different sets of policies. When certain snapshots expire and are ready for deletion, the technique is directed to finding those data blocks that are no longer referenced (used) by any valid snapshots.

Abstract: An indexing technique provides an index data structure for efficient retrieval of a snapshot from a long-term storage service (LTSS) of an archival storage system. The snapshot is generated from typed data of a logical entity, such as a virtual disk (vdisk). The data of the snapshot is replicated to a frontend data service of the LTSS sequentially and organized as one or more data objects for storage by a backend data service of LTSS in an object store of the archival storage system. Metadata associated with the snapshot (i.e., snapshot metadata) is recorded as a log and persistently stored on storage media local to the frontend data service. The snapshot metadata includes information describing the snapshot data, e.g., a logical offset range of a snapshot of the vdisk and, thus, is used to construct the index data structure. Notably, construction of the index data structure is deferred until after the entirety of the snapshot data has been replicated and received by the frontend data service.

Abstract: In one embodiment, a computing device accesses an entity database storing information about entities in a virtualization environment. Each of the entities is associated with a single entity type and a shard. The shard further corresponds to the single entity type. The computing device determines an entity trail for the single shard, based on a modification history for one or more of the entities associated with the shard. The entity trail is divided into a plurality of time segments. The computing device receives an update from a client device indicating an update to an entity that is associated with the shard, and a client timestamp. The computing device selects a time segment corresponding to the client timestamp, and updates that portion of the entity trail with the client update.

Abstract: In particular embodiments, a system for managing a virtualization environment includes host machines, each of the host machines including a hypervisor, user virtual machines (UVMs) and a virtual machine controller. The virtualization environment also includes virtual disks comprising a plurality of storage devices, and being accessible by the virtual machine controllers. The virtual machine controllers conduct I/O transactions with the virtual disks. The system stores an entity-relationship graph representing elements in the virtualization environment. Each of the elements is represented by an entity-type node in the entity-relationship graph, and relationships between the elements are represented by edges between the nodes.

Abstract: In particular embodiments, a system for managing a virtualization environment includes host machines, each of the host machines including a hypervisor, user virtual machines (UVMs) and a virtual machine controller. The virtualization environment also includes virtual disks comprising a plurality of storage devices, and being accessible by the virtual machine controllers. The virtual machine controllers conduct I/O transactions with the virtual disks. The system stores an entity-relationship graph representing elements in the virtualization environment. Each of the elements is represented by an entity-type node in the entity-relationship graph, and relationships between the elements are represented by edges between the nodes.

Abstract: In particular embodiments, a system for managing a virtualization environment includes host machines, each of the host machines including a hypervisor, user virtual machines (UVMs) and a virtual machine controller. The virtualization environment also includes virtual disks comprising a plurality of storage devices, and being accessible by the virtual machine controllers. The virtual machine controllers conduct I/O transactions with the virtual disks. The system stores an entity-relationship graph representing elements in the virtualization environment. Each of the elements is represented by an entity-type node in the entity-relationship graph, and relationships between the elements are represented by edges between the nodes.

Abstract: In particular embodiments, a system for managing a virtualization environment includes host machines, each of the host machines including a hypervisor, user virtual machines (UVMs) and a virtual machine controller. The virtualization environment also includes virtual disks comprising a plurality of storage devices, and being accessible by the virtual machine controllers. The virtual machine controllers conduct I/O transactions with the virtual disks. The system stores an entity-relationship graph representing elements in the virtualization environment. Each of the elements is represented by an entity-type node in the entity-relationship graph, and relationships between the elements are represented by edges between the nodes.

Abstract: In particular embodiments, a system for managing a virtualization environment includes host machines, each of the host machines including a hypervisor, user virtual machines (UVMs) and a virtual machine controller. The virtualization environment also includes virtual disks comprising a plurality of storage devices, and being accessible by the virtual machine controllers. The virtual machine controllers conduct I/O transactions with the virtual disks. The system stores an entity-relationship graph representing elements in the virtualization environment. Each of the elements is represented by an entity-type node in the entity-relationship graph, and relationships between the elements are represented by edges between the nodes.

Abstract: In particular embodiments, a system for managing a virtualization environment includes host machines, each of the host machines including a hypervisor, user virtual machines (UVMs) and a virtual machine controller. The virtualization environment also includes virtual disks comprising a plurality of storage devices, and being accessible by the virtual machine controllers. The virtual machine controllers conduct I/O transactions with the virtual disks. The system stores an entity-relationship graph representing elements in the virtualization environment. Each of the elements is represented by an entity-type node in the entity-relationship graph, and relationships between the elements are represented by edges between the nodes.

Abstract: In one embodiment, a computing device accesses an entity database storing information about entities in a virtualization environment. Each of the entities is associated with a single entity type and a shard. The shard further corresponds to the single entity type. The computing device determines an entity trail for the single shard, based on a modification history for one or more of the entities associated with the shard. The entity trail is divided into a plurality of time segments. The computing device receives an update from a client device indicating an update to an entity that is associated with the shard, and a client timestamp. The computing device selects a time segment corresponding to the client timestamp, and updates that portion of the entity trail with the client update.

Abstract: In particular embodiments, a system for managing a virtualization environment includes host machines, each of the host machines including a hypervisor, user virtual machines (UVMs) and a virtual machine controller. The virtualization environment also includes virtual disks comprising a plurality of storage devices, and being accessible by the virtual machine controllers. The virtual machine controllers conduct I/O transactions with the virtual disks. The system stores an entity-relationship graph representing elements in the virtualization environment. Each of the elements is represented by an entity-type node in the entity-relationship graph, and relationships between the elements are represented by edges between the nodes.

Abstract: In particular embodiments, a system for managing a virtualization environment includes host machines, each of the host machines including a hypervisor, user virtual machines (UVMs) and a virtual machine controller. The virtualization environment also includes virtual disks comprising a plurality of storage devices, and being accessible by the virtual machine controllers. The virtual machine controllers conduct I/O transactions with the virtual disks. The system stores an entity-relationship graph representing elements in the virtualization environment. Each of the elements is represented by an entity-type node in the entity-relationship graph, and relationships between the elements are represented by edges between the nodes.

Abstract: The present invention extends to methods, systems, computer program products, and data structures for managing database snapshot storage. Spatial properties database transactions are used to manage database snapshot storage allocation and reclamation. When a read transaction and subsequent write transactions refer to disjoint portions of a database, the read transaction can continue to read portions of data directly from the database. When a read transaction and subsequent write transactions refer to intersecting portions of a database, the read transaction can be transitioned to “snapshot” mode. The write transaction can publish a database snapshot can configure the read transaction to read contested portions of data from the published snapshot.