Abstract: A distributed storage system that dispatches an input/output request is described. In an exemplary embodiment, a storage controller client receives the input/output request, wherein the distributed storage system includes the storage controller client, a plurality of storage controller servers, a plurality of virtual nodes distributed among a plurality of physical nodes, and each of the plurality of physical nodes is hosted on one of the plurality of storage controller servers. The storage controller client further computes a target virtual node for the input/output request, where the target virtual node is one of the plurality of virtual nodes. Using the computed target virtual node, the storage controller client determines a target physical node that corresponds to the target virtual node, where the target physical node is one of the plurality of physical nodes.

Abstract: A distributed multi-layer cache in a distributed storage system is described, where the storage controller functions of the distributed storage system are separated from that of distributed storage system storage media. In an exemplary embodiment, a storage controller server determines if an object is in a cache that stores a plurality of objects. In addition, the distributed storage system includes the cache and a distributed object layer for persistently storing the plurality of objects. The cache further includes a de-duplicated cache layer. The storage controller server accesses the object from the cache if the object is in the cache and accesses the object from the distributed object layer if the object is not in the cache.

Abstract: A distributed storage system that dispatches an input/output request is described. In an exemplary embodiment, a storage controller client receives the input/output request, wherein the distributed storage system includes the storage controller client, a plurality of storage controller servers, a plurality of virtual nodes distributed among a plurality of physical nodes, and each of the plurality of physical nodes is hosted on one of the plurality of storage controller servers. The storage controller client further computes a target virtual node for the input/output request, where the target virtual node is one of the plurality of virtual nodes. Using the computed target virtual node, the storage controller client determines a target physical node that corresponds to the target virtual node, where the target physical node is one of the plurality of physical nodes.

Abstract: A distributed garbage collection in a distributed storage system is described, where the storage controller functions of the distributed storage system are separated from that of distributed storage system storage media. In an exemplary embodiment, a storage controller server generates a live object map of live objects stored on the distributed storage system in a plurality of block segments distributed across a plurality of storage controller servers. The storage controller server further scans the plurality of block segments to generate segment summary statistics, where the segment summary statistics indicates the number of live objects stored in the plurality of block segments. In addition, the storage controller server compacts each of the plurality of block segments that have a low utilization based on the segment summary statistics. Furthermore, the live object map is a probabilistic data structure storing a list of valid objects.

Abstract: I/O bandwidth reduction using storage-level common page information is implemented by a storage server, in response to receiving a request from a client for a page stored at a first virtual address, determining that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address or that the first and second virtual addresses map to a deduplicated page within a storage system, and transmitting metadata to the client mapping the first virtual address to a second virtual address that also maps to the deduplicated page. For one embodiment, the metadata is transmitted in anticipation of a request for the redundant/deduplicated page via the second virtual address. For an alternate embodiment, the metadata is sent in response to a determination that a page that maps to the second virtual address was previously sent to the client.

Abstract: An alignment data structure is used to map a logical data block start address to a physical data block start address dynamically, to service a client data access request. A separate alignment data structure can be provided for each data object managed by the storage system. Each such alignment data structure can be stored in, or referenced by a pointer in, the inode of the corresponding data object. A consequence of the mapping is that certain physical storage medium regions are not mapped to any logical data blocks. These unmapped regions may be visible only to the file system layer and layers that reside between the file system layer and the mass storage subsystem. They can be used, if desired, to store system information, i.e., information that is not visible to any storage client.

Abstract: A distributed storage system that performs automated load balancing is described. In an exemplary embodiment, a storage controller server determines if there is duplicative data in a distributed storage system. In this embodiment, the storage controller server detects a load balancing event in the distributed storage system, where the distributed storage system includes a plurality of virtual nodes distributed across a plurality of physical nodes. In response to detecting the load balancing event, the storage controller server determines that a current virtual node is to move from a source physical node to a destination physical node. In addition, the current virtual node is one of the plurality of virtual nodes and the source and destination physical nodes are in the plurality of physical nodes. The storage controller server further moves the current virtual node from the source physical node to the destination physical node.

Abstract: One or more techniques and/or systems are provided for describing virtual machine dependencies. In particular, data objects, such as virtual hard drives, associated with virtual machines may be identified and/or examined to identify data structures, such as configuration files, comprising configuration data. The configuration data may be analyzed to determine dependency relationships between virtual machines to describe virtual machine dependencies. Identifying virtual machine dependencies, among other things, allows virtual machines that are no longer used to be repurposed, deleted, reset, etc. with little to no adverse effect on other virtual machines.

Abstract: I/O bandwidth reduction using storage-level common page information is implemented by a storage server. In response to receiving a request from a client for a page stored at a first virtual address, the storage server determines that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address. Or the storage server determines that the first and second virtual addresses map to a deduplicated page within a storage system. The storage server then transmits metadata to the client. The metadata maps the first virtual address to a second virtual address that also maps to the deduplicated page.

Abstract: Techniques for fast provisioning of virtual machine images using deduplication metadata are described, including receiving a request to copy a first virtual machine to form a second virtual machine, identifying a first portion of memory comprising data for the first virtual machine; and forming the second virtual machine based on the first portion of memory comprising data for the first virtual machine, wherein forming the second virtual machine further comprises linking the second virtual machine to the first portion of memory comprising data for the first virtual machine, and implementing a second portion of memory to store data for the second virtual machine independent of the first virtual machine.

Abstract: One or more techniques and/or systems are provided for hosting a virtual machine from a snapshot. In particular, a snapshot of a virtual machine hosted on a primary computing device may be created. The virtual machine may be hosted on a secondary computing device using the snapshot, for example, when a failure of the virtual machine on the primary computing device occurs. If a virtual machine type (format) of the snapshot is not supported by the secondary computing device, then the virtual machine within the snapshot may be converted to a virtual machine type supported by the secondary computing device. In this way, the virtual machine may be operable and/or accessible on the secondary computing device despite the failure. Hosting the virtual machine on the secondary computing device provides, among other things, fault tolerance for the virtual machine and/or applications comprised therein.

Abstract: One or more techniques and/or systems are provided for hosting a virtual machine from a snapshot. In particular, a snapshot of a virtual machine hosted on a primary computing device may be created. The virtual machine may be hosted on a secondary computing device using the snapshot, for example, when a failure of the virtual machine on the primary computing device occurs. If a virtual machine type (format) of the snapshot is not supported by the secondary computing device, then the virtual machine within the snapshot may be converted to a virtual machine type supported by the secondary computing device. In this way, the virtual machine may be operable and/or accessible on the secondary computing device despite the failure. Hosting the virtual machine on the secondary computing device provides, among other things, fault tolerance for the virtual machine and/or applications comprised therein.

Abstract: Techniques for deduplication of virtual machine files in a virtualized desktop environment are described, including receiving data into a page cache, the data being received from a virtual machine and indicating a write operation, and deduplicating the data in the page cache prior to committing the data to storage, the data being deduplicated in-band and in substantially real-time.

Abstract: One or more techniques and/or systems are provided for describing virtual machine dependencies. In particular, data objects, such as virtual hard drives, associated with virtual machines may be identified and/or examined to identify data structures, such as configuration files, comprising configuration data. The configuration data may be analyzed to determine dependency relationships between virtual machines to describe virtual machine dependencies. Identifying virtual machine dependencies, among other things, allows virtual machines that are no longer used to be repurposed, deleted, reset, etc. with little to no adverse effect on other virtual machines.

Abstract: Methods and system for securely capturing workloads at a live network for replaying at a test network. The disclosed system captures file system states and workloads of a live server at the live network. In one embodiment the captured data is anonymized to protect confidentiality of the data. A file system of a test server at the test network is mirrored from a captured state of the live server. An anonymized version of the captured workloads is replayed as a request to the test server. A lost or incomplete command is recreated from the states of the live server. An order of the commands during replay can be based on an order in the captured workload, or based on a causal relationship. Performance characteristics of the live network are determined based on the response to the replayed command.

Abstract: Techniques for fast provisioning of virtual machine images using deduplication metadata are described, including receiving a request to copy a first virtual machine to form a second virtual machine, identifying a first portion of memory comprising data for the first virtual machine; and forming the second virtual machine based on the first portion of memory comprising data for the first virtual machine, wherein forming the second virtual machine further comprises linking the second virtual machine to the first portion of memory comprising data for the first virtual machine, and implementing a second portion of memory to store data for the second virtual machine independent of the first virtual machine.

Abstract: An alignment data structure is used to map a logical data block start address to a physical data block start address dynamically, to service a client data access request. A separate alignment data structure can be provided for each data object managed by the storage system. Each such alignment data structure can be stored in, or referenced by a pointer in, the inode of the corresponding data object. A consequence of the mapping is that certain physical storage medium regions are not mapped to any logical data blocks. These unmapped regions may be visible only to the file system layer and layers that reside between the file system layer and the mass storage subsystem. They can be used, if desired, to store system information, i.e., information that is not visible to any storage client.

Abstract: Methods and system for securely capturing workloads at a live network for replaying at a test network. The disclosed system captures file system states and workloads of a live server at the live network. In one embodiment the captured data is anonymized to protect confidentiality of the data. A file system of a test server at the test network is mirrored from a captured state of the live server. An anonymized version of the captured workloads is replayed as a request to the test server. A lost or incomplete command is recreated from the states of the live server. An order of the commands during replay can be based on an order in the captured workload, or based on a causal relationship. Performance characteristics of the live network are determined based on the response to the replayed command.

Abstract: One or more techniques and/or systems are provided for hosting a virtual machine from a snapshot. In particular, a snapshot of a virtual machine hosted on a primary computing device may be created. The virtual machine may be hosted on a secondary computing device using the snapshot, for example, when a failure of the virtual machine on the primary computing device occurs. If a virtual machine type (format) of the snapshot is not supported by the secondary computing device, then the virtual machine within the snapshot may be converted to a virtual machine type supported by the secondary computing device. In this way, the virtual machine may be operable and/or accessible on the secondary computing device despite the failure. Hosting the virtual machine on the secondary computing device provides, among other things, fault tolerance for the virtual machine and/or applications comprised therein.

Abstract: One or more techniques and/or systems are provided for hosting a virtual machine from a snapshot. In particular, a snapshot of a virtual machine hosted on a primary computing device may be created. The virtual machine may be hosted on a secondary computing device using the snapshot, for example, when a failure of the virtual machine on the primary computing device occurs. If a virtual machine type (format) of the snapshot is not supported by the secondary computing device, then the virtual machine within the snapshot may be converted to a virtual machine type supported by the secondary computing device. In this way, the virtual machine may be operable and/or accessible on the secondary computing device despite the failure. Hosting the virtual machine on the secondary computing device provides, among other things, fault tolerance for the virtual machine and/or applications comprised therein.