Abstract: Reducing buffer space that is reserved when a VM attempts to write to disk. A shared memory maintains stored information available to all VM's, allowing translation between accessible to RSD's, independent of how that VM's storage is divided into VSD's. When a guest OS writes to a VSD, an “extent”, designating a location in the shared memory and amount of data to be transferred, which can be passed directly from the guest OS, through each element involved in translating the operation into one or more operations on RSD's, keeping copying to a minimum. No substantial changes need be made to the guest OS.

Abstract: Reliably testing distributed systems in which multiple components interact, including: provisioning resources by a test leader, directing components to perform actions required by test suites, and reporting results. Provisioning resources might be batched in advance, and might overlap resources for multiple test suites. Provisioning and directing might occur on an as-needed basis, might be performed concurrently, and might be reordered for relatively better performance. Directing actions might occur on command by test leaders, might occur by synchronization of resources, or might occur by resources requesting permission to proceed. Resources might act only upon direction by test leaders, or might act on their own initiative when synchronized. Results might be reported to the test leader for comparison, maintained in a database for access by the test leader, or compared by resources on their own initiative in response to instructions from the test leader.

Abstract: In a reliable distributed computing system, techniques where user actions or events affecting real system devices (RSD's), or causing the system to make changes to metadata, require fewer accesses to RSD's. When a virtual system device (VSD) is mounted, substantially all the metadata, or at least that associated with data recovery, is read into memory. Changes to that metadata can be made substantially without any metadata accesses to RSD's. No metadata accesses to RSD's are made for read operations, as the RSD would remain unchanged. Metadata is flushed from memory upon write operations. Metadata about data locations can be maintained in memory, so as to be available if any RSD becomes corrupted or otherwise comes to lose data.

Abstract: Providing differing degrees of reliability or security for distinct devices within a reliable distributed system. Allowing virtual machines to operate in a reliable distributed system without either exposing the actual available devices to guest operating systems, or requiring the guest OS to maintain, or to maintain information about, reliability or security. Methods are responsive to a hierarchy of possible reliability or security failures, to guard more carefully against simultaneous failures of reliability or breaches of security, without additional work or delay. Functions invoked by the user that relate to reliability and security are hidden, so the user can specify a degree of reliability or security without having to implement those requirements themselves. Providing a uniform set of resources available to all users, separating out those individual resources that are allocated to particular users, and emulating particular devices at the request of those particular users.

Abstract: Assigning virtual machines to physical devices in a cluster, without need for substantial operator decision-making or intervention. An operator console, coupled to the cluster, receives information from those physical devices about resource use by virtual machines. Each physical device reports virtual machines assigned thereto, and their resource usage. The console presents information regarding resource use by virtual machines, and presents a control panel to the operator, allowing manipulation of virtual machines and physical devices: starting or stopping virtual machines, moving virtual machines, adding or removing physical devices. The operator can create a new virtual machine, or can restart a paused or stopped virtual machine.

Abstract: A system, method, and computer program product are provided for implementing a protocol stack for handling memory access requests associated with one or more block devices. The method includes the steps of allocating a thread to process a task associated with a memory access request for a block device; and executing the thread to process the task, wherein the thread implements a state machine that manages calls to two or more layers of the protocol stack. A task dispatcher is implemented for managing the execution of tasks using at least one thread pool, the tasks managing the calls to the various layers of the protocol stack. The protocol stack may implement abstraction layers for virtual storage devices as well as the real block devices.

Abstract: Techniques that can minimize latency during remote access between nodes in a reliable distributed computing system. A local computing device includes a protocol data unit that interacts with a substantially similar unit at a remote computing device. Each unit executes a finite state machine that is designed to minimize both the amount of wait time between messages and the number of round-trip message exchanges. The state machine resolves each request as quickly as possible to one of three states: SUCCESS, in which the transfer is successful and it is feasible to free the data transfer buffer, COMPLETE, in which the transfer is finished and a next transfer can be initiated, ABORT, in which either the sending or receiving unit declares the transfer to have failed (or aborted by a higher level element).

Abstract: Reducing buffer space that is reserved when a VM attempts to write to disk. A shared memory maintains stored information available to all VM's, allowing translation between accessible to RSD's, independent of how that VM's storage is divided into VSD's. When a guest OS writes to a VSD, an “extent”, designating a location in the shared memory and amount of data to be transferred, which can be passed directly from the guest OS, through each element involved in translating the operation into one or more operations on RSD's, keeping copying to a minimum. No substantial changes need be made to the guest OS.

Abstract: Presenting information to operators of a cluster of physical devices, regarding resources being used by physical machines assigned to physical devices in those clusters. An operator console, coupled to the cluster of physical devices, receives information from those physical devices with respect to use of resources by virtual machines assigned to those physical devices. Each physical device reports virtual machines assigned thereto, and an physical memory assigned to each one. The operator console presents information to the operator regarding use of resources by virtual machines, in a natural and visible way. Physical devices are disposed horizontally to indicate a selected device, and vertically to indicate allocated resources. Virtual machines are disposed with each virtual machine occupying a vertical amount proportional to allocated resources. The graphical interface allows the operator to reallocate virtual machines to physical devices, or reallocate resources to virtual machines.

Abstract: Managing resources in a VM cluster; allocation of resources among competing VMs. Assigning VMs to real devices, responsive to needs for resources: processor usage, disk I/O, network I/O, memory space, disk space. Assigning VMs responsive to needs for cluster activity: network response latency, QoS. Using predictive models of resource usage by VMs. Transferring VMs, improving utilization. Failing-soft onto alternative VM resources. Providing a resource buffer for collective VM resource demand. Transferring VMs to a cloud service that charges for resources.

Abstract: In a reliable distributed computing system, techniques where user actions or events affecting real system devices (RSD's), or causing the system to make changes to metadata, require fewer accesses to RSD's. When a virtual system device (VSD) is mounted, substantially all the metadata, or at least that associated with data recovery, is read into memory. Changes to that metadata can be made substantially without any metadata accesses to RSD's. No metadata accesses to RSD's are made for read operations, as the RSD would remain unchanged. Metadata is flushed from memory upon write operations. Metadata about data locations can be maintained in memory, so as to be available if any RSD becomes corrupted or otherwise comes to lose data.

Abstract: A system, method, and computer program product are provided for implementing a reliable placement engine for a block device. The method includes the steps of tracking one or more parameters associated with a plurality of real storage devices (RSDs), generating a plurality of RSD objects in a memory associated with a first node, generating a virtual storage device (VSD) object in the memory, and selecting one or more RSD objects in the plurality of RSD objects based on the one or more parameters. Each RSD object corresponds to a particular RSD in the plurality of RSDs. The method also includes the step of, for each RSD object in the one or more RSD objects, allocating a block of memory in the RSD associated with the RSD object to store data corresponding to a first block of memory associated with the VSD object.

Abstract: Real system devices (RSD's) emulate virtual system devices (VSD's); system elements can use RSD's and manipulate VSD's, while users can only use VSD's. Identifying when RSD's have erroneous data. Providing a mapping of RSD's to VSD's, to recover a maximum set of VSD's. Providing a mapping of VSD's to RSD's, to recover a specified most valuable VSD. Reconstructing VSD's from RSD's having erroneous data, and coordinating with a user's assistance to couple RSD's in an order allowing data recovery. Metadata is maintained coupling data identifiable on VSD's with corresponding data on RSD's; it is possible to trace connections between RSD's and VSD's even without complete data.

Abstract: A system, method, and computer program product are provided for performing fast migration of a virtual resource from one node to another node. The method includes the steps of receiving a first request to migrate a resource from a first node to a second node, transmitting a second request to the second node to create a new instance of the resource, collecting a set of changes associated with the resource in a data structure, and transmitting the data structure that includes the set of changes to the second node. The second node generates the new instance of the resource based on a snapshot of the resource captured by the first node at a previous point in time and updates the new instance of the resource based on the set of changes such that the new instance of the resource on the second node matches the resource on the first node.

Abstract: A system, method, and computer program product are provided for implementing shared reference counters among a plurality of virtual storage devices. The method includes the steps of allocating a first portion of a real storage device to store data, wherein the first portion is divided into a plurality of blocks of memory and allocating a second portion of the real storage device to store a plurality of reference counters that correspond to the plurality of blocks of memory. The reference counters may be updated by two or more virtual storage devices hosted in one or more nodes to manage the allocation of the blocks of memory in the real storage device.

Abstract: A system, method, and computer program product are provided for implementing a reliable placement engine for a block device. The method includes the steps of tracking one or more parameters associated with a plurality of real storage devices (RSDs), generating a plurality of RSD objects in a memory associated with a first node, generating a virtual storage device (VSD) object in the memory, and selecting one or more RSD objects in the plurality of RSD objects based on the one or more parameters. Each RSD object corresponds to a particular RSD in the plurality of RSDs. The method also includes the step of, for each RSD object in the one or more RSD objects, allocating a block of memory in the RSD associated with the RSD object to store data corresponding to a first block of memory associated with the VSD object.

Abstract: A system, method, and computer program product are provided for implementing a reliable placement engine for a block device. The method includes the steps of tracking one or more parameters associated with a plurality of real storage devices (RSDs), generating a plurality of RSD objects in a memory associated with a first node, generating a virtual storage device (VSD) object in the memory, and selecting one or more RSD objects in the plurality of RSD objects based on the one or more parameters. Each RSD object corresponds to a particular RSD in the plurality of RSDs. The method also includes the step of, for each RSD object in the one or more RSD objects, allocating a block of memory in the RSD associated with the RSD object to store data corresponding to a first block of memory associated with the VSD object.

Abstract: Reliably testing distributed systems in which multiple components interact, including: provisioning resources by a test leader, directing components to perform actions required by test suites, and reporting results. Provisioning resources might be batched in advance, and might overlap resources for multiple test suites. Provisioning and directing might occur on an as-needed basis, might be performed concurrently, and might be reordered for relatively better performance. Directing actions might occur on command by test leaders, might occur by synchronization of resources, or might occur by resources requesting permission to proceed. Resources might act only upon direction by test leaders, or might act on their own initiative when synchronized. Results might be reported to the test leader for comparison, maintained in a database for access by the test leader, or compared by resources on their own initiative in response to instructions from the test leader.

Abstract: Virtual machine localization in a distributed fault-tolerant system. Network traffic is minimized between nodes, while maintaining fault-tolerant behavior. Communication is measured between VM's, which are moved to minimize real network use. An availability group defines which resources cannot safely be disposed on a single device. When two VM's are in the same availability group, they are disposed as near as practical, but not on the same device. VM's and other resources they use are also disposed to minimize real network use. Minimizing real network use includes minimizing both channels occupied and communication distance. Availability groups can include both “hard” availability groups, for which data could be irretrievably lost, or “soft” availability groups, for which data could be lost, but could still be recovered with difficulty. Resource localization responds to both distance between VM's and their resources, and risk associated with losing data.

Abstract: Reliably making configuration changes to distributed systems, including receiving commands for multiple configuration changes, subdividing configuration changes into separate tasks, and performing those tasks at each node. A configuration element receives sets of configuration change commands, acknowledging them so the user need not wait before issuing additional commands. Tasks are determined, each including consistent changes to system configuration, and each including single-device tasklets. Each particular tasklet might be assigned to a particular single device, or to any single device in the system. Next tasks are performed when tasklets are complete. If tasklets are not timely performed due to nodes which are relatively unresponsive, those nodes are marked “failed.” When a failed node returns to responsiveness, it marks itself “recovering.” When a recovering node catches up, it marks itself “operational.” Updates by failed or recovering nodes are skipped while synchronizing with operational nodes.