Abstract: Techniques for implementing IOMMU-based DMA tracking for enabling live migration of VMs that use passthrough physical devices are provided. In one set of embodiments, these techniques leverage an IOMMU feature known as dirty bit tracking which is available in most, if not all, modern IOMMU implementations. The use of this feature allows for the tracking of passthrough DMA in a manner that is device/vendor/driver agnostic, resulting in a solution that is universally applicable to all passthrough physical devices.

Abstract: Techniques for implementing IOMMU-based DMA tracking for enabling live migration of VMs that use passthrough physical devices are provided. In one set of embodiments, these techniques leverage an IOMMU feature known as dirty bit tracking which is available in most, if not all, modern IOMMU implementations. The use of this feature allows for the tracking of passthrough DMA in a manner that is device/vendor/driver agnostic, resulting in a solution that is universally applicable to all passthrough physical devices.

Abstract: A virtual machine (VM) has direct access to an I/O device having physical and virtual functions and a mailbox register, and includes a guest driver for controlling the virtual functions. The VM runs on system software that includes a physical driver for controlling the physical function (PF) and maintains VM page tables, which include an entry that references a memory space into which the mailbox register is mapped. The system software registers a callback function with the physical driver, which the physical driver invokes upon receiving a trigger for communication with the guest driver. In response, the system software alters the page tables so that access to the mailbox register causes a PF intercept, and the callback function handles the communication with the guest driver. After completion of the communication, the system software alters the page tables so that access to the mailbox register does not cause a PF intercept.

Abstract: A method of migrating a virtual machine having a virtual device that is backed by direct passthrough hardware, from a source host to a destination host, includes the steps of determining whether or not the destination host has direct passthrough hardware that can back the virtual device, and upon determining that the destination host has direct passthrough hardware that can back the virtual device, determining if a version of the direct passthrough hardware at the source host matches a version of the direct passthrough hardware at the destination host. If the versions do not match, the steps further include quiescing the virtual device, deleting data structures relating to the virtual device, and then migrating the virtual machine from the source host to the destination host. If the versions match, the virtual machine is migrated without quiescing the virtual device and without deleting the data structures relating to the virtual device.

Abstract: Mechanisms to protect the integrity of memory of a virtual machine are provided. The mechanisms involve utilizing certain capabilities of the hypervisor underlying the virtual machine to monitor writes to memory pages of the virtual machine. A guest integrity driver communicates with the hypervisor to request such functionality. Additional protections are provided for protecting the guest integrity driver and associated data, as well as for preventing use of these mechanisms by malicious software. These additional protections include an elevated execution mode, termed “integrity mode,” which can only be entered from a specified entry point, as well as protections on the memory pages that store the guest integrity driver and associated data.

Abstract: A method of migrating a virtual machine having a virtual device that is backed by direct passthrough hardware, from a source host to a destination host, includes the steps of determining whether or not the destination host has direct passthrough hardware that can back the virtual device, and upon determining that the destination host has direct passthrough hardware that can back the virtual device, determining if a version of the direct passthrough hardware at the source host matches a version of the direct passthrough hardware at the destination host. If the versions do not match, the steps further include quiescing the virtual device, deleting data structures relating to the virtual device, and then migrating the virtual machine from the source host to the destination host. If the versions match, the virtual machine is migrated without quiescing the virtual device and without deleting the data structures relating to the virtual device.

Abstract: A virtual machine (VM) has direct access to an I/O device having physical and virtual functions and a mailbox register, and includes a guest driver for controlling the virtual functions. The VM runs on system software that includes a physical driver for controlling the physical function (PF) and maintains VM page tables, which include an entry that references a memory space into which the mailbox register is mapped. The system software registers a callback function with the physical driver, which the physical driver invokes upon receiving a trigger for communication with the guest driver. In response, the system software alters the page tables so that access to the mailbox register causes a PF intercept, and the callback function handles the communication with the guest driver. After completion of the communication, the system software alters the page tables so that access to the mailbox register does not cause a PF intercept.

Abstract: Techniques for enabling live migration of VMs with passthrough PCI devices are provided. In one set of embodiments, a hypervisor of a host system can create a copy of a DMA buffer used by a VM of the host system and a passthrough PCI device of the VM. The hypervisor can further designate one of the DMA buffer or the copy of the DMA buffer as a vCPU buffer that is accessible by the VM, and designate the other of the DMA buffer or the copy of the DMA buffer as a device buffer that is accessible by the passthrough PCI device. The hypervisor can then synchronize the vCPU buffer and the device buffer with each other as the VM and passthrough PCI device interact with their respective buffers, and as part of the synchronization can intercept DMA work requests submitted by the VM/completed by the passthrough PCI device.

Abstract: Techniques for enabling live migration of VMs with passthrough PCI devices are provided. In one set of embodiments, a hypervisor of a host system can create a copy of a DMA buffer used by a VM of the host system and a passthrough PCI device of the VM. The hypervisor can further designate one of the DMA buffer or the copy of the DMA buffer as a vCPU buffer that is accessible by the VM, and designate the other of the DMA buffer or the copy of the DMA buffer as a device buffer that is accessible by the passthrough PCI device. The hypervisor can then synchronize the vCPU buffer and the device buffer with each other as the VM and passthrough PCI device interact with their respective buffers, and as part of the synchronization can intercept DMA work requests submitted by the VM/completed by the passthrough PCI device.

Abstract: Techniques for enabling live migration of VMs with passthrough PCI devices are provided. In one set of embodiments, a hypervisor of a host system can create a copy of a DMA buffer used by a VM of the host system and a passthrough PCI device of the VM. The hypervisor can further designate one of the DMA buffer or the copy of the DMA buffer as a vCPU buffer that is accessible by the VM, and designate the other of the DMA buffer or the copy of the DMA buffer as a device buffer that is accessible by the passthrough PCI device. The hypervisor can then synchronize the vCPU buffer and the device buffer with each other as the VM and passthrough PCI device interact with their respective buffers, and as part of the synchronization can intercept DMA work requests submitted by the VM/completed by the passthrough PCI device.

Abstract: Guest memory data structures are read by one or more read operations which are set up to handle page faults and general protection faults generated during the read in various ways. If such a fault occurs while performing the one or more read operations, the fault is handled and the one or more read operation is terminated. The fault is handled by either dropping the fault and reporting an error instead of the fault, by dropping the fault and invoking an error handler that is set up prior to performing the read operations, or by forwarding the fault to a fault handler that is setup prior to performing the read operations. If no fault occurs, the read operations complete successfully. Thus, under normal circumstances, no fault is incurred in a read operation on guest memory data structures.

Abstract: Function exits are instrumented in tail-call optimized code in which calls to target functions and return instructions are replaced by jump instructions. A probe engine identifies a tail-call jump and instruments the jumps to raise an exception. In response to an exception raised at the tail-call jump, an exception handler loads various registers and transferring control to a trampoline, which calls the jump target. After the target function returns, an exit probe is fired when the trampoline itself returns.

Abstract: Guest memory data structures are read by one or more read operations which are set up to handle page faults and general protection faults generated during the read in various ways. If such a fault occurs while performing the one or more read operations, the fault is handled and the one or more read operation is terminated. The fault is handled by either dropping the fault and reporting an error instead of the fault, by dropping the fault and invoking an error handler that is set up prior to performing the read operations, or by forwarding the fault to a fault handler that is setup prior to performing the read operations. If no fault occurs, the read operations complete successfully. Thus, under normal circumstances, no fault is incurred in a read operation on guest memory data structures.

Abstract: Mechanisms to protect the integrity of memory of a virtual machine are provided. The mechanisms involve utilizing certain capabilities of the hypervisor underlying the virtual machine to monitor writes to memory pages of the virtual machine. A guest integrity driver communicates with the hypervisor to request such functionality. Additional protections are provided for protecting the guest integrity driver and associated data, as well as for preventing use of these mechanisms by malicious software. These additional protections include an elevated execution mode, termed “integrity mode,” which can only be entered from a specified entry point, as well as protections on the memory pages that store the guest integrity driver and associated data.

Abstract: Probes are employed to inject errors into code. In response to a function-entry trigger event, a probe writes a predefined test value to a return value register. The probe then cause function execution to be skipped such that the test value is returned in lieu of the value which would otherwise be returned by the function. Behavior after the error is injected may then be observed, data collected, etc. such that undesired behavior (e.g., crashes) can be identified and/or corrected. In an alternative embodiment, the probe which is triggered may write a test value to a given memory address.

Abstract: Probes are instrumented in multiple software modules of a computer system having virtual machines running therein and executed in a coordinated manner. An output of one probe may be used to conditionally trigger another probe so that the precision of collected data may be improved. In addition, outputs of probes that are triggered in different software modules by related events may be synchronized and analyzed collectively. Probes also may be parallel processed in different processors so that multiple probes can be processed concurrently.

Abstract: Probes are instrumented into a boot sequence of a computer system to enable probing of the boot sequence. As part of the boot sequence, a value stored in a predetermined storage location within a boot device is read and, if the value indicates that probing of the boot sequence has been enabled, executable code for probing the boot sequence is injected into the boot sequence. Outputs of the probing during the boot process are collected into a buffer and analyzed after the completion of the boot process.

Abstract: Function exits are instrumented in tail-call optimized code in which calls to target functions and return instructions are replaced by jump instructions. A probe engine identifies a tail-call jump and instruments the jumps to raise an exception. In response to an exception raised at the tail-call jump, an exception handler loads various registers and transferring control to a trampoline, which calls the jump target. After the target function returns, an exit probe is fired when the trampoline itself returns.

Abstract: Probes are employed to inject errors into code. In response to a function-entry trigger event, a probe writes a predefined test value to a return value register. The probe then cause function execution to be skipped such that the test value is returned in lieu of the value which would otherwise be returned by the function. Behavior after the error is injected may then be observed, data collected, etc. such that undesired behavior (e.g., crashes) can be identified and/or corrected. In an alternative embodiment, the probe which is triggered may write a test value to a given memory address.

Abstract: Probes are instrumented into a boot sequence of a computer system to enable probing of the boot sequence. As part of the boot sequence, a value stored in a predetermined storage location within a boot device is read and, if the value indicates that probing of the boot sequence has been enabled, executable code for probing the boot sequence is injected into the boot sequence. Outputs of the probing during the boot process are collected into a buffer and analyzed after the completion of the boot process.