Abstract: A distributed computing application is described that provides a highly elastic and multi-tenant platform for Hadoop applications and other workloads running in a virtualized environment. Multiple instances of a distributed computing framework, such as Hadoop, may be executed concurrently. A centralized manager detects when contention for computing resources, such as memory and CPU, causes tasks to run slower on VMs executing on a given host, and scales up or scales down a cluster based on the detected resource contention.

Abstract: A system and method for allocating power resources among host computers in a cluster uses lower and upper bounds with respect to a power budget to be distributed to each of the hosts. Each host is allocated a portion of the cluster power capacity. Any excess amount of the capacity is then allocated to the hosts based at least partly on the lower bound (reserve capacity) and the upper bound (host power limit) of each of the clients.

Abstract: Application resource scheduler module is provided to achieve cooperative application workload scheduling for a consolidated virtual environment. The application resource scheduler aids an application workload scheduler that is part of a distributed computing application, such as Hadoop, to achieve a specified relative priority of the application workload virtual machines to other virtual machines in the virtual environment. The application resource scheduler assists in achieving cooperative workload scheduling by revising the amount of resources that the application workload scheduler sees as available and by setting resource controls for the virtual machines of the distributed computing application to influence the resources the virtual machines receive from the underlying consolidated virtual environment.

Abstract: A distributed computing application is described that provides a highly elastic and multi-tenant platform for Hadoop applications and other workloads running in a virtualized environment. Multiple instances of a distributed computing framework, such as Hadoop, may be executed concurrently. A centralized manager detects when contention for computing resources, such as memory and CPU, causes tasks to run slower on VMs executing on a given host, and scales up or scales down a cluster based on the detected resource contention.

Abstract: A method of determining compatibility of a virtual machine or virtual machine disk file with a host, including a storage host, is disclosed. A lookup matrix is created to provide a fast compatibility lookup. To create the lookup matrix, computing object properties are retrieved from each of the plurality of computing objects. The computing object properties include resource requirements of each of the plurality of computing objects. Further, host properties are retrieved from each of the plurality of hosts. The method further includes creating host logical groups of a subset of the plurality of hosts having substantially same host properties and creating a plurality of computing object logical groups of a subject of the plurality of computing objects having substantially same computing object properties. The lookup matrix answers whether each member of a selected computing object logical group can be transferred to a selected host logical group.

Abstract: A system and method for allocating power resources among host computers in a cluster uses lower and upper bounds with respect to a power budget to be distributed to each of the hosts. Each host is allocated a portion of the cluster power capacity. Any excess amount of the capacity is then allocated to the hosts based at least partly on the lower bound (reserve capacity) and the upper bound (host power limit) of each of the clients.

Abstract: Application resource scheduler module is provided to achieve cooperative application workload scheduling for a consolidated virtual environment. The application resource scheduler aids an application workload scheduler that is part of a distributed computing application, such as Hadoop, to achieve a specified relative priority of the application workload virtual machines to other virtual machines in the virtual environment. The application resource scheduler assists in achieving cooperative workload scheduling by revising the amount of resources that the application workload scheduler sees as available and by setting resource controls for the virtual machines of the distributed computing application to influence the resources the virtual machines receive from the underlying consolidated virtual environment.

Abstract: Hardware virtualization support is used to isolate kernel extensions. A kernel and various kernel extensions are executed in a plurality of hardware protection domains. Each hardware protection domain defines computer resource privileges allowed to code executing in that hardware protection domain. Kernel extensions execute with appropriate computer resource privileges to complete tasks without comprising the stability of the computer system.

Abstract: In a virtualized system running one or more virtual machines on a first hypervisor, a second hypervisor is installed and control of the hardware resources of the physical computer supporting the virtualized system is migrated from the first hypervisor to the second hypervisor without interrupting the operation of the first hypervisor and the virtual machines. Initially a minimal set of hardware resources is hot-removed from control by the first hypervisor, and the second hypervisor is launched on the minimal set of hardware resources. Both the remaining hardware resources and the virtual machines are then migrated from the first hypervisor to the second hypervisor until all the virtual machines have been migrated over to the second hypervisor, while the virtual machines and the first hypervisor continue running largely unaffected by the migration process.

Abstract: In one approach, a method is described of generating an interface for an operating system kernel. The method calls for creating an input file, where the input file includes a node structure for the interface. A kernel component of the interface is generated from the input file. A user space component of the interface is also generated from the input file.

Abstract: A method of determining compatibility of a virtual machine or virtual machine disk file with a host, including a storage host, is disclosed. A lookup matrix is created to provide a fast compatibility lookup for a group of computing objects and a group of hosts. To create the lookup matrix, computing object properties are retrieved from each of the plurality of computing objects. The computing object properties include resource requirements of each of the plurality of computing objects. Further, host properties are retrieved from each of the plurality of hosts. The method further includes creating a plurality of host logical groups of a subset of the plurality of hosts having substantially same host properties and creating a plurality of computing object logical groups of a subject of the plurality of computing objects having substantially same computing object properties.

Abstract: In a virtualized system running one or more virtual machines on a first hypervisor, a second hypervisor is installed and control of the hardware resources of the physical computer supporting the virtualized system is migrated from the first hypervisor to the second hypervisor without interrupting the operation of the first hypervisor and the virtual machines. Initially a minimal set of hardware resources is hot-removed from control by the first hypervisor, and the second hypervisor is launched on the minimal set of hardware resources. Both the remaining hardware resources and the virtual machines are then migrated from the first hypervisor to the second hypervisor until all the virtual machines have been migrated over to the second hypervisor, while the virtual machines and the first hypervisor continue running largely unaffected by the migration process.

Abstract: One embodiment is a method of transferring data from a computer system to a Universal Serial Bus (USB) device after a computer system crash where interrupts are masked, the method comprising: (a) detecting the computer system crash; (b) transferring at least a portion of the data to a USB driver for the USB device; (c) the USB driver transferring the portion of the data to a USB controller driver for a USB controller for the USB device; (d) the USB controller driver causing the USB controller to transfer the portion of the data to the USB device; (e) polling the USB controller to determine whether the data transfer was completed; and (f) if the data transfer was completed, providing a notification to the computer system.

Abstract: A method of determining compatibility of a virtual machine or virtual machine disk file with a host, including a storage host, is disclosed. A lookup matrix is created to provide a fast compatibility lookup for a group of computing objects and a group of hosts. To create the lookup matrix, computing object properties are retrieved from each of the plurality of computing objects. The computing object properties include resource requirements of each of the plurality of computing objects. Further, host properties are retrieved from each of the plurality of hosts. The method further includes creating a plurality of host logical groups of a subset of the plurality of hosts having substantially same host properties and creating a plurality of computing object logical groups of a subject of the plurality of computing objects having substantially same computing object properties.

Abstract: In a virtualized system running one or more virtual machines on a first hypervisor, a second hypervisor is installed and control of the hardware resources of the physical computer supporting the virtualized system is migrated from the first hypervisor to the second hypervisor without interrupting the operation of the first hypervisor and the virtual machines. Initially a minimal set of hardware resources is hot-removed from control by the first hypervisor, and the second hypervisor is launched on the minimal set of hardware resources. Both the remaining hardware resources and the virtual machines are then migrated from the first hypervisor to the second hypervisor until all the virtual machines have been migrated over to the second hypervisor, while the virtual machines and the first hypervisor continue running largely unaffected by the migration process.

Abstract: In a virtualized system running one or more virtual machines on a first hypervisor, a second hypervisor is installed and control of the hardware resources of the physical computer supporting the virtualized system is migrated from the first hypervisor to the second hypervisor without interrupting the operation of the first hypervisor and the virtual machines. Initially a minimal set of hardware resources is hot-removed from control by the first hypervisor, and the second hypervisor is launched on the minimal set of hardware resources. Both the remaining hardware resources and the virtual machines are then migrated from the first hypervisor to the second hypervisor until all the virtual machines have been migrated over to the second hypervisor, while the virtual machines and the first hypervisor continue running largely unaffected by the migration process.

Abstract: One embodiment is a method of transferring data from a computer system to a Universal Serial Bus (USB) device after a computer system crash where interrupts are masked, the method comprising: (a) detecting the computer system crash; (b) transferring at least a portion of the data to a USB driver for the USB device; (c) the USB driver transferring the portion of the data to a USB controller driver for a USB controller for the USB device; (d) the USB controller driver causing the USB controller to transfer the portion of the data to the USB device; (e) polling the USB controller to determine whether the data transfer was completed; and (f) if the data transfer was completed, providing a notification to the computer system.

Abstract: Hardware virtualization support is used to isolate kernel extensions. A kernel and various kernel extensions are executed in a plurality of hardware protection domains. Each hardware protection domain defines computer resource privileges allowed to code executing in that hardware protection domain. Kernel extensions execute with appropriate computer resource privileges to complete tasks without comprising the stability of the computer system.

Abstract: A computer has a hardware memory arranged into portions that are separately addressable using first identifiers, which are represented using a first number of address bits. A subsystem that is able to address a second space of the hardware memory using second identifiers initiates I/O requests directed to a device that is able to address a different, first memory space using first identifiers, which are represented using a second number of address bits. The second identifiers are initially mapped into the second memory space, but for any I/O request that meets a remapping criterion, the corresponding second identifier is remapped to one of the first identifiers that identifies a portion of the memory in the first memory space. The second space is different from the first space and the second number of address bits is greater less than the first number of address bits.

Abstract: A subsystem that is able to address a second memory region initiates I/O requests directed to a device that is able to address a first memory region that is different from the second memory region. Requests for memory are mapped at least once, for example from virtual to physical page numbers. The I/O requests are conditionally remapped to pages in the first region as a function of how often they are involved in the I/O operations and would normally otherwise need to be copied. Remapping may also be made conditional on a function of availability of memory in the first region. In a preferred embodiment of the invention, the I/O requests are initiated by a subsystem within a virtual machine, which runs via an intermediate software layer such as a virtual machine monitor on an underlying hardware and software platform. A typical application of the invention is DMA.