Abstract: Embodiments support instant forking of virtual machines (VMs) and state customization. A computing device initiates execution of a first group of services (e.g., identity-independent) in a first VM. A second VM is instantiated from the first VM. The second VM shares memory and storage with the first VM. The computing device customizes the second VM based on configuration data associated with the second VM. A second group of services (e.g., identity-dependent) starts executing on the second VM after configuring the identity of the second VM. Customizing the second VM includes configuring one or more identities of the second VM. In some embodiments, a domain identity is selected from a pool of previously-created identities and applied to the second VM, before bootup completes on the second VM.

Abstract: A distributed computing application is described that provides a highly elastic and multi-tenant platform for Hadoop applications and other workloads running in a virtualized environment. Production, test, and development deployments of a Hadoop application may be executed using multiple compute clusters and a shared instance of a distributed filesystem, or in other cases, multiple instances of the distributed filesystem. Data nodes executing as virtual machines (VMs) for test and development deployments can be linked clones of data nodes executing as VMs for a production deployment to reduce duplicated data and provide a shared storage space.

Abstract: A distributed computing application is described that provides a highly elastic and multi-tenant platform for Hadoop applications and other workloads running in a virtualized environment. Data and compute nodes are separated into different virtual machines (VM). Compute VMs are used to launch containers from different tenants. Compute VMs are organized in pools of hot spare VMs that are immediately available for launching a container and executing a task, and pools of cold spare VMs. Each compute VM may include a mounted network filesystem provided by a node manager to share intermediate outputs across VMs executing on the same host.

Abstract: A distributed computing application is described that provides a highly elastic and multi-tenant platform for Hadoop applications and other workloads running in a virtualized environment. Deployments of a distributed computing application, such as Hadoop, may be executed concurrently with a distributed database application, such as HBase, using a shared instance of a distributed filesystem, or in other cases, multiple instances of the distributed filesystem. Computing resources allocated to region server nodes executing as VMs may be isolated from compute VMs of the distributed computing application, as well as from data nodes executing as VMs of the distributed filesystem.

Abstract: A distributed computing application is described that provides a highly elastic and multi-tenant platform for Hadoop applications and other workloads running in a virtualized environment. Multiple instances of a distributed computing framework, such as Hadoop, may be executed concurrently. A centralized manager detects when contention for computing resources, such as memory and CPU, causes tasks to run slower on VMs executing on a given host, and scales up or scales down a cluster based on the detected resource contention.

Abstract: Application resource scheduler module is provided to achieve cooperative application workload scheduling for a consolidated virtual environment. The application resource scheduler aids an application workload scheduler that is part of a distributed computing application, such as Hadoop, to achieve a specified relative priority of the application workload virtual machines to other virtual machines in the virtual environment. The application resource scheduler assists in achieving cooperative workload scheduling by revising the amount of resources that the application workload scheduler sees as available and by setting resource controls for the virtual machines of the distributed computing application to influence the resources the virtual machines receive from the underlying consolidated virtual environment.

Abstract: A system and method embodiments for optimally allocating compute kernels to different types of processors, such as CPUs and GPUs, in a heterogeneous computer system are disclosed. These include comparing a kernel profile of a compute kernel to respective processor profiles of a plurality of processors in a heterogeneous computer system, selecting at least one processor from the plurality of processors based upon the comparing, and scheduling the compute kernel for execution in the selected at least one processor.

Abstract: A system and method embodiments for optimally allocating compute kernels to different types of processors, such as CPUs and GPUs, in a heterogeneous computer system are disclosed. These include comparing a kernel profile of a compute kernel to respective processor profiles of a plurality of processors in a heterogeneous computer system, selecting at least one processor from the plurality of processors based upon the comparing, and scheduling the compute kernel for execution in the selected at least one processor.

Abstract: A method for executing a plurality of data-parallel threads of a processing logic on a processor core includes grouping the plurality of data-parallel threads into one or more workgroups, associating a first workgroup from the one or more workgroups with an operating system thread on the processor core, and configuring threads from the first workgroup as user-level threads within the operating system thread. In an example, a method enables the execution of GPU-kernels that has been previously configured for a GPU, to execute on a CPU such as a multi-core CPU. The mapping of the numerous data-parallel threads to the CPU is done in such a manner as to reduce the number of costly operating system threads instantiated on the CPU, and to enable efficient debugging.