Abstract: Methods, systems, and computer programs for providing an application server appliance utilizing one or more virtual machines are described. The application server appliance may be a virtual machine having a reduced guest operating system, a runtime environment, and a management agent installed therein. An appliance controller automatically determines one or more configurations and/or settings for the runtime environment based on a variety of factors, including the set up of the virtual machine appliance. The appliance controller generates an application package having the determined settings and transmits the package to the application server appliance, wherein the application package is configured to be executed by the runtime environment.

Abstract: An elastic filesystem for temporary data provides storage space for virtual machines (VMs) in a distributed computing system. The filesystem redirects accesses to virtual disks in VMs to a common pool file. The system provides performance and storage efficiency at least on par with local, direct attached virtual disks, while providing a single pool of shared storage that is provisioned and managed independently of the VMs. The system provides storage isolation between VMs storing temporary data in that shared pool. Also, storage space for temporary data may be allocated on demand and reclaimed when no longer needed, thereby supporting a wide variety of temporary space requirements for different Hadoop jobs.

Abstract: A distributed computing application is described that provides a highly elastic and multi-tenant platform for Hadoop applications and other workloads running in a virtualized environment. Data and compute nodes are separated into different virtual machines (VM). Compute VMs are used to launch containers from different tenants. Compute VMs are organized in pools of hot spare VMs that are immediately available for launching a container and executing a task, and pools of cold spare VMs. Each compute VM may include a mounted network filesystem provided by a node manager to share intermediate outputs across VMs executing on the same host.

Abstract: Methods, systems, and computer programs for managing memory in a host where virtual machines (VMs) execute are presented. In one example, a method includes an operation for determining which amount of heap memory has been reserved in a Java virtual machine (JVM) that is in excess of the heap memory needed by the JVM. If there is excess heap memory, a Java balloon agent reserves a Java object in the heap memory. Typically, the Java object will be the size of one or more memory pages in the host. Further, the Java balloon agent loads the Java object with a certain value, for example, by zeroing out the page of memory. When a virtual machine monitor (VMM) in the host detects that a machine physical memory page associated with the Java object has the first value, then the VMM frees the machine physical memory page to make the memory available to other VMs or to other processes executing in the host.

Abstract: Application resource scheduler module is provided to achieve cooperative application workload scheduling for a consolidated virtual environment. The application resource scheduler aids an application workload scheduler that is part of a distributed computing application, such as Hadoop, to achieve a specified relative priority of the application workload virtual machines to other virtual machines in the virtual environment. The application resource scheduler assists in achieving cooperative workload scheduling by revising the amount of resources that the application workload scheduler sees as available and by setting resource controls for the virtual machines of the distributed computing application to influence the resources the virtual machines receive from the underlying consolidated virtual environment.

Abstract: Methods, systems, and computer programs for managing memory in a host where virtual machines (VMs) execute are presented. In one embodiment, a method includes an operation for determining which amount of heap memory has been reserved in a Java virtual machine (JVM) that is in excess of the heap memory needed by the JVM. If there is excess heap memory, a Java balloon agent reserves a Java object in the heap memory. Typically, the Java object will be the size of one or more memory pages in the host. Further, the Java balloon agent loads the Java object with a certain value, for example, by zeroing out the page of memory. When a virtual machine monitor (VMM) in the host detects that a machine physical memory page associated with the Java object has the first value, then the VMM frees the machine physical memory page to make the memory available to other VMs or to other processes executing in the host.

Abstract: Application resource scheduler module is provided to achieve cooperative application workload scheduling for a consolidated virtual environment. The application resource scheduler aids an application workload scheduler that is part of a distributed computing application, such as Hadoop, to achieve a specified relative priority of the application workload virtual machines to other virtual machines in the virtual environment. The application resource scheduler assists in achieving cooperative workload scheduling by revising the amount of resources that the application workload scheduler sees as available and by setting resource controls for the virtual machines of the distributed computing application to influence the resources the virtual machines receive from the underlying consolidated virtual environment.

Abstract: A method for dirty region logging of a file that includes receiving a request to open the file, determining each of a plurality of component files associated with the file, opening each of the plurality of component files, writing to a region of the at least one of the plurality component files, and updating a dirty region log (DRL) associated with the one of the plurality of component files to reflect the write to the region.

Abstract: A file is striped across multiple filers, file servers or other devices, to create a sparsely striped multi-component file. Each filer stores one sparse component. In particular, each component physically stores only those stripes allocated to that component. The other stripes are represented as holes. Thus, instead of contiguously packing each component's stripes at the block level, each component is a file having the same logical structure. A component of a sparsely striped multi-component file can be easily converted to a mirror by filling in its holes. Similarly, a mirror can be easily converted to one component of a sparsely striped multi-component file by removing or ignoring it unallocated stripes. In either case, the layout or logical of the component does not need to be reconfigured.

Abstract: A system and method for facilitating a consistent point-in-time copy (PITC). A data provider offers access to data by one or more clients. A client issues a PITC request for a set of data to the data provider, or the provider initiates its own request. The data provider instructs one or more clients, particularly clients accessing the set of data, to flush any pending updates involving the files. The clients acknowledge the flush request, flush their data, and notify the data provider when their flushes are complete. Post-consistency updates are deferred (e.g., using a write-aside buffer). If a client fails to acknowledge the flush request or notify of flush completion, the data provider may abort the PITC or continue because the data may not be consistent or consistency may not be assured. If not aborted, the PITC may be marked to indicate that it is incomplete.