Abstract: A method for tuning workflow settings in a distributed computing workflow comprising sequential interdependent jobs includes pairing a terminal stage of a first job and a leading stage of a second, sequential job to form an optimization pair, in which data segments output by the terminal stage of the first job comprises data input for the leading stage of the second job. The performance of the optimization pair is tuned by determining, with a computational processor, an estimated minimum execution time for the optimization pair and increasing the minimum execution time to generate an increased execution time. The method further includes calculating a minimum number of data segments that still permit execution of the optimization pair within the increased execution time.

Abstract: Determining a schedule of a batch workload of MapReduce jobs is disclosed. A set of multi-stage jobs for processing in a MapReduce framework is received, for example, in a master node. Each multi-stage job includes a duration attribute, and each duration attribute includes a stage duration and a stage type. The MapReduce framework is separated into a plurality of resource pools. The multi-stage jobs are separated into a plurality of subgroups corresponding with the plurality of pools. Each subgroup is configured for concurrent processing in the MapReduce framework. The multi-stage jobs in each of the plurality of subgroups are placed in an order according to increasing stage duration. For each pool, the multi-stage jobs in increasing order of stage duration are sequentially assigned from either a front of the schedule or a tail of the schedule by stage type.

Abstract: A method comprises receiving, by a workload profiler, a representative workload of a computing system under analysis. The workload profiler determines a workload profile of the computing system that reflects a transaction mix that varies over time. A capacity analyzer receives the workload profile, and determines a maximum capacity of the computing system under analysis for serving the workload profile while satisfying a defined quality of service (QoS) target.

Abstract: A system comprises a workload manager evaluator operable to receive a representative workload that is representative of competing workloads that share access to at least one shared computing resource. The workload manager evaluator is operable to evaluate performance of a scheduler that schedules access of the competing workloads to the shared computing resource according to defined control parameter values, wherein the workload manager evaluator evaluates performance of the scheduler under the representative workload for a plurality of different values of the control parameters. In certain embodiments, the workload manager evaluator determines an optimal value for the control parameters of the scheduler for scheduling access to the at least one shared computing resource for the representative workload to satisfy defined performance desires of the system.

Abstract: Parameter values are extracted from information regarding a workload including map tasks and reduce tasks. A benchmark specification is produced based on the extracted parameter values, the benchmark specification including parameters and respective collections of values for the parameters. Based on the benchmark specification, benchmarks are produced that describe respective characteristics of the map and reduce tasks.

Abstract: A method comprises receiving a pattern of resource demands in a workload trace. The method further comprises identifying a plurality of occurrences of the determined pattern in the workload trace, and analyzing the occurrences to determine a trend of the workload trace. The method further comprises generating at least one synthetic workload trace representative of expected resource demands of the received workload trace accounting for the determined trend.

Abstract: At least one embodiment is for a method for estimating resource costs required to process an workload to be completed using at least two different cloud computing models. Historical trace data of at least one completed workload that is similar to the workload to be completed is received by the computer. The processing of the completed workload is simulated using a t-shirt cloud computing model and a time-sharing model. The t-shirt and time-sharing resource costs are estimated based on their respective simulations. The t-shirt and resource costs are then compared.

Abstract: One embodiment is a method that backups up objects to a storage device. A number of objects that are concurrently backed up to the storage device is limited.

Abstract: Determining a completion time of a job in a distributed network environment, the method includes determining completion times of a map task and a reduce task of a job and executing at least one test to collect a training dataset that characterizes the completion times of the map task and the reduce task.

Abstract: Embodiments are directed to methods and apparatus that backup filesystems to a storage device. A backup simulation is used to determine a number of agents to backup the filesystems.

Abstract: According to at least one embodiment, a method comprises observing communication from a virtual machine (VM) to a virtual machine monitor (VMM). The method further comprises determining, based on the observed communication, CPU utilization of the VMM that is attributable to the VM. According to at least one embodiment, a system comprises a Central Processing Unit (CPU), Virtual Machines (VMs), and a Virtual Machine Monitor (VMM) operable to receive requests for resource access from the VMs. The system further comprises a CPU utilization monitor operable to determine an amount of CPU utilization of the VMM in processing the received requests that is attributable to each of the VMs.

Abstract: A job profile describes characteristics of a job. A performance parameter is calculated based on the job profile, and using a value of the performance parameter, an allocation of resources is determined to assign to the job to meet a performance goal associated with a job.

Abstract: At least one benchmark is determined. The at least one benchmark is run on first and second computing platforms to generate platform profiles. Based on the generated platform profiles, a model is generated that characterizes a relationship between a MapReduce job executing on the first platform and the MapReduce job executing on the second platform, wherein the MapReduce job includes map tasks and reduce tasks.

Abstract: A method for tuning workflow settings in a distributed computing workflow comprising sequential interdependent jobs includes pairing a terminal stage of a first job and a leading stage of a second, sequential job to form an optimization pair, in which data segments output by the terminal stage of the first job comprises data input for the leading stage of the second job. The performance of the optimization pair is tuned by determining, with a computational processor, an estimated minimum execution time for the optimization pair and increasing the minimum execution time to generate an increased execution time. The method further includes calculating a minimum number of data segments that still permit execution of the optimization pair within the increased execution time.

Abstract: A method comprises determining a flexible limit on an amount of resource usage by a driver domain on behalf of a given virtual machine (VM). The method further comprises controlling the resource usage by the driver domain on behalf of the given VM so as not to exceed the flexible limit except under a permitted condition. In certain embodiments the resource usage by the driver domain on behalf of the given VM is controlled so as not to exceed the flexible limit except when a slack share of resource usage is available to the driver domain. Such a slack share of resource usage is a share of resource usage allocated to the driver domain that will not otherwise be used on behalf of another VM.

Abstract: A plurality of job profiles is received. Each job profile describes a job to be executed, and each job includes map tasks and reduce tasks. An execution duration for a map stage including the map tasks and an execution duration for a reduce stage including the reduce tasks of each job is estimated. The jobs are scheduled for execution based on the estimated execution duration of the map stage and the estimated execution duration of the reduce stage of each job.

Abstract: A system includes a number of hard disk drives and a controller. Each hard disk drive is selected from a number of different hard disk drive types. Each hard disk drive type has a different speed at which the hard disk drives of the hard disk drive type rotate to read and write data. At least one of the hard disk drives is of a different hard disk drive type than at least one other of the hard disk drives. The controller, responsive to a request to write particular data to the hard disk drives, is to select a given hard disk drive of the hard disk drives based on a type of the particular data, and is to write the particular data to the given hard disk drive.

Abstract: A job profile includes characteristics of a job to be executed, where the characteristics of the job profile relate to map tasks and reduce tasks of the job, and where the map tasks produce intermediate results based on input data, and the reduce tasks produce an output based on the intermediate results. In response to a failure in a system, numbers of failed map tasks and reduce tasks of the job based on a time of the failure are computed, and numbers of remaining map tasks and reduce tasks are computed. A performance model is provided, and a performance parameter of the job is estimated using the performance model.

Abstract: Storage device libraries, machine readable media, and methods are provided for determining a number of storage devices to backup objects in view of quality of service considerations. An example of a storage device library that determines the number of storage devices to backup objects includes a plurality of storage devices and a controller to control backup of the objects to an assigned number of the storage devices. The controller determines the assigned number of the storage devices before the backup of the objects based upon assigned parameters for backup of the objects that include a time window and a number of concurrent disk agents per storage device.

Abstract: Allocations of resources are determined for jobs that have map tasks and reduce tasks. The jobs are ordered according to performance goals of the jobs. The tasks of the jobs are scheduled for execution according to the ordering and the allocations of resources for the respective jobs.