Abstract: A system identifies a performance bottleneck status in a parallel data processing environment by examining data flow associated with the parallel data processing environment to identify at least one operator, where an operator type is associated with at least one operator, at least one buffer, and a relationship that the buffer has with the operator, where the relationship is associated with the operator type. The system monitors the buffer to determine a buffer status associated with the buffer. The system applies a set of rules to identify an operator bottleneck status associated with the operator. The set of rules is applied to the operator, based on the operator type, the buffer status, and relationship that the buffer has with the operator. The system then determines a performance bottleneck status associated with the parallel data processing environment, based on the operator bottleneck status.

Abstract: Provided are techniques for multiple stage workload management. A staging queue and a run queue are provided. A workload is received. In response to determining that application resources are not available and that the workload has not been previously semi-started, the workload is added to the staging queue. In response to determining that the application resources are not available and that the workload has been semi-started, and, in response to determining that run resources are available, the workload is started. In response to determining that the application resources are not available and that the workload has been semi-started, and, in response to determining that the run resources are not available, adding the workload to the run queue.

Abstract: According to one embodiment of the present invention, a system tests jobs in a computing environment. The system creates a test case for one or more existing executable jobs without modifying the job design or recompiling the executable itself, wherein the test case includes one or more capture points in a job flow of the executable jobs and corresponding rules for capturing data, identification of data for testing the one or more executable jobs, and rules for comparing the captured data to expected results. The system captures the data at the one or more capture points in the job flow in accordance with the test case and generates a baseline of expected results.

Abstract: A data driven parallel sorting method includes distributing input data records to n partitions one by one in a circular manner. Each partition corresponds to a parallel sorting process with an allocated memory chunk sized to store m data records. The method also includes sorting, in parallel, current data records in respective memory chunks in respective partitions. The method also includes in response to distribution of data records of ?m/n? rounds, circularly controlling one of the n partitions, and writing data records that have been sorted in the memory chunk of the partition into a mass storage as an ordered data chunk, and emptying the memory chunk. The method also includes in response to all data records being distributed, writing data chunks that have been sorted in respective memory chunks into the mass storage, and performing a merge sort on all ordered data chunks in the mass storage.

Abstract: A method, computer program product and system for workload management for an Extract, Transform, and Load (ETL) system. A priority of each workload in a set of workloads is determined using a priority rule. In response to determining that the priority of a workload to be checked has a highest priority, it is indicated that the workload has the highest priority. It is determined whether at least one logical resource representing an ETL metric is available for executing the workload. In response to determining that the workload has the highest priority and that the at least one logical resource is available, it is determined that the workload is runnable.

Abstract: A method, computer program product, and system for dynamically distributing data for parallel processing in a computing system, comprising allocating a data buffer to each of a plurality of data partitions, where each data buffer stores data to be processed by its corresponding data partition, distributing data in multiple rounds to the data buffers for processing by the data partitions, where in each round the data is distributed based on a determined data processing capacity for each data partition, and where a greater amount of data is distributed to the data partitions with higher determined processing capacities, and periodically monitoring usage of each data buffer and re-determining the determined data processing capacity of each data partition based on its corresponding data buffer usage.

Abstract: A method, computer program product, and system for dynamically distributing data for parallel processing in a computing system, comprising allocating a data buffer to each of a plurality of data partitions, where each data buffer stores data to be processed by its corresponding data partition, distributing data in multiple rounds to the data buffers for processing by the data partitions, where in each round the data is distributed based on a determined data processing capacity for each data partition, and where a greater amount of data is distributed to the data partitions with higher determined processing capacities, and periodically monitoring usage of each data buffer and re-determining the determined data processing capacity of each data partition based on its corresponding data buffer usage.

Abstract: An embodiment of the present invention discloses an enterprise application packaging method and archive file. Applications and support components are developed by a developer, which are then included in the archive file. A revision control file indicates which files correlate to a present revision of the application package. A naming convention is applied to the application and support component files to indicate versions which correspond to each other. The archive file contains the file corresponding to the present version, as well as previous versions, as well as previous version revision control files so that, in the event of a failure or other error, the executing machine may revert to a previous version.

Abstract: Methods and systems for graphically emphasizing a selected path through a diagram, where the diagram includes a number of nodes and a number of lines, the methods and systems including: applying a node highlight effect to a node of the number of nodes in the selected path, where applying the node highlight effect includes applying a node shadow to the node, applying a line highlight effect to a line of the number of lines in the selected path, where applying the line highlight effect includes applying a line shadow to the line, applying a node fade effect to a node of the number of nodes not in the selected path, and applying a line fade effect to a line of the number of lines not in the selected path.