Abstract: The visualization of progress of a distributed computational job at multiple points of execution. After a computational job is compiled into multiple vertices, and then those multiple vertices are scheduled on multiple processing nodes in a distributed environment, a processing gathering module gathers processing information regarding processing of multiple vertices of a computational job, and at multiple instances in time in the execution of the computational job. A user interface module graphically presents a representation of an execution structure representing multiple nodes of the computational job, and dependencies between the multiple nodes, where the nodes may be a single vertex or a group of vertices (such as a stage).

Abstract: The visualization of progress of a distributed computational job at multiple points of execution. After a computational job is compiled into multiple vertices, and then those multiple vertices are scheduled on multiple processing nodes in a distributed environment, a processing gathering module gathers processing information regarding processing of multiple vertices of a computational job, and at multiple instances in time in the execution of the computational job. A user interface module graphically presents a representation of an execution structure representing multiple nodes of the computational job, and dependencies between the multiple nodes, where the nodes may be a single vertex or a group of vertices (such as a stage).

Abstract: The visualization of progress of a distributed computational job at multiple points of execution. After a computational job is compiled into multiple vertices, and then those multiple vertices are scheduled on multiple processing nodes in a distributed environment, a processing gathering module gathers processing information regarding processing of multiple vertices of a computational job, and at multiple instances in time in the execution of the computational job. A user interface module graphically presents a representation of an execution structure representing multiple nodes of the computational job, and dependencies between the multiple nodes, where the nodes may be a single vertex or a group of vertices (such as a stage).

Abstract: Reducing data skew when performing computational jobs (such as responding to complex queries or queries on big data sets). To gather performance information for a vertex in a distributed stage, the distributed stage is processed and the resulting performance information being generated. The system then identifies a performance outlier of the vertices, which varies from a performance norm of other vertices. A trigger instruction is then generated that is structured to trigger the performance outlier vertex to gather data distribution information of the data that the performance outlier vertex previously processed. The trigger instruction may thus be executed to cause the performance outlier vertex to gather the data distribution information for evaluation for data skew.

Abstract: Comparing recurring processes. A method includes automatically identifying a plurality of recurring process instances having one or more commonalities. The method further includes displaying the plurality of recurring process instances having one or more commonalities to a user in a user interface. The method further includes receiving user input at the user interface selecting a first recurring process instance from the plurality of recurring process instances. The method further includes receiving user input at the user interface selecting a second recurring process instance from the plurality of recurring process instances. The method further includes automatically identifying differences in the first and second recurring process instances. The method further includes presenting the differences in the graphical user interface to the user.

Abstract: The visualization of progress of a distributed computational job at multiple points of execution. After a computational job is compiled into multiple vertices, and then those multiple vertices are scheduled on multiple processing nodes in a distributed environment, a processing gathering module gathers processing information regarding processing of multiple vertices of a computational job, and at multiple instances in time in the execution of the computational job. A user interface module graphically presents a representation of an execution structure representing multiple nodes of the computational job, and dependencies between the multiple nodes, where the nodes may be a single vertex or a group of vertices (such as a stage).

Abstract: The supporting of debugging of failed vertex code on a test machine. This debugging is made possible even though the vertex code failed while on a remote processing node, and is but one of multiple, and potentially innumerable vertices that run in a distributed environment. This represents a vast technical improvement over prior ad hoc methods for trying to debug a large distributed application, since time is not wasted on vertices that operated properly, but rather debugging is focused on the problem vertex. Even reproducing the failure is a huge technical step forward.

Abstract: The visualization of progress of a distributed computational job at multiple points of execution. After a computational job is compiled into multiple vertices, and then those multiple vertices are scheduled on multiple processing nodes in a distributed environment, a processing gathering module gathers processing information regarding processing of multiple vertices of a computational job, and at multiple instances in time in the execution of the computational job. A user interface module graphically presents a representation of an execution structure representing multiple nodes of the computational job, and dependencies between the multiple nodes, where the nodes may be a single vertex or a group of vertices (such as a stage).

Abstract: Comparing recurring processes. A method includes automatically identifying a plurality of recurring process instances having one or more commonalities. The method further includes displaying the plurality of recurring process instances having one or more commonalities to a user in a user interface. The method further includes receiving user input at the user interface selecting a first recurring process instance from the plurality of recurring process instances. The method further includes receiving user input at the user interface selecting a second recurring process instance from the plurality of recurring process instances. The method further includes automatically identifying differences in the first and second recurring process instances. The method further includes presenting the differences in the graphical user interface to the user.

Abstract: Identifying a path for a distributed job. A method includes dynamically collecting timing and relationship information for vertices in stages of a running job. The method further includes identifying a particular vertex. The method further includes iteratively identifying a single path parent-child sequence path for the identified vertex. The method further includes displaying a Gantt chart showing the identified path, wherein the Gantt chart shows timing relationships and execution progress information.

Abstract: The visualization of progress of a distributed computational job at multiple points of execution. After a computational job is compiled into multiple vertices, and then those multiple vertices are scheduled on multiple processing nodes in a distributed environment, a processing gathering module gathers processing information regarding processing of multiple vertices of a computational job, and at multiple instances in time in the execution of the computational job. A user interface module graphically presents a representation of an execution structure representing multiple nodes of the computational job, and dependencies between the multiple nodes, where the nodes may be a single vertex or a group of vertices (such as a stage).

Abstract: The supporting of debugging of failed vertex code on a test machine. This debugging is made possible even though the vertex code failed while on a remote processing node, and is but one of multiple, and potentially innumerable vertices that run in a distributed environment. This represents a vast technical improvement over prior ad hoc methods for trying to debug a large distributed application, since time is not wasted on vertices that operated properly, but rather debugging is focused on the problem vertex. Even reproducing the failure is a huge technical step forward.

Abstract: Reducing data skew when performing computational jobs (such as responding to complex queries or queries on big data sets). To gather performance information for a vertex in a distributed stage, the distributed stage is processed and the resulting performance information being generated. The system then identifies a performance outlier of the vertices, which varies from a performance norm of other vertices. A trigger instruction is then generated that is structured to trigger the performance outlier vertex to gather data distribution information of the data that the performance outlier vertex previously processed. The trigger instruction may thus be executed to cause the performance outlier vertex to gather the data distribution information for evaluation for data skew.

Abstract: In a method of determining the availability of a network resource in a digital communications network, atomic resources that are not composed of other resources are first identified. A degradation factor is assigned to each atomic resource. Subsequently a degradation factor for a composite resource is derived from the degradation factors of the atomic resources in accordance with a predetermined relationship.