Abstract: A data platform for running a subset of cells in a notebook is provided. The data platform receives a run cells message from a notebook user interface (UI) application specifying the subset of cells to run. For each cell in the subset, the data platform runs the cell to generate a set of results, generates a cell execution stream using the results, stores the stream, and transmits the stream to the notebook UI application. The notebook UI application generates a display for the user using the cell execution stream. The data platform provides an efficient way to run specific cells in a notebook and display the results to the user.

Abstract: A data platform for running a subset of cells in a notebook is provided. The data platform receives a run cells message from a notebook user interface (UI) application specifying the subset of cells to run. For each cell in the subset, the data platform runs the cell to generate a set of results, generates a cell execution stream using the results, stores the stream, and transmits the stream to the notebook UI application. The notebook UI application generates a display for the user using the cell execution stream. The data platform provides an efficient way to run specific cells in a notebook and display the results to the user.

Abstract: A data platform for running a subset of cells in a notebook is provided. The data platform receives a run cells message from a notebook user interface (UI) application specifying the subset of cells to run. For each cell in the subset, the data platform runs the cell to generate a set of results, generates a cell execution stream using the results, stores the stream, and transmits the stream to the notebook UI application. The notebook UI application generates a display for the user using the cell execution stream. The data platform provides an efficient way to run specific cells in a notebook and display the results to the user.

Abstract: A computerized mechanism to formulate a data flow representation from a syntax tree. The syntax tree may be, for instance, an abstract syntax tree (AST) that is formulated by compiling query script, such as big data query script. Each node in the syntax tree is composed of one or more tokens (script portions) that are taken from the query script, which relationships between the tokens being syntactically represented by links between those tokens. Accordingly, the data flow representation may also be a data flow representation of the original query script itself. In order to formulate the data flow representation from the syntax tree, the data types of the various inputs and outputs of the syntax tree nodes are identified and bound to a corresponding data flow. This may be performed whilst honoring dependencies between the nodes. The data flow may be visualized to an author or reviewer of the script.

Abstract: Reducing data skew when performing computational jobs (such as responding to complex queries or queries on big data sets). To gather performance information for a vertex in a distributed stage, the distributed stage is processed and the resulting performance information being generated. The system then identifies a performance outlier of the vertices, which varies from a performance norm of other vertices. A trigger instruction is then generated that is structured to trigger the performance outlier vertex to gather data distribution information of the data that the performance outlier vertex previously processed. The trigger instruction may thus be executed to cause the performance outlier vertex to gather the data distribution information for evaluation for data skew.

Abstract: Simulation of a computational job using various sets of resources, and potentially also the automated or semi-automated allocation of an appropriate set of resources for accomplishing a computational job comprising multiple vertices. For each of multiple potential sets of resources, a simulation module simulates processing of the computational job. While the simulation does not actually perform the vertex on each processing node, the simulation does use dependencies between vertices, and historical data regarding the processing of instances of such vertices, in order to determine the efficacy of processing of each vertex, and to put the estimations together into an overall simulation result.

Abstract: The supporting of debugging of failed vertex code on a test machine. This debugging is made possible even though the vertex code failed while on a remote processing node, and is but one of multiple, and potentially innumerable vertices that run in a distributed environment. This represents a vast technical improvement over prior ad hoc methods for trying to debug a large distributed application, since time is not wasted on vertices that operated properly, but rather debugging is focused on the problem vertex. Even reproducing the failure is a huge technical step forward.

Abstract: Identifying a path for a distributed job. A method includes dynamically collecting timing and relationship information for vertices in stages of a running job. The method further includes identifying a particular vertex. The method further includes iteratively identifying a single path parent-child sequence path for the identified vertex. The method further includes displaying a Gantt chart showing the identified path, wherein the Gantt chart shows timing relationships and execution progress information.

Abstract: The supporting of debugging of failed vertex code on a test machine. This debugging is made possible even though the vertex code failed while on a remote processing node, and is but one of multiple, and potentially innumerable vertices that run in a distributed environment. This represents a vast technical improvement over prior ad hoc methods for trying to debug a large distributed application, since time is not wasted on vertices that operated properly, but rather debugging is focused on the problem vertex. Even reproducing the failure is a huge technical step forward.

Abstract: Reducing data skew when performing computational jobs (such as responding to complex queries or queries on big data sets). To gather performance information for a vertex in a distributed stage, the distributed stage is processed and the resulting performance information being generated. The system then identifies a performance outlier of the vertices, which varies from a performance norm of other vertices. A trigger instruction is then generated that is structured to trigger the performance outlier vertex to gather data distribution information of the data that the performance outlier vertex previously processed. The trigger instruction may thus be executed to cause the performance outlier vertex to gather the data distribution information for evaluation for data skew.

Abstract: A computerized mechanism to formulate a data flow representation from a syntax tree. The syntax tree may be, for instance, an abstract syntax tree (AST) that is formulated by compiling query script, such as big data query script. Each node in the syntax tree is composed of one or more tokens (script portions) that are taken from the query script, which relationships between the tokens being syntactically represented by links between those tokens. Accordingly, the data flow representation may also be a data flow representation of the original query script itself. In order to formulate the data flow representation from the syntax tree, the data types of the various inputs and outputs of the syntax tree nodes are identified and bound to a corresponding data flow. This may be performed whilst honoring dependencies between the nodes. The data flow may be visualized to an author or reviewer of the script.

Abstract: A computerized mechanism to automatically correlate positions of query script to portions of a data flow representation of the query script. When parsing the query script to generate the tokens, at least some of the tokens have an associated script location marker that identifies a location in the query script where the token originated from. The syntax tree of multiple nodes is then formulated, each node comprising one or more of the tokens parsed from the query script. Accordingly, the syntax tree retains the script location markers. A data flow representation of the query script is then formulated into a data flow representation. That data flow representation might, for instance, be based on the syntax tree, but augmented with data types of the various data flows. Nevertheless, the location marker is retained within the data flow representation.

Abstract: Simulation of a computational job using various sets of resources, and potentially also the automated or semi-automated allocation of an appropriate set of resources for accomplishing a computational job comprising multiple vertices. For each of multiple potential sets of resources, a simulation module simulates processing of the computational job. While the simulation does not actually perform the vertex on each processing node, the simulation does use dependencies between vertices, and historical data regarding the processing of instances of such vertices, in order to determine the efficacy of processing of each vertex, and to put the estimations together into an overall simulation result.

Abstract: A DataGrid user interface (UI) control is presented that is configured to display a table that includes columns corresponding to fields in a data source and rows corresponding to records in the data source. The DataGrid UI control is also configured to display a details section associated with each row in the table. One or all of the details sections can be displayed or hidden based upon properties specified by a developer or input made by a user. A property may also be set to cause the details section for a row to be displayed only when a user interface cursor is hovering over the row associated with the details section. A property may also be set to cause a details section for a row to be displayed only when the associated row is selected.

Abstract: Described is a technology in which a user interface element (a control, e.g., a datagrid) provides smoother scrolling by computing and updating a row height estimate. For example, the row height estimate along with the total number of rows may be used to configure a scrollbar. A control outputs visible rows according to user scrolling interaction based on a current row index and current pixel values corresponding to that current row index. The estimate is obtained based upon the current row index and the current pixel value for the row index. Whenever a row index is encountered that is larger than any previously-encountered row index, the row height estimate is updated, e.g., to reconfigure the scrollbar. Also described is outputting partially visible rows based on a number of pixels available in a display area below a last fully visible row, and/or above a first fully visible row.

Abstract: Architecture that facilitates the recycling of view components (e.g., item controls, visual containers, etc.) in a user interface rather than recreating the view components when needed again in response to a view change such as a scrolling operation or other user navigation operation such as a paging control. In the context of a datagrid, the datagrid can now be more performant when scrolling through a large number of rows. The architecture includes a recycling component that recycles the visual containers by collecting the visual containers when rows are scrolled off the view, and reusing the visual containers with different data when rows are scrolled back into the view. Thus, all the containers stay the same—new data is simply applied to the recycled container.

Abstract: A DataGrid user interface (UI) control is presented that is configured to display a table that includes columns corresponding to fields in a data source and rows corresponding to records in the data source. The DataGrid UI control is also configured to display a details section associated with each row in the table. One or all of the details sections can be displayed or hidden based upon properties specified by a developer or input made by a user. A property may also be set to cause the details section for a row to be displayed only when a user interface cursor is hovering over the row associated with the details section. A property may also be set to cause a details section for a row to be displayed only when the associated row is selected.

Abstract: Application logic and user interfaces are separated to allow more than one interface or view to be easily employed for given application logic. A particular interface can be utilized to display data and/or facilitate interaction simply by identifying the interface. Available interfaces are identified to users for selection thereof. The specific interfaces and manner of identification can be filtered or otherwise controlled or arranged. Upon selection of an interface, data can be rendered accordingly. A transition can also be applied between switched interfaces.