Abstract: A computer system displays a flow diagram having a plurality of nodes. Each node corresponds to a respective dataset having respective data fields. In accordance with receiving a first user input to select a first node, corresponding to a first dataset, and a second node, corresponding to a second dataset, from the flow diagram, the computer system displays one or more join candidates for joining data from the first dataset and the second dataset. Each of the join candidates is a respective data field that exists on both the first and second datasets. The computer system receives a second user input selecting a first join candidate. In response to receiving the second user input, the computer system combines data columns from the first and second datasets into a single table and displays, in the flow diagram, a new node that graphically connects the first node and the second node.

Abstract: A computer system displays a user interface that includes a data flow pane, a profile pane, and a data pane. The data flow pane displays a flow diagram including a plurality of linked nodes. Each node specifies a respective operation and a respective intermediate data set generated upon execution of the respective operation. The profile pane displays schemas corresponding to selected nodes in the flow diagram, including (1) data elements showing information about data fields and (2) statistical information about data values for the data fields. User interaction to modify any of the data elements in the profile pane modifies the flow diagram displayed in the data flow pane. The data pane displays rows of intermediate data sets corresponding to selected nodes in the flow diagram. User interaction with individual data values from the displayed rows of data modifies the flow diagram displayed in the data flow pane.

Abstract: A computer system displays a user interface that includes a flow pane, a tool pane, a profile pane, and a data pane. The flow pane displays a node/link flow diagram that identifies data sources, operations, and output datasets. The tool pane includes a data source selector that enables users to add data sources to the flow diagram, and includes an operation palette that enables users to insert nodes into the flow diagram for performing specific transformation operations. The profile pane displays schemas corresponding to selected nodes in the flow diagram, including information about data fields and statistical information about data values for the data fields and enables users to modify the flow diagram by interacting with individual data elements. The data pane displays rows of data corresponding to selected nodes in the flow diagram, and enables users to modify the flow diagram by interacting with individual data values.

Abstract: An exemplary method includes displaying a UI that includes a data flow pane and a data grid pane, and in response to user selection of a node, displaying, in the data grid pane, a data grid corresponding to the user-selected node. The method further comprises receiving user input on a first cell in the data grid that changes a first data value v1 for the first cell (intersection between a first column and a first row of the data grid) to a first modified data value w1. The method further comprises determining a formula f(v1)=w1 corresponding to the input, inserting a new node into the flow diagram immediately after the user-selected node, and applying the formula f to the first column for each of the plurality of rows in the data grid, including changing a second data value v2 to a second modified data value w2, where v2?v1 and w2?w1.

Abstract: A computer system prepares data for subsequent analysis. The system displays a user interface that includes a data flow pane, a profile pane, and a data pane. The data flow pane displays a node/link diagram, including a first node representing a join between first and second data sources. When the first node is selected, the system updates the profile pane to display data value histograms for a plurality of data fields and updates the data pane to display rows from the data sources. The system also displays a join control region, including: join type selector; data field pairs that define the join between the data sources; and a join overview graphic including three portions, showing aggregate proportion of rows having an inner join, left outer join, or right outer join relationship. Selecting a portion from the join overview graphic updates the data value histograms and rows in the data pane.

Abstract: A method prepares data for analysis. A user interface includes a flow pane, which includes a flow diagram with nodes, each node having an intermediate set and specifying an operation. A first user input selects a first node, which specifies a first operation. In response, the method displays node internals for the first node, including a description of the first operation. A second user input combines a second node with the first node. The second node specifies a second operation. In response to the second user input, the method (i) removes the second node from the flow diagram, (ii) determines ordering for the first and second operations, (iii) updates the node internals for the first node to specify the first and second operations, ordered according to the ordering, and (iv) displays the updated node internals, including descriptions of the first and second operations, sequenced according to the determined ordering.

Abstract: A method is performed at a computer. The computer retrieves a dataset from a database. The dataset includes a plurality of fields, including a first field. The computer displays a data-preparation graphical user interface, which includes a first region displaying a grid with columns corresponding to the fields and a second region that displays a sequence of operations applied to the dataset. The computer receives a user action to specify a first filter for the first field. The computer removes rows from the grid whose corresponding data values in the first column do not satisfy the first filter. The computer identifies some additional rows from the dataset, not previously displayed in the grid, whose data values for the first field satisfy the first filter, and displays the additional rows in the grid. The computer inserts an operation identifier into the third region, which identifies the first filter.

Abstract: A computer system transforms data. A first user builds a flow diagram. Each node in the flow diagram specifies an operation: to retrieve data, to transform data, or to create an output dataset. The flow diagram includes a subgraph having a plurality of nodes. When the first user initiates execution, the system performs the operations in the subgraph sequentially as specified. When the first user requests the system to save the flow diagram, the system stores the subgraph. The system displays a user interface and the subgraph in the user interface. A second user requests the system to build another flow diagram based on the subgraph. In response, the system retrieves the subgraph from memory. When the second user requests the system to execute the flow diagram built by the second user, the system performs the operations of the nodes in the subgraph sequentially as specified in the subgraph.

Abstract: An exemplary method includes displaying a user interface that includes multiple panes, and receiving a first user input to select an existing node from a flow diagram or a flow element template. The system displays a moveable icon representing a new node for placement in the flow diagram. The new node specifies an operation corresponding to the selected existing node or the selected flow element template. Drop targets are displayed in the flow diagram according to dependencies between the data flow operation of the new node and operations of existing nodes. A second user input places the moveable icon over a first drop target. In response to ceasing to detect the second user input, the system inserts the new node into the flow diagram at the first drop target. The new node performs the specified data flow operation.

Abstract: A computer system transforms data. A first user builds a flow diagram. Each node in the flow diagram specifies an operation: to retrieve data, to transform data, or to create an output dataset. The flow diagram includes a subgraph having a plurality of nodes. When the first user initiates execution, the system performs the operations in the subgraph sequentially as specified. When the first user requests the system to save the flow diagram, the system stores the subgraph. The system displays a user interface and the subgraph in the user interface. A second user requests the system to build another flow diagram based on the subgraph. In response, the system retrieves the subgraph from memory. When the second user requests the system to execute the flow diagram built by the second user, the system performs the operations of the nodes in the subgraph sequentially as specified in the subgraph.

Abstract: A computer system transforms data. The system displays a user interface including a data flow pane. A user builds a flow diagram in the data flow pane. Each node in the flow diagram specifies an operation: to retrieve data, to transform data, or to create an output dataset. The flow diagram includes a subtree having a data source node and transformation operation nodes. When the user initiates execution and the nodes in the subtree are configured to execute imperatively, the system performs the operations in the subtree sequentially as specified, retrieving data from the data source, transforming the data, and forming an intermediate dataset. When the user initiates execution and the nodes in the subtree are configured to execute declaratively, the system constructs a database query that is logically equivalent to the operations specified in the subtree and transmits the query to the data source to retrieve the intermediate dataset.

Abstract: A computer system prepares data for subsequent analysis. The system displays a user interface that includes a data flow pane, a profile pane, and a data pane. The data flow pane displays a node/link diagram, including a first node representing a join between first and second data sources. When the first node is selected, the system updates the profile pane to display data value histograms for a plurality of data fields and updates the data pane to display rows from the data sources. The system also displays a join control region, including: join type selector; data field pairs that define the join between the data sources; and a join overview graphic including three portions, showing aggregate proportion of rows having an inner join, left outer join, or right outer join relationship. Selecting a portion from the join overview graphic updates the data value histograms and rows in the data pane.

Abstract: A method is performed at a computer. The computer retrieves a dataset from a database. The dataset includes a plurality of fields, including a first field. The computer displays a data-preparation graphical user interface, which includes a first region displaying a grid with columns corresponding to the fields and a second region that displays a sequence of operations applied to the dataset. The computer receives a user action to specify a first filter for the first field. The computer removes rows from the grid whose corresponding data values in the first column do not satisfy the first filter. The computer identifies some additional rows from the dataset, not previously displayed in the grid, whose data values for the first field satisfy the first filter, and displays the additional rows in the grid. The computer inserts an operation identifier into the third region, which identifies the first filter.

Abstract: An exemplary method includes displaying a user interface that includes multiple panes, and receiving a first user input to select an existing node from a flow diagram or a flow element template. The system displays a moveable icon representing a new node for placement in the flow diagram. The new node specifies an operation corresponding to the selected existing node or the selected flow element template. Drop targets are displayed in the flow diagram according to dependencies between the data flow operation of the new node and operations of existing nodes. A second user input places the moveable icon over a first drop target. In response to ceasing to detect the second user input, the system inserts the new node into the flow diagram at the first drop target. The new node performs the specified data flow operation.

Abstract: A computer system transforms data. The system displays a user interface including a data flow pane. A user builds a flow diagram in the data flow pane. Each node in the flow diagram specifies an operation: to retrieve data, to transform data, or to create an output dataset. The flow diagram includes a subtree having a data source node and transformation operation nodes. When the user initiates execution and the nodes in the subtree are configured to execute imperatively, the system performs the operations in the subtree sequentially as specified, retrieving data from the data source, transforming the data, and forming an intermediate dataset. When the user initiates execution and the nodes in the subtree are configured to execute declaratively, the system constructs a database query that is logically equivalent to the operations specified in the subtree and transmits the query to the data source to retrieve the intermediate dataset.

Abstract: An exemplary method includes displaying a UI that includes a data flow pane and a data grid pane, and in response to user selection of a node, displaying, in the data grid pane, a data grid corresponding to the user-selected node. The method further comprises receiving user input on a first cell in the data grid that changes a first data value v1 for the first cell (intersection between a first column and a first row of the data grid) to a first modified data value w1. The method further comprises determining a formula f(v1)=w1 corresponding to the input, inserting a new node into the flow diagram immediately after the user-selected node, and applying the formula f to the first column for each of the plurality of rows in the data grid, including changing a second data value v2 to a second modified data value w2, where v2?v1 and w2?w1.

Abstract: A computer system displays a user interface that includes a flow pane, a tool pane, a profile pane, and a data pane. The flow pane displays a node/link flow diagram that identifies data sources, operations, and output datasets. The tool pane includes a data source selector that enables users to add data sources to the flow diagram, and includes an operation palette that enables users to insert nodes into the flow diagram for performing specific transformation operations. The profile pane displays schemas corresponding to selected nodes in the flow diagram, including information about data fields and statistical information about data values for the data fields and enables users to modify the flow diagram by interacting with individual data elements. The data pane displays rows of data corresponding to selected nodes in the flow diagram, and enables users to modify the flow diagram by interacting with individual data values.

Abstract: A software debugging environment uses multiple abstract views to provide the flexibility required to perform effective debugging on an executing software program using data structures. Developers can select between one or more views to see different aspects of the software being examined. Each view can have a filter with specific properties determining which quantities are displayed, the format they are displayed in and any editing capability. In some embodiments, the selection of views, filters defining the views, and the editing properties for the contents shown in the views may be set interactively. In some cases, this interaction occurs through the interface of an Integrated Development Environment (IDE) containing the debugger. In other cases, the properties of views in the debugging environment are determined by definitions supplied in configuration files. Special facilities are provided for the debugging of JSP programs.