Abstract: A computer-implemented system and method for data revision control in a large-scale data analytic systems. In one embodiment, for example, a computer-implemented method comprises the operations of storing a first version of a dataset that is derived by executing a first version of driver program associated with the dataset; and storing a first build catalog entry comprising an identifier of the first version of the dataset and comprising an identifier of the first version of the driver program.

Abstract: In some embodiments, a method comprises obtaining a pipeline of operations, the pipeline of operations including a plurality of functions providing any of one or more modification operations or visualization operations for a plurality of datasets. A first dynamic visualization of the pipeline of operations at a first level of granularity is generated. A second dynamic visualization of the pipeline of operations at a second level of granularity is generated in response to user input.

Abstract: A resource dependency system may track data dependencies and data transformations for individual columns of the data sets over the span of the data pipeline (referred to as a provenance or lineage of a column). Column provenance/lineage can be logged using metadata or graph-like data structures, which the resource dependency system can generate, store, manage, and access. Column provenance/lineage can be used to generate user interfaces displaying visual node graphs with columns as nodes and the data dependencies and data transformations associated with the columns as edges between the nodes.

Abstract: A computer-implemented system and method for data revision control in a large-scale data analytic systems. In one embodiment, for example, a computer-implemented method comprises the operations of storing a first version of a dataset that is derived by executing a first version of driver program associated with the dataset; and storing a first build catalog entry comprising an identifier of the first version of the dataset and comprising an identifier of the first version of the driver program.

Abstract: A resource dependency system displays two dynamically interactive interfaces in a resource dependency user interface, a hierarchical resource repository and a dependency graph user interface. User interactions on each interface can dynamically update either interface. For example, a selection of a particular resource in the dependency graph user interface causes the system to update the dependency graph user interface to indicate the selection and also updates the hierarchical resource repository to navigate to the appropriate folder corresponding to the stored location of the selected resource. In another example, a selection of a particular resource in the hierarchical resource repository causes the system to update the hierarchical resource repository to indicate the selection and also updates the dependency graph user interface to display an updated graph, indicate the selection and, in some embodiments, focus on the selected resource by zooming into a portion of the graph.

Abstract: Systems and methods are provided for creating and managing a data integration workspace. The workspace may comprise one or more views of data (or datasets) stored in or accessible by the system. Models may be generated and updated based on the plurality of datasets and presented via a graphical user interface. Feedback received via a graphical user interface presenting a model may be used to annotate an underlying dataset associated with the model. Responsive to a modification of the underlying dataset or the rules for using the underlying dataset to generate the model, other related datasets and/or models may be automatically updated accordingly. Templates associated with one or more types of users may be defined. Each template may comprise one or more specific models related to a specific type of user.

Abstract: Systems, methods, and non-transitory computer readable media are provided for providing parameterized states. Resource customization information may be obtained. The resource customization information may identify a resource and define a customized view of the resource. A parameterized state may be generated based on the resource customization information. Activation of the parameterized state may cause the customized view of the resource to be presented. An interface through which the parameterized state is accessible may be provided.

Abstract: A resource dependency system displays two dynamically interactive interfaces in a resource dependency user interface, a hierarchical resource repository and a dependency graph user interface. User interactions on each interface can dynamically update either interface. For example, a selection of a particular resource in the dependency graph user interface causes the system to update the dependency graph user interface to indicate the selection and also updates the hierarchical resource repository to navigate to the appropriate folder corresponding to the stored location of the selected resource. In another example, a selection of a particular resource in the hierarchical resource repository causes the system to update the hierarchical resource repository to indicate the selection and also updates the dependency graph user interface to display an updated graph, indicate the selection and, in some embodiments, focus on the selected resource by zooming into a portion of the graph.

Abstract: A resource dependency system may track data dependencies and data transformations for individual columns of the data sets over the span of the data pipeline (referred to as a provenance or lineage of a column). Column provenance/lineage can be logged using metadata or graph-like data structures, which the resource dependency system can generate, store, manage, and access. Column provenance/lineage can be used to generate user interfaces displaying visual node graphs with columns as nodes and the data dependencies and data transformations associated with the columns as edges between the nodes.

Abstract: Systems and methods are provided for improved graphical user interfaces. The system enables multiple separate applications, each of which may typically be in their own separate window or tab, to be interacted within a single window, such as a tab of a web browser application. The main window includes smaller sub-windows that can correspond to a distinct application with its own graphical user interface. A large sub-window within the main window is opened for the primary application where the user is currently interacting with a graphical user interface of the primary application. The user then is able to switch between applications (all within the same main window) and applications that are no longer being used can be minimized in smaller sub-windows off to the side of the primary sub-window. The system enables a user to drag and drop an item from one sub-window to another sub-window. Some of the interactions between the windows and data transformations are stored and can be visually presented in a graph.

Abstract: Systems, methods, and non-transitory computer readable media are provided for providing parameterized states. Resource customization information may be obtained. The resource customization information may identify a resource and define a customized view of the resource. A parameterized state may be generated based on the resource customization information. Activation of the parameterized state may cause the customized view of the resource to be presented. An interface through which the parameterized state is accessible may be provided.

Abstract: Systems and methods are provided for creating and managing a data integration workspace. The workspace may comprise one or more views of data (or datasets) stored in or accessible by the system. Models may be generated and updated based on the plurality of datasets and presented via a graphical user interface. Feedback received via a graphical user interface presenting a model may be used to annotate an underlying dataset associated with the model. Responsive to a modification of the underlying dataset or the rules for using the underlying dataset to generate the model, other related datasets and/or models may be automatically updated accordingly. Templates associated with one or more types of users may be defined. Each template may comprise one or more specific models related to a specific type of user.

Abstract: Systems and methods are provided for improved graphical user interfaces. The system enables multiple separate applications, each of which may typically be in their own separate window or tab, to be interacted within a single window, such as a tab of a web browser application. The main window includes smaller sub-windows that can correspond to a distinct application with its own graphical user interface. A large sub-window within the main window is opened for the primary application where the user is currently interacting with a graphical user interface of the primary application. The user then is able to switch between applications (all within the same main window) and applications that are no longer being used can be minimized in smaller sub-windows off to the side of the primary sub-window. The system enables a user to drag and drop an item from one sub-window to another sub-window. Some of the interactions between the windows and data transformations are stored and can be visually presented in a graph.

Abstract: Techniques for propagation of deletion operations among a plurality of related datasets are described herein.

Abstract: Systems, methods, and non-transitory computer readable media are provided for providing parameterized states. Resource customization information may be obtained. The resource customization information may identify a resource and define a customized view of the resource. A parameterized state may be generated based on the resource customization information. Activation of the parameterized state may cause the customized view of the resource to be presented. An interface through which the parameterized state is accessible may be provided.

Abstract: A resource dependency system displays two dynamically interactive interfaces in a resource dependency user interface, a hierarchical resource repository and a dependency graph user interface. User interactions on each interface can dynamically update either interface. For example, a selection of a particular resource in the dependency graph user interface causes the system to update the dependency graph user interface to indicate the selection and also updates the hierarchical resource repository to navigate to the appropriate folder corresponding to the stored location of the selected resource. In another example, a selection of a particular resource in the hierarchical resource repository causes the system to update the hierarchical resource repository to indicate the selection and also updates the dependency graph user interface to display an updated graph, indicate the selection and, in some embodiments, focus on the selected resource by zooming into a portion of the graph.

Abstract: Systems and methods are provided for improved graphical user interfaces. The system enables multiple separate applications, each of which may typically be in their own separate window or tab, to be interacted within a single window, such as a tab of a web browser application. The main window includes smaller sub-windows that can correspond to a distinct application with its own graphical user interface. A large sub-window within the main window is opened for the primary application where the user is currently interacting with a graphical user interface of the primary application. The user then is able to switch between applications (all within the same main window) and applications that are no longer being used can be minimized in smaller sub-windows off to the side of the primary sub-window. The system enables a user to drag and drop an item from one sub-window to another sub-window. Some of the interactions between the windows and data transformations are stored and can be visually presented in a graph.

Abstract: Systems and methods are provided for creating and managing a data integration workspace. The workspace may comprise one or more views of data (or datasets) stored in or accessible by the system. Models may be generated and updated based on the plurality of datasets and presented via a graphical user interface. Feedback received via a graphical user interface presenting a model may be used to annotate an underlying dataset associated with the model. Responsive to a modification of the underlying dataset or the rules for using the underlying dataset to generate the model, other related datasets and/or models may be automatically updated accordingly. Templates associated with one or more types of users may be defined. Each template may comprise one or more specific models related to a specific type of user.

Abstract: A computer-implemented system and method for data revision control in a large-scale data analytic systems. In one embodiment, for example, a computer-implemented method comprises the operations of storing a first version of a dataset that is derived by executing a first version of driver program associated with the dataset; and storing a first build catalog entry comprising an identifier of the first version of the dataset and comprising an identifier of the first version of the driver program.