Abstract: Embodiments include a method and system for supporting a common data processing definition across multiple data processing nodes in a data swamp comprising at least one local data storage system and one or more remote data storage systems including remote databases, cloud-based data storage, or one or more computational clusters. Users can create a document in a common data processing language (e.g., XML) that describes a set of data processing tasks and a set of data processing resources to perform the data processing tasks in a graphical interface. The interface can then generate the document and send it to remote agents located in close proximity to one or more of the data processing nodes to access the necessary data and processing resources to execute the processing tasks at the appropriate node where the data and resources are located without having to migrate the data for processing.

Abstract: The present disclosure includes techniques pertaining to computer implemented systems and methods for automatic generation of data transformations. In one embodiment, a user manipulates a spreadsheet of data in a user interface. On a backend, the user's manipulations trigger actions that cause the data to be modified. Actions may automatically cause data flow objects and data flow graphs to be produced. The transformations defined by the data flow graphs are automatically executed by a software engine and the results are displayed to the user. The user may access and manipulate data from a variety of data sources while the underlying complexities of the transformation process are performed in an automated manner.

Abstract: Embodiments include a method and system for supporting a common data processing definition across multiple data processing nodes in a data swamp comprising at least one local data storage system and one or more remote data storage systems including remote databases, cloud-based data storage, or one or more computational clusters. Users can create a document in a common data processing language (e.g., XML) that describes a set of data processing tasks and a set of data processing resources to perform the data processing tasks in a graphical interface. The interface can then generate the document and send it to remote agents located in close proximity to one or more of the data processing nodes to access the necessary data and processing resources to execute the processing tasks at the appropriate node where the data and resources are located without having to migrate the data for processing.

Abstract: The present disclosure includes techniques pertaining to computer implemented systems and methods for transforming data. In one embodiment, data sources are specified by a user and copies of specified data are received in local storage. Data is presented to the user as a spreadsheet of columns and rows and the user manipulates the data in the spreadsheet. The user manipulations are automatically translated into data flow objects and a data flow graph, which is executed to transform the data. The modified data is presented to the user.

Abstract: The present disclosure includes techniques pertaining to computer implemented systems and methods for transforming data. In one embodiment, data sources are specified by a user and copies of specified data are received in local storage. Data is presented to the user as a spreadsheet of columns and rows and the user manipulates the data in the spreadsheet. The user manipulations are automatically translated into data flow objects and a data flow graph, which is executed to transform the data. The modified data is presented to the user.

Abstract: The present disclosure includes techniques pertaining to computer implemented systems and methods for automatic generation of data transformations. In one embodiment, a user manipulates a spreadsheet of data in a user interface. On a backend, the user's manipulations trigger actions that cause the data to be modified. Actions may automatically cause data flow objects and data flow graphs to be produced. The transformations defined by the data flow graphs are automatically executed by a software engine and the results are displayed to the user. The user may access and manipulate data from a variety of data sources while the underlying complexities of the transformation process are performed in an automated manner.