Abstract: Mining data transformation flows in spreadsheets includes identifying operations defined in a spreadsheet, identifying source data, in the spreadsheet, on which the operations operate, automatically creating an extract, transform, load (ETL) data transformation flow, and executing the created ETL data transformation flow. Creating the ETL data transformation flow includes selecting, in the ETL system, source data endpoint(s) for data extraction, selecting target data endpoint(s) for data loading, mapping at least one of the identified operations to ETL operation(s) for data transformation, and building the ETL data transformation flow, which defines extraction from the selected source data endpoint(s), transformation based on the ETL operation(s), and loading to the selected target data endpoint(s).

Abstract: A hybrid cluster environment with a public cloud cluster having nodes storing data and a plurality of private clusters is provided, wherein each of the plurality of private clusters has nodes storing data. Registration data that indicates a customer identifier, a new private cluster, and a file transfer server is received. The new private cluster is added to the plurality of private clusters in the hybrid cluster environment. Input to design a job to process data in the hybrid cluster environment is received. It is determined that the job is to be deployed to the new private cluster. The job is deployed to the new private cluster using the file transfer server, wherein the job is executed at the new private cluster. Job status information and one or more job logs are received with the file transfer server.

Abstract: A hybrid cluster environment with a public cloud cluster having nodes storing data and a plurality of private clusters is provided, wherein each of the plurality of private clusters has nodes storing data. Registration data that indicates a customer identifier, a new private cluster, and a file transfer server is received. The new private cluster is added to the plurality of private clusters in the hybrid cluster environment. Input to design a job to process data in the hybrid cluster environment is received. It is determined that the job is to be deployed to the new private cluster. The job is deployed to the new private cluster using the file transfer server, wherein the job is executed at the new private cluster. Job status information and one or more job logs are received with the file transfer server.

Abstract: Mining data transformation flows in spreadsheets includes identifying operations defined in a spreadsheet, identifying source data, in the spreadsheet, on which the operations operate, automatically creating an extract, transform, load (ETL) data transformation flow, and executing the created ETL data transformation flow. Creating the ETL data transformation flow includes selecting, in the ETL system, source data endpoint(s) for data extraction, selecting target data endpoint(s) for data loading, mapping at least one of the identified operations to ETL operation(s) for data transformation, and building the ETL data transformation flow, which defines extraction from the selected source data endpoint(s), transformation based on the ETL operation(s), and loading to the selected target data endpoint(s).

Abstract: A mechanism is provided for applying matching data transformation information based on a user's editing of data within a document. User input identifying inputs provided by a user while editing a document within an application executing on the data processing system is received. A matching algorithm is executed based on user input to identify one or more candidate transformation operations within a transformation operation data structure that matches the user input. Responsive to failing to identify any candidate transformation operations, an indication is provided that no candidate transformation operations are identifiable. Responsive to one or more candidate transformation operations being identified, a list of transformation operations is provided that includes the one or more candidate transformation operations to the user via the data processing system.

Abstract: A hybrid cluster environment with a public cloud cluster having nodes storing data and a plurality of private clusters is provided, wherein each of the plurality of private clusters has nodes storing data. Registration data that indicates a customer identifier, a new private cluster, and a file transfer server is received. The new private cluster is added to the plurality of private clusters in the hybrid cluster environment. Input to design a job to process data in the hybrid cluster environment is received. It is determined that the job is to be deployed to the new private cluster. The job is deployed to the new private cluster using the file transfer server, wherein the job is executed at the new private cluster. Job status information and one or more job logs are received with the file transfer server.

Abstract: A hybrid cluster environment with a public cloud cluster having nodes storing data and a plurality of private clusters is provided, wherein each of the plurality of private clusters has nodes storing data. Registration data that indicates a customer identifier, a new private cluster, and a file transfer server is received. The new private cluster is added to the plurality of private clusters in the hybrid cluster environment. Input to design a job to process data in the hybrid cluster environment is received. It is determined that the job is to be deployed to the new private cluster. The job is deployed to the new private cluster using the file transfer server, wherein the job is executed at the new private cluster. Job status information and one or more job logs are received with the file transfer server.

Abstract: A mechanism is provided for applying matching data transformation information based on a user's editing of data within a document. User input identifying inputs provided by a user while editing a document within an application executing on the data processing system is received. A matching algorithm is executed based on user input to identify one or more candidate transformation operations within a transformation operation data structure that matches the user input. Responsive to failing to identify any candidate transformation operations, an indication is provided that no candidate transformation operations are identifiable. Responsive to one or more candidate transformation operations being identified, a list of transformation operations is provided that includes the one or more candidate transformation operations to the user via the data processing system.

Abstract: A mechanism is provided for applying matching data transformation information based on a user's editing of data within a document. User input identifying inputs provided by a user while editing a document within an application executing on the data processing system is received. A matching algorithm is executed based on user input to identify one or more candidate transformation operations within a transformation operation data structure that matches the user input. Responsive to failing to identify any candidate transformation operations, an indication is provided that no candidate transformation operations are identifiable. Responsive to one or more candidate transformation operations being identified, a list of transformation operations is provided that includes the one or more candidate transformation operations to the user via the data processing system.

Abstract: A mechanism is provided for applying matching data transformation information based on a user's editing of data within a document. User input identifying inputs provided by a user while editing a document within an application executing on the data processing system is received. A matching algorithm is executed based on user input to identify one or more candidate transformation operations within a transformation operation data structure that matches the user input. Responsive to failing to identify any candidate transformation operations, an indication is provided that no candidate transformation operations are identifiable. Responsive to one or more candidate transformation operations being identified, a list of transformation operations is provided that includes the one or more candidate transformation operations to the user via the data processing system.

Abstract: A user-defined function (UDF) is received in a central Computer System, which causes registration of the UDF and distributes the UDF to a cluster of computer system nodes configured for performing, in volatile memory of the nodes, extract-transform-load processing of data cached in the volatile memory of the nodes. First and second job specifications that include the UDF are received by the central Computer System, and the central computer system distributes instructions for the job specifications to the nodes including at least one instruction that invokes the UDF for loading and executing the UDF in the volatile memory of at least one of the nodes during runtime of the jobs. The central Computer System does not cause registration of the UDF again after receiving the first job specification.

Abstract: A user-defined function (UDF) is received in a central Computer System, which causes registration of the UDF and distributes the UDF to a cluster of computer system nodes configured for performing, in volatile memory of the nodes, extract-transform-load processing of data cached in the volatile memory of the nodes. First and second job specifications that include the UDF are received by the central Computer System, and the central computer system distributes instructions for the job specifications to the nodes including at least one instruction that invokes the UDF for loading and executing the UDF in the volatile memory of at least one of the nodes during runtime of the jobs. The central Computer System does not cause registration of the UDF again after receiving the first job specification.

Abstract: Correspondence is determined among i) groups in a first version of one or more groups of related tables and ii) groups in a second version of one or more groups of related tables. The tables have schema and table names and determining the correspondence includes matching groups that have matching tables, wherein the matching tables have matching schema and table names. A user interface is displayed, including first and second portions. A group name list is displayed for the one or more groups of the first version in the first user interface portion and of the second version in the second portion. A list of one or more tables within each respectively displayed group is displayed. Within each group of the first version the one or more table names are in visual alignment with respective matching one or more table names of the respective corresponding group of the second version.

Abstract: Correspondence is determined among i) groups in a first version of one or more groups of related tables and ii) groups in a second version of one or more groups of related tables. The tables have schema and table names and determining the correspondence includes matching groups that have matching tables, wherein the matching tables have matching schema and table names. A user interface is displayed, including first and second portions. A group name list is displayed for the one or more groups of the first version in the first user interface portion and of the second version in the second portion. A list of one or more tables within each respectively displayed group is displayed. Within each group of the first version the one or more table names are in visual alignment with respective matching one or more table names of the respective corresponding group of the second version.

Abstract: Correspondence is determined among i) groups in a first version of one or more groups of related tables and ii) groups in a second version of one or more groups of related tables. The tables have schema and table names and determining the correspondence includes matching groups that have matching tables, wherein the matching tables have matching schema and table names. A user interface is displayed, including first and second portions. A group name list is displayed for the one or more groups of the first version in the first user interface portion and of the second version in the second portion. A list of one or more tables within each respectively displayed group is displayed. Within each group of the first version the one or more table names are in visual alignment with respective corresponding one or more table names of the respective corresponding group of the second version.

Abstract: Correspondence is determined among i) groups in a first version of one or more groups of related tables and ii) groups in a second version of one or more groups of related tables. The tables have schema and table names and determining the correspondence includes matching groups that have matching tables, wherein the matching tables have matching schema and table names. A user interface is displayed, including first and second portions. A group name list is displayed for the one or more groups of the first version in the first user interface portion and of the second version in the second portion. A list of one or more tables within each respectively displayed group is displayed. Within each group of the first version the one or more table names are in visual alignment with respective corresponding one or more table names of the respective corresponding group of the second version.