Abstract: Generation of a performance determination report for an Extract, Transform, Load (ETL) job includes decomposing the ETL job into two or more stage instances, and identifying one or more conditions for each of the stage instances. A set of tests for each of the identified conditions are generated. A first set of test results are generated by performing the set of tests. It is determined whether a test result from the first set of test results is outside of a first range. Conditions that can be identified include a non-volatile free memory condition, a network reliability condition, a network configuration condition, an application availability condition, a database availability condition, a database performance condition, a schema validity condition, an installed libraries condition, a configuration parameter condition, a volatile free memory condition, and a third party tool condition.

Abstract: Provided is a method for processing input data in a storage system and in communication with a repository. Views are generated that comprise a tree of nodes selected from a subset of nodes in a hierarchical representation of a schema. The views are saved to the repository. At least one of the views are used to create a job comprising a sequence of data transformation steps to transform the input data described by input schemas to the output data described by output schemas.

Abstract: According to one embodiment of the present invention, a method automatically generates one or more Extract, Transform and Load (ETL) jobs. Input data in a source format and output data in a target format is received. The input data and output data is analyzed to determine properties and relationships thereof. One or more mapping models are automatically generated using the properties and relationships, wherein the mapping models describe the mapping and transformation of the input data to the output data. One or more ETL jobs are generated using the mapping models. Embodiments further include a system and program product apparatus for automatically generating one or more ETL jobs.

Abstract: Techniques are disclosed for consuming and integrating with representational state transfer (REST) based services. In one embodiment, a multiple-step extract, transform, load (ETL) job includes one or more REST steps. Each step in the ETL job performs a particular well-defined task, and the REST step in particular constructs REST requests, invokes REST services, and retrieves REST responses. The REST step may be mapped to a vector of inputs such that multiple REST service calls are made for each of the inputs. Further, a reusable connection may be employed so that one connection can be used to make multiple REST service calls. URLs used to make REST service calls may be dynamically constructed based on input data parameters and ETL job parameters. Additionally, a graphical user interface (GUI) allows users to configure aspects of a REST step.

Abstract: Provided are a computer program product, system, and method for generating a view for a schema. A schema provides a definition of elements, wherein at least one of the elements comprises a recursive element of a recursive data type, wherein the recursive data type allows for instances of the recursive element to include instances of that same recursive data type at different levels in a hierarchy. Indication is received to transform the recursive types to non-recursive structures. A view for the schema is generated including information on the indication to transform the recursive types to non-recursive structure, wherein the processing of the data defined by the view causes a parser to generate a structured element for each instance of the recursive element in schema data being parsed, wherein the generated structured element includes a unique identifier of the instance of the recursive element.

Abstract: Provided are techniques for auto-generating Representational State Transfer (REST) services for quality assurance. One or more test cases and artifacts are received for a project. A test Representational State Transfer (REST) service is generated for the project using the one or more test cases and the artifacts. The test REST service is deployed on an application server for use in testing features of a REST service client application.

Abstract: According to one embodiment of the present invention, a method automatically generates one or more Extract, Transform and Load (ETL) jobs. Input data in a source format and output data in a target format is received. The input data and output data is analyzed to determine properties and relationships thereof. One or more mapping models are automatically generated using the properties and relationships, wherein the mapping models describe the mapping and transformation of the input data to the output data. One or more ETL jobs are generated using the mapping models. Embodiments further include a system and program product apparatus for automatically generating one or more ETL jobs.

Abstract: Techniques are disclosed to determine data quality issues in data processing jobs. The data processing job is received, the data processing job specifying one or more processing steps designed based on one or more data schemas and further specifies one or more desired quality metrics to measure at the one or more processing steps. One or more state machines are provided, that are generated based on the quality metrics and on the data schemas. Input data to the data process job are processed using the one or more state machines, in order to generate output data and a set of data quality records characterizing a set of data quality issues identified during the execution of the data processing job.

Abstract: Techniques are disclosed to determine data quality issues in data processing jobs. The data processing job is received, the data processing job specifying one or more processing steps designed based on one or more data schemas and further specifies one or more desired quality metrics to measure at the one or more processing steps. One or more state machines are provided, that are generated based on the quality metrics and on the data schemas. Input data to the data process job are processed using the one or more state machines, in order to generate output data and a set of data quality records characterizing a set of data quality issues identified during the execution of the data processing job.

Abstract: Provided are a computer program product, system, and method for limiting the rendering of instances of recursive elements in view output. A schema provides a definition of elements, wherein at least one of the elements comprises a recursive element of a recursive data type, wherein the recursive data type allows for instances of the recursive element to include instances of that same recursive data type at different levels in a hierarchy. A condition is received for terminating recursion for the recursive element. A view is generated for the schema including information on the condition for terminating the recursion, wherein the processing of the view causes a parser to chunk instances of the recursive element in schema data in response to the condition indicating to terminate recursion at the level of the instance of the recursive element being processed.

Abstract: Provided are a computer program product, system, and method for limiting the rendering of instances of recursive elements in view output. A schema provides a definition of elements, wherein at least one of the elements comprises a recursive element of a recursive data type, wherein the recursive data type allows for instances of the recursive element to include instances of that same recursive data type at different levels in a hierarchy. A condition is received for terminating recursion for the recursive element. A view is generated for the schema including information on the condition for terminating the recursion, wherein the processing of the view causes a parser to chunk instances of the recursive element in schema data in response to the condition indicating to terminate recursion at the level of the instance of the recursive element being processed.

Abstract: Generation of a performance determination report for an Extract, Transform, Load (ETL) job includes decomposing the ETL job into two or more stage instances, and identifying one or more conditions for each of the stage instances. A set of tests for each of the identified conditions are generated. A first set of test results are generated by performing the set of tests. It is determined whether a test result from the first set of test results is outside of a first range. Conditions that can be identified include a non-volatile free memory condition, a network reliability condition, a network configuration condition, an application availability condition, a database availability condition, a database performance condition, a schema validity condition, an installed libraries condition, a configuration parameter condition, a volatile free memory condition, and a third party tool condition.

Abstract: Techniques describe generating an end-to-end web application. A user interface (UI) design tool generates a first application that includes a UI having one or more UI elements. One or more classifications of server properties (e.g., REST resource names, JSON names, CRUD actions, etc.) are mapped to each of the UI elements. A data schema is generated from the classifications. A machine-readable description of the data schema and mapped classifications is generated. The UI design tool generates a second application from the machine-readable description. The second application is an implementation of the UI of the first application and the machine-readable description.

Abstract: A computational device receives input data and control data, where the control data includes instructions to modify one or more operations performed during a runtime execution associated with the input data. The control data is processed to modify the one or more operations during the runtime execution associated of the input data.

Abstract: A computational device receives input data and control data, where the control data includes instructions to modify one or more operations performed during a runtime execution associated with the input data. The control data is processed to modify the one or more operations during the runtime execution associated of the input data.

Abstract: Provided are techniques for interactive social media access to data systems. A social media gateway interface with adapters for each of multiple social media applications is provided. An event is published from a data system to at least one of the multiple social media applications. A social media bot is created to represent the data system. The social media bot and one of the adapters are used to enable the data system to interact with at least one of the multiple social media applications.

Abstract: Provided are techniques for personalizing data system components and data sources as chatbots in a group chat session. A social media gateway interface with a chat adapter for a chat application is provided. One or more chatbots that each represent a corresponding component of a data system are created. The one or more chatbots and the chatbot adapter are used to manage each corresponding component of the data system.

Abstract: Provided are techniques for social media bot to Representational State Transfer (REST) proxy for data systems. An adapter for a social media application and message to Representational State Transfer (REST) mappings are provided. A social media bot is created to represent a data system component. The social media bot and the adapter are used to enable the data system component to interact with the social media application by using the message to REST mappings to map social media messages for the social media application and REST calls that are used to access the data system component.

Abstract: Provided are techniques for personalizing data system components and data sources as chatbots in a group chat session. A social media gateway interface with a chat adapter for a chat application is provided. One or more chatbots that each represent a corresponding component of a data system are created. The one or more chatbots and the chatbot adapter are used to manage each corresponding component of the data system.

Abstract: Provided are techniques for social media bot to Representational State Transfer (REST) proxy for data systems. An adapter for a social media application and message to Representational State Transfer (REST) mappings are provided. A social media bot is created to represent a data system component. The social media bot and the adapter are used to enable the data system component to interact with the social media application by using the message to REST mappings to map social media messages for the social media application and REST calls that are used to access the data system component.