Abstract: To make a connection between a user application and a data source fault tolerant, a user may use a wrapper generator to create an API wrapper that retries request that may have failed due to a service disruption. Generally, the wrapper manages a cache and contains the necessary information for retrying a connection after a service disruption. To create the wrapper, the wrapper generator may identify the different commands used by the user application to access the data source. The wrapper generator presents these commands to the user who organizes the commands to generate usage patterns for the different requests the user application may send to the data source. The user may also define a caching preference associated the commands in the usage patterns. To restart the connection, the wrapper may use the cache data to generate a new request to complete the original failed request.

Abstract: Techniques are disclosed to determine data quality issues in data processing jobs. The data processing job is received, the data processing job specifying one or more processing steps designed based on one or more data schemas and further specifies one or more desired quality metrics to measure at the one or more processing steps. One or more state machines are provided, that are generated based on the quality metrics and on the data schemas. Input data to the data process job are processed using the one or more state machines, in order to generate output data and a set of data quality records characterizing a set of data quality issues identified during the execution of the data processing job.

Abstract: According to one embodiment of the present invention, a system decomposes a set of schema files. The system receives a set of schema files and automatically identifies a plurality of root schema files in the set, where a root schema file is determined based on remaining schema files in the set lacking a reference to that schema file. For each root schema file, the system creates a subset of the original set of schema files. The subset contains the root schema file, and at least one subset further includes one or more schema files that provide information for that root schema file. Embodiments of the present invention further include a method and computer program product for decomposing a set of schema files in substantially the same manners described above.

Abstract: A computer receives a first JSON data that includes at least one JSON array or JSON object value. The computer parses a stream of JSON data, wherein the stream of JSON data includes at least a part of the first JSON data. The computer determines the logical structure of the first JSON data using the parsed stream of JSON data. The computer generates an XML schema based on the logical structure of the first JSON data.

Abstract: Provided are a computer program product, system, and method for processing schemas in a storage system. A presentation of a schema in a graphical user interface (GUI) is comprised of multiple type nodes in a tree structure. Each type node comprises a hierarchical arrangement of a plurality of nodes including group nodes including a plurality of nodes and content nodes providing values. First user input selects one of the type nodes in the schema for a view. Second user input selects one of the nodes in the selected type node in the schema for a view. Third user input selects a node in the schema to indicate a root node of the schema for the view. The view includes the root node and at least one sub node of the root node and is added as a child to the selected type node.

Abstract: Provided are a computer program product, system, and method for processing input data in a storage system and in communication with a repository. Views are generated that comprise a tree of nodes selected from a subset of nodes in a hierarchical representation of a schema. The views are saved to the repository. At least one of the views are used to create a job comprising a sequence of data transformation steps to transform the input data described by input schemas to the output data described by output schemas.

Abstract: Provided is a method for processing input data in a storage system and in communication with a repository. Views are generated that comprise a tree of nodes selected from a subset of nodes in a hierarchical representation of a schema. The views are saved to the repository. At least one of the views are used to create a job comprising a sequence of data transformation steps to transform the input data described by input schemas to the output data described by output schemas.

Abstract: Provided is a method for processing input data in a storage system and in communication with a repository. Views are generated that comprise a tree of nodes selected from a subset of nodes in a hierarchical representation of a schema. The views are saved to the repository. At least one of the views are used to create a job comprising a sequence of data transformation steps to transform the input data described by input schemas to the output data described by output schemas.

Abstract: Provided are a computer program product, system, and method for processing schemas in a storage system. A presentation of a schema in a graphical user interface (GUI) is comprised of multiple type nodes in a tree structure. Each type node comprises a hierarchical arrangement of a plurality of nodes including group nodes including a plurality of nodes and content nodes providing values. First user input selects one of the type nodes in the schema for a view. Second user input selects one of the nodes in the selected type node in the schema for a view. Third user input selects a node in the schema to indicate a root node of the schema for the view. The view includes the root node and at least one sub node of the root node and is added as a child to the selected type node.

Abstract: Provided is a method for processing schemas in a storage system. A presentation of a schema in a graphical user interface (GUI) is comprised of multiple type nodes in a tree structure. Each type node comprises a hierarchical arrangement of a plurality of nodes including group nodes including a plurality of nodes and content nodes providing values. First user input selects one of the type nodes in the schema for a view. Second user input selects one of the nodes in the selected type node in the schema for a view. Third user input selects a node in the schema to indicate a root node of the schema for the view. The view includes the root node and at least one sub node of the root node and is added as a child to the selected type node.

Abstract: Provided are a computer program product, system, and method for processing input data in a storage system and in communication with a repository. Views are generated that comprise a tree of nodes selected from a subset of nodes in a hierarchical representation of a schema. The views are saved to the repository. At least one of the views are used to create a job comprising a sequence of data transformation steps to transform the input data described by input schemas to the output data described by output schemas.