Abstract: An approach to implement an extract, transform and load system, a queue is provided for holding units of data between extraction and transformation. When units of data suffer a rate limit error in the load phase, they are requeued so they can be resubmitted for transformation. The contents of the queue are monitored and, if too many requeued units of data are detected in the queue, then this is taken as an indicator of an unacceptable number of rate limit errors and active pacing management is triggered. A retry schedule is defined for the requeued units of data. Extraction is temporarily halted to allow the requeued units of data to be retransformed without more units of data queuing up. Then, after the suspension is lifted, a pacing delay is inserted between subsequent extract events to avoid the same load phase bottleneck recurring.

Abstract: A method, system, and computer program product are configured to: receive, from a user device of a user, input indicating that a first extract, transform, and load (ETL) flow is incorrect and that a second ETL flow is correct; in response to receiving the input, determine whether a change made by the first ETL flow is reversible; in response to determining the change made by the first ETL flow is reversible, generate a corrective ETL flow that is configured to reverse the change made by the first ETL flow; and in response to determining the change made by the first ETL flow is not reversible, notify a user that the change made by the first ETL flow is not reversible, and inform the user which records have been changed such that they can do a manual correction.

Abstract: An approach to implement an extract, transform and load system, a queue is provided for holding units of data between extraction and transformation. When units of data suffer a rate limit error in the load phase, they are requeued so they can be resubmitted for transformation. The contents of the queue are monitored and, if too many requeued units of data are detected in the queue, then this is taken as an indicator of an unacceptable number of rate limit errors and active pacing management is triggered. A retry schedule is defined for the requeued units of data. Extraction is temporarily halted to allow the requeued units of data to be retransformed without more units of data queuing up. Then, after the suspension is lifted, a pacing delay is inserted between subsequent extract events to avoid the same load phase bottleneck recurring.

Abstract: An approach to implement an extract, transform and load system, a queue is provided for holding units of data between extraction and transformation. When units of data suffer a rate limit error in the load phase, they are requeued so they can be resubmitted for transformation. The contents of the queue are monitored and, if too many requeued units of data are detected in the queue, then this is taken as an indicator of an unacceptable number of rate limit errors and active pacing management is triggered. A retry schedule is defined for the requeued units of data. Extraction is temporarily halted to allow the requeued units of data to be retransformed without more units of data queuing up. Then, after the suspension is lifted, a pacing delay is inserted between subsequent extract events to avoid the same load phase bottleneck recurring.

Abstract: A pull request from a managed cluster may be authorized by, for each managed cluster managed by the service provider, generating an encrypted pull secret for a managed cluster, assigning the generated encrypted pull secret to the managed cluster, and responsive to receiving a pull request from a managed cluster at the service provider. The pull request may have an accompanying encrypted pull secret. The pull request may also determine whether the pull request was initiated by an intended managed cluster by decrypting the encrypted pull secret assigned to the managed cluster using a decryption key. The authorizing the pull request based on whether the decrypted pull secret is assigned to an intended managed cluster.

Abstract: An application configuration tool and associated method for supporting deployment of an application on a server that has a set of configurations available for applications deployed on the server. The method attempts to match every configuration required by the application to a configuration available on the server. In case of multiple candidates that match, one is selected. When there is no match an error message is generated. The method is iterated to inspect each selected configuration to identify any references contained in the selected configuration that themselves need further configurations. The iteration of inspecting and the matching to follow the references is continued until all such references are exhausted either by matching or failure to match and consequent error message generation. Finally, a configuration report is output specifying the selected configurations and, to the extent that not all required configurations have been matched to available configurations, the error messages.

Abstract: An application configuration tool and associated method for supporting deployment of an application on a server that has a set of configurations available for applications deployed on the server. The method attempts to match every configuration required by the application to a configuration available on the server. In case of multiple candidates that match, one is selected. When there is no match an error message is generated. The method is iterated to inspect each selected configuration to identify any references contained in the selected configuration that themselves need further configurations. The iteration of inspecting and the matching to follow the references is continued until all such references are exhausted either by matching or failure to match and consequent error message generation. Finally, a configuration report is output specifying the selected configurations and, to the extent that not all required configurations have been matched to available configurations, the error messages.

Abstract: An approach to implement an extract, transform and load system, a queue is provided for holding units of data between extraction and transformation. When units of data suffer a rate limit error in the load phase, they are requeued so they can be resubmitted for transformation. The contents of the queue are monitored and, if too many requeued units of data are detected in the queue, then this is taken as an indicator of an unacceptable number of rate limit errors and active pacing management is triggered. A retry schedule is defined for the requeued units of data. Extraction is temporarily halted to allow the requeued units of data to be retransformed without more units of data queuing up. Then, after the suspension is lifted, a pacing delay is inserted between subsequent extract events to avoid the same load phase bottleneck recurring.

Abstract: A method, system, and computer program product for determining valid drop targets for nodes within an integration flow using mapping that includes: identifying a first node, identifying one or more input nodes, where the one or more input nodes are nodes within an integration flow of the first node, analyzing a mapping of the first node, where the analyzing includes determining whether the mapping contains one or more inputs from the one or more input nodes, and in response to determining that the mapping contains one or more inputs from an input node of the one or more input nodes, identifying valid drop points for the first node using the input node.

Abstract: Concepts for sharing processing resource of a multi-tenant extract transform load, ETL, system are presented. In such concepts, a total workload of the multi-tenant ETL system is considered along with the queued workload of tenant in order control deliver of the queued workload to the system. Such control is undertaken, for example, by delaying the work of the tenant. Proposed embodiments therefore seek to devise a policy to achieve fairness amongst tenants.

Abstract: Concepts for sharing processing resource of a multi-tenant extract transform load, ETL, system are presented. In such concepts, a total workload of the multi-tenant ETL system is considered along with the queued workload of tenant in order control deliver of the queued workload to the system. Such control is undertaken, for example, by delaying the work of the tenant. Proposed embodiments therefore seek to devise a policy to achieve fairness amongst tenants.

Abstract: A method, system, and computer program product for determining valid drop targets for nodes within an integration flow using mapping that includes: identifying a first node, identifying one or more input nodes, where the one or more input nodes are nodes within an integration flow of the first node, analyzing a mapping of the first node, where the analyzing includes determining whether the mapping contains one or more inputs from the one or more input nodes, and in response to determining that the mapping contains one or more inputs from an input node of the one or more input nodes, identifying valid drop points for the first node using the input node.

Abstract: Passing a result of batch processing into a data integration flow is provided. Each data record of a plurality of data records corresponding to a specified data record collection is processed within a different thread of a different execution branch in parallel. At an end of each processing of each different execution branch, a new value is computed for each processed data record based on a previously stored value and an execution branch context for each processed data record. It is determined whether all data records in the plurality of data records have been processed. In response to determining that all of the data records in the plurality of data records have been processed, a data accumulation object corresponding to the batch processing is updated within a data assembly context based on a computed new value for each processed data record to form an updated data assembly context.

Abstract: Passing a result of batch processing into a data integration flow is provided. Each data record of a plurality of data records corresponding to a specified data record collection is processed within a different thread of a different execution branch in parallel. At an end of each processing of each different execution branch, a new value is computed for each processed data record based on a previously stored value and an execution branch context for each processed data record. It is determined whether all data records in the plurality of data records have been processed. In response to determining that all of the data records in the plurality of data records have been processed, a data accumulation object corresponding to the batch processing is updated within a data assembly context based on a computed new value for each processed data record to form an updated data assembly context.

Abstract: A method of identifying desired objects of information determines whether an existing rule is appropriate to identify a new desired object of information, defines a new rule to include at least one search query string when one of the existing rules is not appropriate to identify the new desired object of information, and defines an initial new search query string to identify the new desired object of information, wherein the initial search query string has a search query string input value. Furthermore, the method includes identifying objects having an object value equal to the search query string input value, and identifying the objects as the results of the processing operation and as having an equivalence relationship with the initial search query string. When the results do not satisfy the new rule, subsequent search query strings are defined to form a search query string chain.