Abstract: A method, computer system, and a computer program product are provided for cleansing steps. These are in accordance with existing data quality and rules in existence for using a plurality of different transformation assets. Information is obtained about a plurality of different transformation assets and their associated data quality and rules are extracted. A plurality of possible cleansing steps to be performed are identified for the plurality of different transformation assets. An analysis is performed for the identified cleansing steps, on impact on the different transformation assets. It is then determined when more than one identified step has a similar semantics across the plurality of different transformation assets and when more than any two of them need to perform a similar step across the same dataset. The relevance of each cleansing step to be performed is then determined and a cleansing step order of performance is provided.

Abstract: An embodiment detects, by an outlier detector, a first potential outlier in a data structure. The embodiment determines, by the outlier detector, whether the first potential outlier is a first outlier based on a first threshold. The embodiment applies, by an outlier corrector, responsive to determining the first potential outlier is a first outlier, a data quality rule to the first outlier. The embodiment detects, by the outlier detector, a second potential outlier in the data structure. The embodiment decreases, by the outlier detector, the first threshold to a second threshold. The embodiment determines, by the outlier detector, whether the second potential outlier is a second outlier based on the second threshold. The embodiment applies, by the outlier corrector, responsive to determining the second potential outlier is a second outlier, the data quality rule to the second outlier.

Abstract: A method, computer system, and a computer program product are provided for cleansing steps. These are in accordance with existing data quality and rules in existence for using a plurality of different transformation assets. Information is obtained about a plurality of different transformation assets and their associated data quality and rules are extracted. A plurality of possible cleansing steps to be performed are identified for the plurality of different transformation assets. An analysis is performed for the identified cleansing steps, on impact on the different transformation assets. It is then determined when more than one identified step has a similar semantics across the plurality of different transformation assets and when more than any two of them need to perform a similar step across the same dataset. The relevance of each cleansing step to be performed is then determined and a cleansing step order of performance is provided.

Abstract: An approach is provided for computing and using a currency score. A currency score of a data element is determined as a weighted average of scores of dimensions of the data element. The dimensions include a combination of change frequency, change size, outdated value, and sources score dimensions. Relative to the data element, the change frequency dimension indicates an update frequency, the change size dimension indicates amounts of data being created, updated, and deleted per time unit, the outdated value dimension indicates a portion of values that are not semantically correct, but were semantically correct in the past, and the sources score dimension indicates a currency of input source(s). Based on the currency score, a currency of data included in the data element is evaluated. Based on the currency of the data, a remedial action is performed to improve the currency of the data.

Abstract: A computer-implemented method for executing a serverless workload on a transient infrastructure is disclosed. The method comprises receiving a request for executing a workload and at least one policy, selecting an infrastructure component of the transient infrastructure for an execution of the workload, wherein the transient infrastructure is implemented utilizing a plurality of geographically distributed computing nodes, and executing the workload on the transient infrastructure. Furthermore, the method comprises upon receiving an interfering event during the execution of the workload causing a stop of the workload execution, freezing the execution of the workload, storing intermediate results and statuses of the workload execution in a cross-domain storage system, terminating the execution of the workload, and continuing the workload execution using the intermediate results and under a constraint defined by the at least one policy.

Abstract: A method, system, and computer program product for implementing automated serverless runtime container allocation is provided. The method includes defining a number of runtime containers and associated characteristics required for each worker node of a plurality of worker nodes for execution of a specified workload. The specified workload is dispatched to the plurality of worker nodes and a specified portion of the specified workload is assigned to each worker node. An application executing a universal runtime container that includes potential application runtimes and associated supported software versions within a layered modifiable format is generated and unused layers are removed from the universal runtime container. The specified workload is executed via the universal runtime container and a set of available universal runtime containers is refilled on an associated work node.

Abstract: In an approach, a processor groups documents into a plurality of groups based on similarity, where: documents of each group have a same document structure; and the document structure is defined by coordinates of text blocks. A processor, for each group of the plurality of groups and for each document of the respective group: retrieves a value of each text block of the respective document in accordance with a document structure of the group; and assigns to each text block of the respective document an attribute that represents the retrieved value of the text block. A processor assigns a first document of the documents to an entity of a database that matches the first document based on the group of text block values and the assigned attributes of the document.

Abstract: A method, system, and computer program product for implementing automated serverless runtime container allocation is provided. The method includes defining a number of runtime containers and associated characteristics required for each worker node of a plurality of worker nodes for execution of a specified workload. The specified workload is dispatched to the plurality of worker nodes and a specified portion of the specified workload is assigned to each worker node. An application executing a universal runtime container that includes potential application runtimes and associated supported software versions within a layered modifiable format is generated and unused layers are removed from the universal runtime container. The specified workload is executed via the universal runtime container and a set of available universal runtime containers is refilled on an associated work node.

Abstract: In an approach, a processor groups documents into a plurality of groups based on similarity, where: documents of each group have a same document structure; and the document structure is defined by coordinates of text blocks. A processor, for each group of the plurality of groups and for each document of the respective group: retrieves a value of each text block of the respective document in accordance with a document structure of the group; and assigns to each text block of the respective document an attribute that represents the retrieved value of the text block. A processor assigns a first document of the documents to an entity of a database that matches the first document based on the group of text block values and the assigned attributes of the document.

Abstract: A computer device identifies a set of documents for classification. The computing device classifies documents of a first subset of the set of documents based, at least in part, on a text analysis of the documents of the first subset. The computing device trains a document classifier using, as training data: (i) results of the classifying of the documents of the first subset, and (ii) metadata associated with the documents of the first subset. The computing device classifies documents of a second subset of the set of documents by providing metadata of the documents of the second subset to the trained document classifier.

Abstract: The present disclosure relates to a method for data quality delta analysis on a dataset. The method provides a set of data quality rules for the dataset. At least one delta rule of a set of data quality rules is defined as relevant for delta analysis of at least part of the dataset, the delta rule being a delta analysis quality rule. Data changes on the dataset are tracked. In response to determining that a number of modified records of the at least part of the dataset is higher than a predefined insert modification threshold, a data quality score may be determined for said modified records using the delta rule.

Abstract: Processing a set of records, each record of the dataset having multiple attributes, including a dimension group of attributes and a measure group of attributes. Each record of the dataset is processed by: comparing a current value of at least one attribute of the record's dimension group with a previous value of the attribute in a previously processed record and creating a new record. The value of each attribute of the new record is determined as a combination of the values of the attribute in the set of records. The values of each attribute of the new record are associated with a quality score determined as a combination of quality scores of the attribute in the set of records.

Abstract: The present disclosure relates to a method for data quality delta analysis on a dataset. The method provides a set of data quality rules for the dataset. At least one delta rule of a set of data quality rules is defined as relevant for delta analysis of at least part of the dataset, the delta rule being a delta analysis quality rule. Data changes on the dataset are tracked. In response to determining that a number of modified records of the at least part of the dataset is higher than a predefined insert modification threshold, a data quality score may be determined for said modified records using the delta rule.

Abstract: Processing a set of records, each record of the dataset having multiple attributes, including a dimension group of attributes and a measure group of attributes. Each record of the dataset is processed by: comparing a current value of at least one attribute of the record's dimension group with a previous value of the attribute in a previously processed record and creating a new record. The value of each attribute of the new record is determined as a combination of the values of the attribute in the set of records. The values of each attribute of the new record are associated with a quality score determined as a combination of quality scores of the attribute in the set of records.

Abstract: A present invention embodiment adjusts application of a set of data quality rules for data within one or more reports. A first report field is selected that receives data from a same source data field as at least one second report field within the one or more reports. An active data quality rule associated with the at least one second report field is identified that is inactive for the first report field. An effectiveness of the identified data quality rule is determined for data quality of the first report field. Application of a set of data quality rules is adjusted for the first report field to include the identified data quality rule in response to the effectiveness satisfying a threshold.

Abstract: A present invention embodiment adjusts application of a set of data quality rules for data within one or more reports. A first report field is selected that receives data from a same source data field as at least one second report field within the one or more reports. An active data quality rule associated with the at least one second report field is identified that is inactive for the first report field. An effectiveness of the identified data quality rule is determined for data quality of the first report field. Application of a set of data quality rules is adjusted for the first report field to include the identified data quality rule in response to the effectiveness satisfying a threshold.