Abstract: Embodiments analyze at least one data source from at least one database to extract characteristics for the at least one data source; generate at least one recommendation based on the extracted characteristics for the at least one data source; perform conflict resolution on the generated at least one recommendation by removing at least one conflicted recommendation in the at least one recommendation; determine a chosen recommendation (CR) set based on the conflict resolution; rank each recommendation in the CR set based on a ranking score; and display a visual summary of each recommendation in the CR set.

Abstract: An example operation includes one or more of determining a predictive task and one or more constraints of the predictive task based on inputs to a graphical user interface of a software application, executing a machine learning model on input data including the predictive task and the one or more constraints to select a LLM from among a plurality of available LLMs for executing the predictive task, additionally executing the machine learning model on the LLM and the input data to determine a plurality of components of the predictive pipeline including the LLM, and instantiating an instance of the predictive pipeline including the plurality of components and the LLM via the software application.

Abstract: A computer-implemented method for transforming data is provided. A processor set receives a number of data pairs. The processor set creates a program graph for each data pair in the number of data pairs. The processor set identifies a number of paths between nodes in each program graph. The processor set identifies a number of common paths from the number of paths based on common characters between the input data and the output data for each data pair. The processor set identifies a set of nodes in the program graphs based on the number of common paths. The processor set generates a prompt for a large language model based on the number of data pairs and the set of nodes that represent positions of unmatched characters between input data and output data in the number of data pairs.

Abstract: Methods, systems, and computer program products for automatically detecting periods of normal activity by analyzing observability data in IT operations environments are provided herein. A computer-implemented method includes obtaining multiple types of data related to one or more artificial intelligence-related information technology operations; modelling at least a portion of the obtained data as time series data; automatically identifying, from the time series data, one or more time periods associated with one or more given levels of data activity; and performing one or more automated actions, in at least one artificial intelligence-related information technology operations environment, based at least in part on the data corresponding to the one or more identified time periods.

Abstract: An embodiment computes a plurality of similarity scores, each similarity score in the plurality of similarity scores measuring a similarity of task data of a first data transformation task to data of a stored data transformation in a plurality of stored data transformations, wherein each of the plurality of stored data transformations comprises a stored data transformation program, each stored data transformation program comprising a data transformation from a first data format to a second data format. An embodiment generates, from a first stored data transformation program in the plurality of stored data transformations, a generated data transformation program. An embodiment performs, by modifying a plurality of records described by the first data transformation task into a second plurality of records according to the generated data transformation program, the first data transformation task.

Abstract: Methods, systems, and computer program products for automatically improving data annotations by processing annotation properties and user feedback are provided herein. A computer-implemented method includes obtaining data annotation pairs, each comprising an input data annotation in a first format and a corresponding output data annotation in a second format; determining, within at least a portion of the data annotation pairs, one or more non-diffs; identifying, across the at least a portion of data annotation pairs, data annotation properties associated with multiple intents by processing the non-diffs using property-related rules; modifying at least a portion of the data annotation pairs based on the identified data annotation properties; outputting the modified data annotation pairs to at least one user; and generating a final collection of data annotation pairs by processing at least a portion of the modified data annotation pairs and user feedback received in response to the outputting.

Abstract: One embodiment provides a computer implemented method, including: obtaining an information document corresponding to an entity, wherein the information document includes redacted information spans; identifying an entity type for each of the redacted information spans, wherein the entity type identifies a relationship between a redacted information span and at least one other entity within the information document; replacing the redacted information spans with replacement entities corresponding to the entity type of a given redacted information span, wherein the replacing is performed in view of a frequency distribution of actual information and wherein the replacing includes maintaining relationships of the redacted information spans; and controlling bias within the replacement entities, wherein the controlling includes detecting bias within the replacement entities.

Abstract: A computer-implemented method for training a machine learning model for managing prompt. A processor set determines patterns of data in a sample dataset to identify representative data from the sample dataset. The processor set combines the representative data with context for a number of tasks to generate a number of simple prompts. Each simple prompt comprises a portion of the representative data and context for a task from the number of tasks. The processor set trains the machine learning model using a training dataset comprises the number of simple prompts. The machine learning model is trained to identify priorities of words in the number of simple prompts.

Abstract: A method, computer program, and computer system are provided for collecting and annotating data based on user preference. Unlabeled data corresponding to one or more entries within a dataset is received. Pseudo-labeled data is generated based on the unlabeled data. Based on one or more quality metrics, each entry from among the pseudo-labeled data is determining to be included within a final dataset. A user is prompted for annotations corresponding to entries of the pseudo-labeled data included within the final dataset. A determination is made as to whether additional data is needed based on comparing the final dataset to the one or more quality metrics, and the additional information is collected if the final dataset does not meet the quality metrics.

Abstract: An embodiment computes a plurality of similarity scores, each similarity score in the plurality of similarity scores measuring a similarity of task data of a first data transformation task to data of a stored data transformation in a plurality of stored data transformations, wherein each of the plurality of stored data transformations comprises a stored data transformation program, each stored data transformation program comprising a data transformation from a first data format to a second data format. An embodiment generates, from a first stored data transformation program in the plurality of stored data transformations, a generated data transformation program. An embodiment performs, by modifying a plurality of records described by the first data transformation task into a second plurality of records according to the generated data transformation program, the first data transformation task.

Abstract: A cleansing operation defined for a data structure of a database managed by a database management system is obtained. The cleansing operation is performed on data of the data structure to obtain clean data. The cleansing operation that is defined for the data structure and performed on data of the data structure is performed by the database management system.

Abstract: Selecting and ordering the execution of data quality rules includes generating a snapshot of a table-formatted dataset. The snapshot comprises a reduced number of rows of the dataset such that each column variation of the dataset is included in the snapshot. A predetermined collection of data quality (DQ) rules is executed on the snapshot. One or more performance statistics is determined for each of the DQ rules. The performance statistics indicate a likelihood that a DQ rule determines a data quality deficiency. Based on the performance statistics, a subset of the DQ rules is generated. Each DQ rule of the subset is selected based on the likelihood that the DQ rule selected detects a quality deficiency. An ordered subset of selected DQ rules is generated by ordering the application of each of the subset of DQ rules selected. The ordering specifies a sequence for executing each selected DQ rule.

Abstract: Selecting and ordering the execution of data quality rules includes generating a snapshot of a table-formatted dataset. The snapshot comprises a reduced number of rows of the dataset such that each column variation of the dataset is included in the snapshot. A predetermined collection of data quality (DQ) rules is executed on the snapshot. One or more performance statistics is determined for each of the DQ rules. The performance statistics indicate a likelihood that a DQ rule determines a data quality deficiency. Based on the performance statistics, a subset of the DQ rules is generated. Each DQ rule of the subset is selected based on the likelihood that the DQ rule selected detects a quality deficiency. An ordered subset of selected DQ rules is generated by ordering the application of each of the subset of DQ rules selected. The ordering specifies a sequence for executing each selected DQ rule.

Abstract: A cleansing operation defined for a data structure of a database managed by a database management system is obtained. The cleansing operation is performed on data of the data structure to obtain clean data. The cleansing operation that is defined for the data structure and performed on data of the data structure is performed by the database management system.

Abstract: A cleansing operation is performed on data of a data structure to obtain clean data. The clean data is stored as part of the data structure; however, the clean data is independent of the data. A mapping is performed to provide a set of ordered data that includes the data and the clean data.

Abstract: Processing within a computing environment is facilitated using a corpus processing system to assess and enhance quality of a corpus of unstructured documents for a specified task. The processing includes referencing, by a corpus processing engine, the corpus of unstructured documents to obtain unstructured document data, and applying, by a corpus quality metrics engine, a set of quality metrics to the document data to obtain a set of quality metric scores. Further, the process includes automatically selecting, by a quality metric selection engine, a subset of task-relevant quality metrics using the quality metric scores and the specified task, and automatically transforming, at least in part, multiple documents of the corpus to remediate one or more identified issues with the documents. The automatically transforming results in remediated documents tuned for the specified task, which are provided for the specified task to be performed.

Abstract: Automatically selecting data for machine learning datasets is provided. The method comprises receiving an input dataset and user-specified data quality metrics. The input dataset is matched to a subset of candidate datasets in a repository according to schema characteristics. A second subset of candidate datasets having a distance from the input dataset above a specified threshold is selected from the first subset of candidate datasets. The second subset of candidate datasets are merged into a merged dataset. Top ranked samples above a specified second threshold are identified from the merged dataset based on the user-specified data quality metrics. The input dataset, augmented with the top ranked samples, is returned to the user.

Abstract: A functionality intent is extracted from a natural language input, the functionality intent comprising an operation on a dataset. A portion of source code implementing the functionality intent is generated. Using a result of executing an executable version of the portion of source code on the dataset, a next functionality intent is recommended, the next functionality intent expressed in natural language form.

Abstract: Processing within a computing environment is facilitated using a corpus processing system to assess and enhance quality of a corpus of unstructured documents for a specified task. The processing includes referencing, by a corpus processing engine, the corpus of unstructured documents to obtain unstructured document data, and applying, by a corpus quality metrics engine, a set of quality metrics to the document data to obtain a set of quality metric scores. Further, the process includes automatically selecting, by a quality metric selection engine, a subset of task-relevant quality metrics using the quality metric scores and the specified task, and automatically transforming, at least in part, multiple documents of the corpus to remediate one or more identified issues with the documents. The automatically transforming results in remediated documents tuned for the specified task, which are provided for the specified task to be performed.

Abstract: A method, computer program product, and computer system for transforming sets of source data having different formats into respective sets of target data having a same format. N source patterns are determined and respectively describe N different formats in which N sets of source data items are formatted, where N?1. A target format pattern is determined and describes a target format in which a target data items are formatted. N graphs are generated and respectively describe transformations of the N source patterns to the target pattern. Each graph includes multiple transformation paths. Each transformation path transforms the source pattern to the target pattern in a manner that maps source strings in the source pattern to each target string in the target pattern. A single transformation path is selected from the multiple transformation paths resulting in N single transformation paths having been selected.