Abstract: A method for providing one or more random sample documents from a corpus of documents using a search engine is provided. The providing of each of the random sample documents comprises selecting randomly a time window from a set of time windows. A search query is sent to the search engine defining a search for documents of the corpus with time-stamps within the time window defined by the randomly selected time window. In response to the sending of the search query, a search result is receiving from the search engine. The search result comprises a set of the documents of the corpus with time-stamps within the time window. One of the documents comprised by the received set of documents is then selected randomly.

Abstract: Provided are techniques for dynamic fact contextualization in support of AI model development. A template from a plurality of templates is selected, where the template includes definitions for identifying facts. The facts are retrieved from a facts repository based on the definitions. It is determined that that the facts are valid based on one or more policies. A FactSheet is generated using the template and the facts. A machine learning model is used to identify one or more deficient facts from the FactSheet. The FactSheet is displayed in a preview with the one or more deficient facts. One or more facts corresponding to the one or more deficient facts are located. The FactSheet is updated to correct the one or more deficient facts with the corresponding facts.

Abstract: Described are techniques for a re-analysis of assignments of terms to assets. The techniques include detecting a change in a term ontology comprising a plurality of terms, and determining at least one selected from a group consisting of: a domain feature change vector (DFCV) for a domain of the term ontology affected by the change, and a term feature change vector (TFCV) for the term affected by the change. The techniques further include identifying assets for the re-analysis of the assignments of terms, wherein each of the identified assets is associated with an impact score value based on the DFCV and/or the TFCV, and performing the re-analysis of the assignments of terms for the identified assets ordered by the impact score value.

Abstract: Several aspects for optimizing unstructured document analysis comprise operating a document system, where the document system comprises a plurality of documents comprising unstructured content and a full-text index; receiving a request to identify documents comprising a type of data elements; selecting a sample out of the plurality of documents; determining data elements of the type in the sample of documents; determining an indicator context expression for the type of data elements out of the determined data elements of the type; determining a query for searching, using a search engine, the full-text index using the indicator context expression; and determining the documents in the document system being compliant to the determined query.

Abstract: The present disclosure relates to a method of metadata enrichment using an enrichment comprising multiple steps. The method comprises: determining for an input data asset a metadata value descriptive of the input data asset. Characteristics of the metadata value of the input data asset may be determined. At least one informativeness score of the metadata value of the input data asset may be computed using the determined characteristics. An execution of the enrichment step may be skipped in case an input characteristic of the enrichment step is not part of the determined characteristics. In case the input characteristic of the enrichment step is part of the determined characteristics, the enrichment step may be adapted and executed or the enrichment step may be executed without adaptation. Labels resulting from the executed enrichment steps may be combined for providing one or more labels of the data asset.

Abstract: Techniques are described relating to unstructured document processing. An associated computer-implemented method includes identifying a plurality of deduplicated data blocks associated with a collection of unstructured documents. The method further includes sorting the plurality of deduplicated data blocks in descending order based upon at least one block frequency metric, selecting a highest sorted unprocessed deduplicated data block, applying text analytics to the selected deduplicated data block, and applying at least one result of the text analytics to any document among the collection of unstructured documents including the selected deduplicated data block. The method is terminated responsive to satisfaction of at least one stopping condition.

Abstract: A method for providing one or more random sample documents from a corpus of documents using a search engine is provided. The providing of each of the random sample documents comprises selecting randomly a time window from a set of time windows. A search query is sent to the search engine defining a search for documents of the corpus with time-stamps within the time window defined by the randomly selected time window. In response to the sending of the search query, a search result is receiving from the search engine. The search result comprises a set of the documents of the corpus with time-stamps within the time window. One of the documents comprised by the received set of documents is then selected randomly.

Abstract: A method for providing one or more random sample documents from a corpus of documents using a search engine is provided. The providing of each of the random sample documents comprises selecting randomly a time window from a set of time windows. A search query is sent to the search engine defining a search for documents of the corpus with time-stamps within the time window defined by the randomly selected time window. In response to the sending of the search query, a search result is received from the search engine. The search result comprises a set of the documents of the corpus with time-stamps within the time window. One of the documents comprised by the received set of documents is then selected randomly.

Abstract: A method for processing electronic documents comprises an iteration including: (i) applying, by a computer device, a first statistical test process to a first subset of the documents, the first statistical test process estimating whether or not content of the documents of the first subset comply with a predefined criterion; (ii) in response to a result of the first statistical test process, estimating, by the computer device, that the documents of the first subset do not comply with the criterion, selecting, by the computer device, a part of the documents of the first subset, and moving, by the computer device, the part of the documents to a second subset of the documents; and (iii) applying, by the computer device, a second statistical test process to the second subset of the documents, the second statistical test process calculating at least one statistical metric related to the documents of the second subset.

Abstract: Techniques are described relating to unstructured document processing. An associated computer-implemented method includes identifying a plurality of deduplicated data blocks associated with a collection of unstructured documents. The method further includes sorting the plurality of deduplicated data blocks in descending order based upon at least one block frequency metric, selecting a highest sorted unprocessed deduplicated data block, applying text analytics to the selected deduplicated data block, and applying at least one result of the text analytics to any document among the collection of unstructured documents including the selected deduplicated data block. The method is terminated responsive to satisfaction of at least one stopping condition.

Abstract: Embodiments of the present invention provide methods, computer program products, and systems. Embodiments of the present invention can extract of structured information for unstructured document analysis. Embodiments of the present invention can extract structured information for unstructured document analysis by identifying tables and columns of a database that correspond to business terms of a business glossary. Embodiments of the present invention can then receive a specification of business terms of interest for recognizing in an unstructured document. Embodiments of the present invention can then generate an analysis module based on the identified tables and columns that enables to identify or recognize attribute values of attributes of the tables and columns. Embodiments of the present invention can then use the analysis module for automatic extraction of values of at least part of the attributes from the unstructured document based on the specification of business terms of interest.

Abstract: A method, computer system, and computer program product for segment differential-based document text-index modeling are provided. The embodiment may include receiving, by a processor, a document with a valid document ID and version ID tuple. The embodiment may also include determining the received document is a new version of a previously stored document and consequently multiplexing versions of the document into a single indexed document. The embodiment may further include segmenting the received document and building a token vector. The embodiment may also include calculating a difference between the received new version of the document and the previously stored document using information obtained from the segmentation. The embodiment may further include in response to the calculated difference being below a pre-configured threshold value, discarding the received new version.

Abstract: A method for providing one or more random sample documents from a corpus of documents using a search engine is provided. The providing of each of the random sample documents comprises selecting randomly a time window from a set of time windows. A search query is sent to the search engine defining a search for documents of the corpus with time-stamps within the time window defined by the randomly selected time window. In response to the sending of the search query, a search result is receiving from the search engine. The search result comprises a set of the documents of the corpus with time-stamps within the time window. One of the documents comprised by the received set of documents is then selected randomly.

Abstract: A method for processing electronic documents comprises an iteration including: (i) applying, by a computer device, a first statistical test process to a first subset of the documents, the first statistical test process estimating whether or not content of the documents of the first subset comply with a predefined criterion; (ii) in response to a result of the first statistical test process, estimating, by the computer device, that the documents of the first subset do not comply with the criterion, selecting, by the computer device, a part of the documents of the first subset, and moving, by the computer device, the part of the documents to a second subset of the documents; and (iii) applying, by the computer device, a second statistical test process to the second subset of the documents, the second statistical test process calculating at least one statistical metric related to the documents of the second subset.

Abstract: A method, computer system, and computer program product for segment differential-based document text-index modeling are provided. The embodiment may include receiving, by a processor, a document with a valid document ID and version ID tuple. The embodiment may also include determining the received document is a new version of a previously stored document and consequently multiplexing versions of the document into a single indexed document. The embodiment may further include segmenting the received document and building a token vector. The embodiment may also include calculating a difference between the received new version of the document and the previously stored document using information obtained from the segmentation. The embodiment may further include in response to the calculated difference being below a pre-configured threshold value, discarding the received new version.

Abstract: The invention relates to a computer-implemented method for performing an electronic discovery process using a blockchain. For at least two of phases of the electronic discovery process individual electronic data processing tools are used for executing the respective phase. For each of the phases of the electronic discovery process metadata of the electronic data objects subject to the respective phase of the electronic discovery process and to the electronic data processing tool used for executing the respective phase are stored in the blockchain. Furthermore, audit information about auditable actions carried out during the respective phase by the electronic data processing tool used for executing the respective phase on the electronic data objects is stored in the blockchain. The stored audit information provides an end-to-end audit trail for each of the electronic data objects subject to at least one of the phases of the electronic discovery process.

Abstract: The invention relates to a computer-implemented method for performing an electronic discovery process using a blockchain. For at least two of phases of the electronic discovery process individual electronic data processing tools are used for executing the respective phase. For each of the phases of the electronic discovery process metadata of the electronic data objects subject to the respective phase of the electronic discovery process and to the electronic data processing tool used for executing the respective phase are stored in the blockchain. Furthermore, audit information about auditable actions carried out during the respective phase by the electronic data processing tool used for executing the respective phase on the electronic data objects is stored in the blockchain. The stored audit information provides an end-to-end audit trail for each of the electronic data objects subject to at least one of the phases of the electronic discovery process.

Abstract: A method and a computer-readable medium for method for searching a plurality of documents. Each document is structured into a set of blocks and each block is associated with a block ID. The method includes receiving a search query including a search term having at least one search term attribute; identifying at least one block ID based on a correlation between the at least one search term attribute and the set of blocks; and identifying at least one document based on a correlation between the set of blocks and the documents. Methods for generating a data structure for searching documents are also described.

Abstract: Providing access to media data shared by multiple users. A predefined edge weight is assigned to each edge of a linked data structure based on a dependency category of the edge. A first access rating value is assigned to each node. A rating residue value is calculated as the difference between the two first access rating values of the nodes connected by each edge. The data structure is traversed from a seed node, and for each edge traversed, calculating a second access rating value using an edge weight value and the first access rating value. Repeating until the rating residue values meet a predefined convergence criterion. The nodes having access rating values meeting a predefined data removal criterion are selected from the nodes of the linked data structure. The data entities corresponding to the selected nodes are then removed.

Abstract: Provided are techniques for creating an inverted index for features of a set of data elements, wherein each of the data elements is represented by a vector of features, wherein the inverted index, when queried with a feature, outputs one or more data elements containing the feature. The features of the set of data elements are ranked. For each feature in the ranked list, the inverted index is queried for data elements having the feature and not having any previously selected feature and a cluster of the data elements is created based on results returned in response to the query.