Abstract: Provided are a computer program product, system, and method for distributed processing of a query with distributed posting lists. A dispatch map has entries, wherein each entry identifies one of a plurality of terms in a dictionary, wherein for each of the terms there is a posting list identifying zero or more objects including the term, wherein at least one of the dispatch map entries indicate at least one distributed processing element including the posting list for the term. The dispatch map is used to dispatch sub-expressions comprising portions of a query to distributed processing elements having the posting lists for terms in the sub-expressions, wherein the distributed processing elements distributed the sub-expressions execute the sub-expressions on the posting lists for the terms in the sub-expression.

Abstract: An electronic-discovery system and method, wherein content items and hold anchors are stored in a repository, tracking objects and representational anchor objects are stored in a database system, and the tracking objects represent the content items and the representational anchor objects represent the hold anchors. A first hold anchor is used for placing a hold on the content items for a first defined period of time, and a first representational anchor object and one or more of the tracking objects are used for representing and tracking the holds for the first defined period of time. When the first defined period of time expires, a second hold anchor is used for placing the hold on the content items for a second defined period of time, and a second representational anchor object and the tracking objects are used for representing and tracking the holds for the second defined period of time.

Abstract: Provided are a computer program product, system, and method for distributed processing of a query with distributed posting lists. A dispatch map has entries, wherein each entry identifies one of a plurality of terms in a dictionary, wherein for each of the terms there is a posting list identifying zero or more objects including the term, wherein at least one of the dispatch map entries indicate at least one distributed processing element including the posting list for the term. The dispatch map is used to dispatch sub-expressions comprising portions of a query to distributed processing elements having the posting lists for terms in the sub-expressions, wherein the distributed processing elements distributed the sub-expressions execute the sub-expressions on the posting lists for the terms in the sub-expression.

Abstract: A method, system and computer program product for detecting sensitive personal information in a storage device. A block delta list containing a list of changed blocks in the storage device is processed. After identifying the changed blocks from the block delta list, a search is performed on those identified changed blocks for sensitive personal information using a character scanning technique. After identifying a changed block deemed to contain sensitive personal information, the changed block is translated from the block level to the file level using a hierarchical reverse mapping technique. By only analyzing the changed blocks to determine if they contain sensitive personal information, a lesser quantity of blocks needs to be processed in order to detect sensitive personal information in the storage device in near real-time. In this manner, sensitive personal information is detected in the storage device using fewer computing resources in a shorter amount of time.

Abstract: Embodiments generally relate to providing litigation management for multiple remote content systems using asynchronous bi-directional replication pipelines. In some embodiments, a method includes retrieving, at one or more inbound replicators of one or more respective bi-directional pipelines, metadata associated with documents stored in one or more content repositories. The method further includes resolving, at a governance control hub, conflicts associated with legal holds on one or more of the documents based on the metadata. The method further includes sending conflict resolution results from one or more outbound applicators of the bi-directional pipelines to the content repositories, where the content repositories enforce legal holds on the documents.

Abstract: A method, system and computer program product for detecting sensitive personal information in a storage device. A block delta list containing a list of changed blocks in the storage device is processed. After identifying the changed blocks from the block delta list, a search is performed on those identified changed blocks for sensitive personal information using a character scanning technique. After identifying a changed block deemed to contain sensitive personal information, the changed block is translated from the block level to the file level using a hierarchical reverse mapping technique. By only analyzing the changed blocks to determine if they contain sensitive personal information, a lesser quantity of blocks needs to be processed in order to detect sensitive personal information in the storage device in near real-time. In this manner, sensitive personal information is detected in the storage device using fewer computing resources in a shorter amount of time.

Abstract: Provided are techniques for heuristic and non-semantic prediction of the cost to find and review data that is relevant to a task. A corpus of documents is accessed for a domain. Terms associated with the domain are accessed, where the terms have an order on a list. For each of the documents, term positional dispersion is determined for each of the terms in the ordered list associated with the domain. Then, a document review quanta is determined for the document based on a summation of the term positional dispersion for each term in that document adjusted by a weight. A subset of documents in the corpus of documents are selected that are to be reviewed based on the document review quanta for each of the selected documents exceeding a threshold.

Abstract: An electronic-discovery system and method, wherein content items and hold anchors are stored in a repository, tracking objects and representational anchor objects are stored in a database system, and the tracking objects represent the content items and the representational anchor objects represent the hold anchors. A first hold anchor is used for placing a hold on the content items for a first defined period of time, and a first representational anchor object and one or more of the tracking objects are used for representing and tracking the holds for the first defined period of time. When the first defined period of time expires, a second hold anchor is used for placing the hold on the content items for a second defined period of time, and a second representational anchor object and the tracking objects are used for representing and tracking the holds for the second defined period of time.

Abstract: Provided are techniques for a domain specific representation of document text for accelerated natural language processing. A document is selected from a set of documents to be analyzed. A character stream from the document is converted into a token stream based on tokenization rules. Irrelevant tokens are removed from the token stream. The tokens remaining in the token stream are converted into an integer domain representation based on a domain specific ontology dictionary. The integer domain representation are stored to a Graphics Processing Unit (GPU) processing queue of each of one or more GPUs. Then, a result set is received from the one or more GPUs.

Abstract: Provided are techniques for a high performance compliance mechanism for structured and unstructured data in an enterprise. A record to represent a collection of structured objects is generated. The record is stored in a file plan container associated with a disposition schedule.

Abstract: Processing a database query for sets of data includes assigning a unique identifier from an integer space to each entity within data and creating one or more sets of entities each pertaining to a corresponding entity within the data. A representation is then generated on disk for each set of entities, wherein each representation encompasses and is suited for a range of the unique identifiers of entities within a corresponding set and indicates a presence of an entity within that corresponding set. Finally, a query is processed based on the representation for each set of entities to retrieve data satisfying the query, wherein the representation provides a constant time for association and dissociation operations that are append-only operations with deferred merge and automatic filtering of deleted and duplicate entities at query time.

Abstract: In an approach for parallelizing document processing in an information handling system, a processor receives a document, wherein the document includes text content. A processor extracts information from the text content, utilizing natural language processing and semantic analysis, to form tokenized semantic partitions, comprising a plurality of sub-documents. A processor schedules a plurality of concurrently executing threads to process the plurality of sub-documents.

Abstract: In an approach for parallelizing document processing in an information handling system, a processor receives a document, wherein the document includes text content. A processor extracts information from the text content, utilizing natural language processing and semantic analysis, to form tokenized semantic partitions, comprising a plurality of sub-documents. A processor schedules a plurality of concurrently executing threads to process the plurality of sub-documents.

Abstract: Systems and associated methods for clustering a plurality of nodes based on connectivity among the plurality of nodes, determining relevant content of the clusters, and applying knowledge regarding the relevant content are described. The nodes can include for example web-based documents such as web pages. The clusters can include for example groups of web pages that are linked together, as via hyperlinks. The relevant content can include one or more topics associated with the web page, as for example determined via text mining. Applying the knowledge regarding the relevant content can include for example using the one or more topics associated with the web pages to augment search results and/or conduct contextual advertising.

Abstract: Provided are techniques for a domain specific representation of document text for accelerated natural language processing. A document is selected from a set of documents to be analyzed. A character stream from the document is converted into a token stream based on tokenization rules. Irrelevant tokens are removed from the token stream. The tokens remaining in the token stream are converted into an integer domain representation based on a domain specific ontology dictionary. The integer domain representation are stored to a Graphics Processing Unit (GPU) processing queue of each of one or more GPUs. Then, a result set is received from the one or more GPUs.

Abstract: According to one embodiment of the present invention, a system extends a content repository by creating an auxiliary data store outside of the content repository and storing auxiliary data in the auxiliary data store, wherein the auxiliary data is associated with a collection of documents in the content repository. The system stores version information for the auxiliary data store and records of operations against the auxiliary data store in a log in the repository. In response to receiving a request for an operation against the auxiliary data store, the system determines that the auxiliary data store and repository are consistent based on the version information and applies the operation against the auxiliary data store. Embodiments of the present invention further include a method and computer program product for extending a content repository data model in substantially the same manners described above.

Abstract: According to one embodiment of the present invention, a system extends a content repository by creating an auxiliary data store outside of the content repository and storing auxiliary data in the auxiliary data store, wherein the auxiliary data is associated with a collection of documents in the content repository. The system stores version information for the auxiliary data store and records of operations against the auxiliary data store in a log in the repository. In response to receiving a request for an operation against the auxiliary data store, the system determines that the auxiliary data store and repository are consistent based on the version information and applies the operation against the auxiliary data store. Embodiments of the present invention further include a method and computer program product for extending a content repository data model in substantially the same manners described above.

Abstract: In searching electronic documents, prior to executing a query, a reviewer indicates whether a result set of the query will be dynamic or static. The query is then executed on the electronic documents to obtain an original result set, which is provided to the reviewer through a user interface. Upon determining that one or more changes to one or more of the electronic documents have occurred, and if the result set is static, then the original result set continues to be provided to the reviewer without re-executing the query. If the result set is dynamic, then the query is re-executed on the electronic documents to obtain an updated result set, and the updated result set is provided to the reviewer through the user interface. The original result set may be associated with a search session and/or may be a random sample of the electronic documents for an overview query.

Abstract: In searching electronic documents, prior to executing a query, a reviewer indicates whether a result set of the query will be dynamic or static. The query is then executed on the electronic documents to obtain an original result set, which is provided to the reviewer through a user interface. Upon determining that one or more changes to one or more of the electronic documents have occurred, and if the result set is static, then the original result set continues to be provided to the reviewer without re-executing the query. If the result set is dynamic, then the query is re-executed on the electronic documents to obtain an updated result set, and the updated result set is provided to the reviewer through the user interface. The original result set may be associated with a search session and/or may be a random sample of the electronic documents for an overview query.

Abstract: Embodiments of the present invention provide a method, system and computer program product for maintaining distributed state consistency in a distributed computing application. In an embodiment of the invention, a method for maintaining distributed state consistency in a distributed computing application can include registering a set of components of a distributed computing application, starting a transaction resulting in changes of state in different ones of the components in the registered set and determining in response to a conclusion of the transaction whether or not an inconsistency of state has arisen amongst the different components in the registered set in consequence of the changes of state in the different ones of the components in the registered set. If an inconsistency has arisen, each of the components in the registered set can be directed to rollback to a previously stored state. Otherwise a committal of state can be directed in each of the components in the registered set.