Abstract: A method comprising receiving a document having multiple sections of different types using a processor. The method also comprises obtaining a plurality of lexicons using the processor, each of the lexicons for interpreting fragments in one or more of the section types. The method further comprises interpreting fragments in a first section of the multiple sections using the processor and one or more lexicons. The method still further comprises determining, based upon the interpretation and using the processor, that a fragment in the first section is misplaced. The method still further comprises re-locating, using the processor, the misplaced fragment to a second section of the multiple sections in the document to generate a re-organized document. The method additionally includes storing the re-organized document in a hardware storage system using the processor.

Abstract: A mechanism is provided in a data processing system comprising at least one processor and at least one memory, the at least one memory comprising instructions executed by the at least one processor to cause the at least one processor to implement an automated lexicon expansion for an identified corpus. For a selected corpus in a set of corpora, the mechanism determines an estimated number of new terms in the selected corpus that are not in the lexicon based on a frequency count known terms in the selected corpus. Responsive to the estimated number of new terms in the selected corpus being greater than a threshold, the mechanism performs lexicon expansion using the selected corpus to form an expanded lexicon. Responsive to the estimated number of new terms in the selected corpus not being greater than the threshold, the mechanism halts lexicon expansion.

Abstract: A system, comprising an input device configured to receive a first item and a second item, and a processor communicably coupled to the input device and configured to determine that the first item is a fragment matching a lexicon, and place the fragment in a section of a document, the section selected based on the matching lexicon, wherein the processor is configured to perform the determination and the placement after it receives the first item but before it receives the second item.

Abstract: A mechanism is provided in a data processing system comprising at least one processor and at least one memory, the at least one memory comprising instructions executed by the at least one processor to cause the at least one processor to implement an automated lexicon expansion for an identified corpus. For a selected corpus in a set of corpora, the mechanism determines an estimated number of new terms in the selected corpus that are not in the lexicon based on a frequency count known terms in the selected corpus. Responsive to the estimated number of new terms in the selected corpus being greater than a threshold, the mechanism performs lexicon expansion using the selected corpus to form an expanded lexicon. Responsive to the estimated number of new terms in the selected corpus not being greater than the threshold, the mechanism halts lexicon expansion.

Abstract: A computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to receive a document having multiple section headers, segment the document into at least first and second sections based on the section headers, segment items in the first section into fragments and identify a section type for each of the fragments, determine that the identified section type for at least one of the fragments better matches a type of the second section than it matches a type of the first section, and re-locate the at least one of the fragments to the second section.

Abstract: Influencers (individuals or groups) over a selected audience (observers or recipients of information, objects and/or events) on a given topic are measured based on influence features, which include a sentiment flipping influence feature indicative of ability of an audience member to influence other audience members to change their sentiment on the selected topic. Other influence features include the ability to influence others: to change followership; to express interest in a topic associated with a hashtag pioneered by the influencer, based on the effectiveness and phrasing of language used. The output of the influence engine can be a score representing the relative influence of audience members over the audience on the topic of interest. Influencers may be ranked according to their total influence score over the audience on the topic.

Abstract: A method for creating a semantically aggregated index in an indexer-agnostic index building system includes: extracting documents from a data source, each document including a data object; distributing the documents to a plurality of processing nodes within the system; for each node: indexing the data objects for each document into fields using semantic rules; and grouping indexed data objects for related fields by: classifying the documents into logical groups based on the semantic rules; and creating a searchable index shard for related logical groups.

Abstract: A computer program product for an indexer-agnostic index building system includes a computer readable storage medium to store a computer readable program, wherein the computer readable program, when executed on a computer, causes the computer to perform operations for creating a semantically aggregated index. The operations include: extracting documents from a data source, wherein each document includes a data object; distributing the documents to a plurality of processing nodes within the system; for each node: indexing the data objects for each document into fields using semantic rules; and grouping indexed data objects for related fields by: classifying the documents into logical groups based on the semantic rules; and creating a searchable index shard for related logical groups.

Abstract: Embodiments of the present invention relate to interactive optimization of messages published to digital media based on past performance of similar messages. In one embodiment, an input token is received. At least one candidate substitute token is retrieved from a dictionary. The dictionary comprises a mapping from the input token to the at least one candidate substitute token. A score associated with the at least one candidate substitute token is determined. A score associated with the input token is determined. The score associated with the input token, the at least one candidate substitute token, and the score associated with the at least one candidate substitute token are outputted.

Abstract: Embodiments of the present invention relate to scoring of messages published to digital media based on past performance of similar messages. In one embodiment, an input token is received. A plurality of messages is selected from a corpus of messages. Each of the plurality of messages has a publication time and contents. The contents of each of the plurality of messages include the input token. A plurality of root messages is determined from the plurality of messages. Each of the plurality of root messages relates to at least one related message. The at least one related message is one of the plurality of messages. Each of the plurality of root messages is the earliest message of the corpus of messages related to its at least one related message. A score is determined for the input token based on the plurality of root messages.

Abstract: A system for transforming domain specific unstructured data into structured data including an intake platform controlled by feed back from a control platform. The intake platform includes an intake acquisition module for acquiring data building baseline data related to a domain and problem of interest, an intake pre-processing module, an intake language module, an intake application descriptors module, and an intake adjudication module. The control platform includes a control data acquisition module, a control data consistency collator, a control auditor, a control event definition and policy repository, an error resolver, and an output that outputs results of the workflow into structured data enabled to be used in data analytics.

Abstract: A method for creating a semantically aggregated index in an indexer-agnostic index building system includes: extracting documents from a data source, each document including a data object; distributing the documents to a plurality of processing nodes within the system; for each node: indexing the data objects for each document into fields using semantic rules; and grouping indexed data objects for related fields by: classifying the documents into logical groups based on the semantic rules; and creating a searchable index shard for related logical groups.

Abstract: Embodiments of the invention relate to building a distributed reverse semantic index. In one general embodiment a plurality of documents are received with each document having at least one defined rule and or semantic. The documents are distributed among a plurality of nodes of a system. The documents are processed in a generally parallel fashion. Processing the documents includes processing text data of each of the document and breaking each document into fields to index the text data to create index data by deferring how to categorize the text data based upon the defined rule and or semantics. The indexed data is combined back together to create an indexer-agnostic semantic index including a plurality of the semantic index shards and to semantically classify the documents based on the index shards into groups based on document type to create the distributed reverse semantic index.

Abstract: The present invention relates to a method for the configurable real time transformation of dissimilar data sources, the method further consisting of the steps of acquiring real time information pertaining to at least one data source, wherein the information comprises reference information that is associated with the data source, data transformation specification information that is associated with the data source, and scheduled event specification information that is associated with the data source, and maintaining the data source information.

Abstract: Embodiments of the invention relate to building a distributed reverse semantic index. In one general embodiment a plurality of documents are received with each document having at least one defined rule and or semantic. The documents are distributed among a plurality of nodes of a system. The documents are processed in a generally parallel fashion. Processing the documents includes processing text data of each of the document and breaking each document into fields to index the text data to create index data by deferring how to categorize the text data based upon the defined rule and or semantics. The indexed data is combined back together to create an indexer-agnostic semantic index including a plurality of the semantic index shards and to semantically classify the documents based on the index shards into groups based on document type to create the distributed reverse semantic index.

Abstract: A computer program product for an indexer-agnostic index building system includes a computer readable storage medium to store a computer readable program, wherein the computer readable program, when executed on a computer, causes the computer to perform operations for creating a semantically aggregated index. The operations include: extracting documents from a data source, wherein each document includes a data object; distributing the documents to a plurality of processing nodes within the system; for each node: indexing the data objects for each document into fields using semantic rules; and grouping indexed data objects for related fields by: classifying the documents into logical groups based on the semantic rules; and creating a searchable index shard for related logical groups.

Abstract: The present invention provides a method, system, and service of analyzing electronic documents in an intranet, where the intranet includes a plurality of web sites. In an exemplary embodiment, the method, system, and service include (1) crawling HTML content and text content in a set of the sites, (2) deep-scanning non-HTML content and non-text content in the set of sites, (3) reverse-scanning the set of sites, (4) performing a semantic analysis of the crawled content and the deep-scanned content, (5) correlating the results of the semantic analysis with the results of the reverse-scanning, and (6) comparing user navigation patterns and content from the members of the set of sites. In a further embodiment, the method, system, and service further include combining the results of the performing, the results of the correlating, and the results of the comparing.

Abstract: A system for transforming domain specific unstructured data into structured data including an intake platform controlled by feed back from a control platform. The intake platform includes an intake acquisition module for acquiring data building baseline data related to a domain and problem of interest, an intake pre-processing module, an intake language module, an intake application descriptors module, and an intake adjudication module. The control platform includes a control data acquisition module, a control data consistency collator, a control auditor, a control event definition and policy repository, an error resolver, and an output that outputs results of the workflow into structured data enabled to be used in data analytics.

Abstract: A method and system for processing complex long running queries with respect to a database in which the database workload is determined in terms of quality of service (QoS) requirements of with respect to short running queries, which can be of a transactional type, in which long running queries are partitioned into a plurality of sub-queries that satisfy the database QoS requirements, are then processed and the results of processing the plurality of sub-queries are aggregated so as to correspond to the processing of the long running query.

Abstract: The present invention relates to a method for the configurable real time transformation of dissimilar data sources, the method further consisting of the steps of acquiring real time information pertaining to at least one data source, wherein the information comprises reference information that is associated with the data source, data transformation specification information that is associated with the data source, and scheduled event specification information that is associated with the data source, and maintaining the data source information.