Abstract: A method, system and a computer program product are provided for scoring candidate answers for geographic relevance by identifying document location information that is associated with a document, associating each token in the document with the document location information, and then comparing geographic foci identified for a candidate answer from the tokens with geographic foci identified for an input question to generate a geographic relevance score for the candidate answer to the input question.

Abstract: A method, system and a computer program product are provided for scoring candidate answers for geographic relevance analyzing an input question to identify one or more first geographic foci of the input question based on geographical contextual information associated with the input question, identifying one or more second geographic foci for a candidate answer generated in response to the input question, and then comparing the first and second geographic foci to generate a geographic relevance score for the candidate answer to the input question.

Abstract: A system includes a knowledge canvassing system executed by a computer, a processor, and a memory coupled to the processor. The memory is encoded with instructions that when executed cause the processor to provide a training system for generating benchmark data for the knowledge canvassing system. The training system is configured to resolve a first entity to an entry in a knowledge base of reference documents, identify a second entity included in the entry, and assign a weight to the second entity based on a location of the second entity in the entry or a number of mentions of the second entity in the entry.

Abstract: A system includes a knowledge canvassing system executed by a computer, a processor, and a memory coupled to the processor. The memory is encoded with instructions that when executed cause the processor to provide a training system configured to generate benchmark data, each benchmark datum including a set of one or more benchmark input entities and a set of one or more benchmark output entities associated with the one or more benchmark input entities, query the knowledge canvassing system with each set of benchmark input entities, receive, for each set of benchmark input entities queried, an output result from the knowledge canvassing system that includes a set of zero or more knowledge canvassing system output entities, and generate an evaluation score for each set of knowledge canvassing system output entities based on a comparison of the knowledge canvassing system output entities with the set of benchmark output entities.

Abstract: A mechanism is provided in a data processing system for identifying nonsense passages in documents being ingested into a corpus. A natural language processing pipeline configured to execute in the data processing system receives an input document to be ingested into a corpus. The natural language processing pipeline divides the input document into a plurality of input passages. A filter component of the natural language processing pipeline identifies whether each input passage is a nonsense passage based on a value of a metric determined according to a set of feature counts. The natural language processing pipeline filters each input passage in the plurality of input passages based on whether the input passage is identified as a nonsense passage or not identified as a nonsense passage to form a filtered plurality of input passages. The natural language processing pipeline adds the filtered plurality of input passages into the corpus.

Abstract: A mechanism is provided in a data processing system for identifying nonsense passages. The mechanism annotates an input passage with linguistic features to form an annotated passage. The mechanism counts a number of instances of each type of linguistic feature in the annotated passage to form a set of feature counts. The mechanism determines a value for a metric based on the set of feature counts and compares the value for the metric to a predetermined model threshold. The mechanism identifies whether the input passage is a nonsense passage based on a result of the comparison.

Abstract: A mechanism is provided in a data processing system for identifying nonsense passages. An annotator in a natural language processing pipeline configured to execute in the data processing system annotates an input passage in a corpus with linguistic features to form an annotated passage. A domain-specific policy is associated with a domain of the corpus. A metric counters component in the natural language processing pipeline counts a number of instances of each type of linguistic feature in the annotated passage to form a set of feature counts. The metric counters component of the natural language processing pipeline determines a value for a metric based on the set of feature counts. The metric is specified in the domain-specific policy. A comparator component of the natural language processing pipeline compares the value for the metric to a predetermined model threshold. The threshold is specified in the domain-specific policy.

Abstract: A mechanism is provided in a data processing system for identifying nonsense passages in documents being ingested into a corpus. A natural language processing pipeline configured to execute in the data processing system receives an input document to be ingested into a corpus. The natural language processing pipeline divides the input document into a plurality of input passages. A filter component of the natural language processing pipeline identifies whether each input passage is a nonsense passage based on a value of a metric determined according to a set of feature counts. The natural language processing pipeline filters each input passage in the plurality of input passages based on whether the input passage is identified as a nonsense passage or not identified as a nonsense passage to form a filtered plurality of input passages. The natural language processing pipeline adds the filtered plurality of input passages into the corpus.