Abstract: Computer technology for determining and tagging parts of speech in a text (that is PoS ragging), where the context used by the natural language processing machine logic (for example, NLP software) includes both: (i) other words in the sentence under analysis where a given word to be tagged appears; and (ii) words in the other sentences besides the sentence under analysis. Other context sentences may be selected randomly, by Next Sentence Prediction technology and/or by choosing sentences in textual proximity to the sentence under analysis.

Abstract: A computer-implemented method is provided for data augmentation. The method includes receiving a set of different base models already pretrained and a set of different test cases. The method further includes collecting a plurality of prediction results of the set of different test cases from the set of different base models. The method also includes identifying a test case as a candidate for the data augmentation based on a number of models in the set of different base models which fail to solve the test case. The method additionally includes augmenting, by a processor device, the identified test case with additional data to form an augmented training dataset. The method further includes retraining at least some of the different base models with the augmented training dataset.

Abstract: A computer-implemented method is provided for data augmentation. The method includes calculating, by a hardware processor for each of words in a text data, a word replacement probability based on a word occurrence frequency in the text data, wherein the word replacement probability decreases with increasing word occurrence frequency. The method additionally includes selectively replacing at least one of the words in the text data with words predicted therefor by a Bidirectional Neural Network Language Model (BiNNLM) to generate augmented text data, based on the word replacement probability.

Abstract: A computer-implemented method is provided for data augmentation. The method includes receiving a set of different base models already pretrained and a set of different test cases. The method further includes collecting a plurality of prediction results of the set of different test cases from the set of different base models. The method also includes identifying a test case as a candidate for the data augmentation based on a number of models in the set of different base models which fail to solve the test case. The method additionally includes augmenting, by a processor device, the identified test case with additional data to form an augmented training dataset. The method further includes retraining at least some of the different base models with the augmented training dataset.

Abstract: A computer-implemented method, a computer program product, and a computer system for dependency tree-based data augmentation for sentence well-formedness judgement. A computer applies a dependency parser to generate a dependency tree for a sentence. The computer removes one or more nodes in the dependency tree, according to a removal ratio for a predetermined rating score. The computer generates, from the dependency tree, a partial tree for the sentence. The computer outputs a rated sentence based on the partial tree. The rated sentence is used as training data.

Abstract: A computer-implemented method for identifying an expression for a target concept, includes: obtaining a set of texts as a target set of texts, with each text being associated with one of images relevant to a target concept. Candidate expressions for the target concept are extracted from the target set of texts. The candidate expressions are characteristic of the target set of texts. Each image relevant to one of the candidate expressions is collected by using an image search engine. A target expression for the target concept is selected from the candidate expressions based on a comparison result of the target concept and the collected images.

Abstract: A computer-implemented method is provided for data augmentation. The method includes calculating, by a hardware processor for each of words in a text data, a word replacement probability based on a word occurrence frequency in the text data, wherein the word replacement probability decreases with increasing word occurrence frequency. The method additionally includes selectively replacing at least one of the words in the text data with words predicted therefor by a Bidirectional Neural Network Language Model (BiNNLM) to generate augmented text data, based on the word replacement probability.

Abstract: A computer-implemented method for finding an asymmetric relation between a plurality of target words is disclosed. The method includes preparing a plurality of image sets, each of which includes one or more images relevant to a corresponding one of the plurality of the target words. The method also includes obtaining a plurality of object labels for each of the plurality of image sets. The method further includes computing a representation for each of the plurality of the target words using the plurality of the object labels obtained for each of the plurality of image sets. The method includes further determining whether there is an asymmetric relation between the plurality of the target words using representations computed for the plurality of the target words.

Abstract: A computer-implemented method for finding an asymmetric relation between a plurality of target words is disclosed. The method includes preparing a plurality of image sets, each of which includes one or more images relevant to a corresponding one of the plurality of the target words. The method also includes obtaining a plurality of object labels for each of the plurality of image sets. The method further includes computing a representation for each of the plurality of the target words using the plurality of the object labels obtained for each of the plurality of image sets. The method includes further determining whether there is an asymmetric relation between the plurality of the target words using representations computed for the plurality of the target words.

Abstract: A computer-implemented method for identifying an expression for a target concept, includes: obtaining a set of texts as a target set of texts, with each text being associated with one of images relevant to a target concept. Candidate expressions for the target concept are extracted from the target set of texts. The candidate expressions are characteristic of the target set of texts. Each image relevant to one of the candidate expressions is collected by using an image search engine. A target expression for the target concept is selected from the candidate expressions based on a comparison result of the target concept and the collected images.

Abstract: A computer-implemented method, a computer program product, and a computer system for extracting an expression in a text for natural language processing. The computer system reads a text to generate a plurality of substrings in which each substring includes one or more units appearing in the text. The computer system obtains an image set for the each substring, using the one or more units as a query for an image search system; wherein the image set includes one or more images. The computer system calculates a deviation in the image set for the each substring. The computer system selects a respective one of the plurality of the substrings as an expression to be extracted, based on the deviation and a length of each substring.