Abstract: A computer implemented method of pre-processing an input text for a natural language processing operation based on a training corpus of documents, can include accessing a set of stop words including predetermined words for de-emphasis in the text for the natural language processing operation, the set of stop words being separated into at least two subsets including a first subset and a second subset, the second subset containing stop words predetermined to be of potential semantic significance to documents in the training corpus; tokenizing documents in a training corpus to an ordered set of corpus tokens; removing, from the set of corpus tokens, tokens corresponding to stop words in the first subset of stop words; generating a set of n-grams by identifying n-grams from groups of tokens in the set of corpus tokens based on predetermined rules for n-gram identification; tokenizing the input text to an ordered set of input text tokens; identifying groups of tokens in the set of input text tokens corresponding t