Abstract: Example embodiments provide techniques for configuring a natural-language processing system to perform a new function given at least one sample invocation of the function. The training data consisting of the sample invocation may be augmented by determining which subset of available training data most closely resembles the sample invocation and/or function. The effect of re-training a component this this augmented training data may be determined, and an annotator may review any annotations corresponding to the invocation if the effect is large.