Abstract: Method and system of training a student neural network using adversarial learning and knowledge distillation, including: training a generator to generate adversarial data samples for respective training data samples by masking parts of the training data samples with an objective of maximizing a divergence between output predictions generated by the student neural network and a teacher neural network model for the adversarial data samples; and training the student neural network based on objectives of (i) minimizing a divergence between output predictions generated by the student neural network and the teacher neural network model for the adversarial data samples, and (ii) minimizing a divergence between output predictions generated by the student neural network and the teacher neural network model for the training data samples.

Abstract: Methods, devices and processor-readable media for re-weighting to improve knowledge distillation are described. A reweighting module may be used to determine relative weights to assign to a ground truth label and dark knowledge distilled from the teacher (i.e. the teacher output logits used as soft labels). A meta-reweighting method is described to optimize the weights for a given labeled data sample.

Abstract: In at least one broad aspect, described herein are systems and methods in which a latent representation shared between two languages is built and/or accessed, and then leveraged for the purpose of text generation in both languages. Neural text generation techniques are applied to facilitate text generation, and in particular the generation of sentences (i.e., sequences of words or subwords) in both languages, in at least some embodiments.

Abstract: In at least one broad aspect, described herein are systems and methods in which a latent representation shared between two languages is built and/or accessed, and then leveraged for the purpose of text generation in both languages. Neural text generation techniques are applied to facilitate text generation, and in particular the generation of sentences (i.e., sequences of words or subwords) in both languages, in at least some embodiments.