Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for decreasing neural network inference times using softmax approximation. One of the methods includes maintaining data specifying a respective softmax weight vector for each output in a vocabulary of possible neural network outputs; receiving a neural network input; processing the neural network input using one or more initial neural network layers to generate a context vector for the neural network input; and generating an approximate score distribution over the vocabulary of possible neural network outputs for the neural network input, comprising: processing the context vector using a screening model configured to predict a proper subset of the vocabulary for the context input; and generating a respective logit for each output that is in the proper subset, comprising applying the softmax weight vector for the output to the context vector.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for decreasing neural network inference times using softmax approximation. One of the methods includes maintaining data specifying a respective softmax weight vector for each output in a vocabulary of possible neural network outputs; receiving a neural network input; processing the neural network input using one or more initial neural network layers to generate a context vector for the neural network input; and generating an approximate score distribution over the vocabulary of possible neural network outputs for the neural network input, comprising: processing the context vector using a screening model configured to predict a proper subset of the vocabulary for the context input; and generating a respective logit for each output that is in the proper subset, comprising applying the softmax weight vector for the output to the context vector.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for polynomial mapping of data for linear SVMs. In one aspect, a method includes training a linear classifier by receiving feature vectors and generating a condensed representation of a mapped vector corresponding to a polynomial mapping of each feature vector, the condensed representation including an index into a weight vector for each non-zero component of the mapped vector. A linear classifier is trained on the condensed representations. In another aspect, a method includes receiving a feature vector, identifying non-zero components resulting from a polynomial mapping of the feature vector, and mapping the combination of one or more elements of each non-zero component to a weight in a weight vector to determine a set of weights. The feature vector is classified according to a classification score derived by summing the set of weights.