Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using mixer neural networks. One of the methods includes obtaining one or more images comprising a plurality of pixels; determining, for each image of the one or more images, a plurality of image patches of the image, wherein each image patch comprises a different subset of the pixels of the image; processing, for each image of the one or more images, the corresponding plurality of image patches to generate an input sequence comprising a respective input element at each of a plurality of input positions, wherein a plurality of the input elements correspond to respective different image patches; and processing the input sequences using a neural network to generate a network output that characterizes the one or more images, wherein the neural network comprises one or more mixer neural network layers.

Abstract: Provided are systems and methods for predicting machine learning model performance from the model parameter values, including for use in making improved decisions with regard to early stopping of training procedures. As one example, the present disclosure discusses the prediction of the accuracy (e.g., relative to a defined task and testing dataset such as a computer vision task) of trained neural networks (e.g., convolutional neural networks (CNNs)), using only the parameter values (e.g., the values of the network's weights) as inputs. As such, one example aspect of the present disclosure is directed to computing systems that include and use a machine-learned performance prediction model that has been trained to predict performance values of machine-learned models based on their parameter values (e.g., weight values and/or hyperparameter values).