Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes a neural network configured to perform the machine learning task, the neural network including one or more expert neural network blocks that each include multiple routers and multiple expert neural networks.

Abstract: Methods and systems for performing black box optimization to identify an output that optimizes an objective.