Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for executing ensemble models that include multiple reservoir computing neural networks. One of the methods includes executing an ensemble model comprising a plurality of reservoir computing neural networks, the ensemble model having been trained by operations comprising, at each training stage in a sequence of training stages: obtaining a current ensemble model that comprises a plurality of current reservoir computing neural networks; determining a respective performance measure for each current reservoir computing neural network in the current ensemble model; determining one or more new reservoir computing neural networks to be added to the current ensemble model based on the performance measures for the current reservoir computing neural networks; and adding the new reservoir computing neural networks to the current ensemble model.

Abstract: In one aspect, there is provided a method performed by one or more data processing apparatus, the method includes: obtaining a network input including a respective data element at each input position in a sequence of input positions, and processing the network input using a neural network to generate a network output that defines a prediction related to the network input, where the neural network includes a sequence of encoder blocks and a decoder block, where each encoder block has a respective set of encoder block parameters, and where the set of encoder block parameters includes multiple brain emulation parameters that, when initialized, represent biological connectivity between multiple biological neuronal elements in a brain of a biological organism.