Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for classifying an input time series into a class from a set of classes. In one aspect, a method comprises: receiving an input time series; processing the input time series using a reconstruction model to generate a reconstruction model output that comprises a plurality of channels, wherein each channel of the plurality of channels defines a respective output time series, and wherein each channel of the plurality of channels corresponds to a respective class from the set of classes; determining a respective reconstruction error for each channel of the plurality of channels based on an error between: (i) the output time series defined by the channel, and (ii) the input time series; and classifying the input time series as being included in a class from the set of classes based on the reconstruction errors.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for executing ensemble models that include multiple reservoir computing neural networks. One of the methods includes executing an ensemble model comprising a plurality of reservoir computing neural networks, the ensemble model having been trained by operations comprising, at each training stage in a sequence of training stages: obtaining a current ensemble model that comprises a plurality of current reservoir computing neural networks; determining a respective performance measure for each current reservoir computing neural network in the current ensemble model; determining one or more new reservoir computing neural networks to be added to the current ensemble model based on the performance measures for the current reservoir computing neural networks; and adding the new reservoir computing neural networks to the current ensemble model.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing a reservoir computing neural network. In one aspect there is provided a reservoir computing neural network comprising: (i) a brain emulation sub-network, and (ii) a prediction sub-network. The brain emulation sub-network is configured to process the network input in accordance with values of a plurality of brain emulation sub-network parameters to generate an alternative representation of the network input. The prediction sub-network is configured to process the alternative representation of the network input in accordance with values of a plurality of prediction sub-network parameters to generate the network output. The values of the brain emulation sub-network parameters are determined before the reservoir computing neural network is trained and are not adjusting during training of the reservoir computing neural network.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an artificial neural network architecture based on a synaptic connectivity graph. According to one aspect, there is provided a method comprising: obtaining a synaptic resolution image of at least a portion of a brain of a biological organism; processing the image to identify: (i) a plurality of neurons in the brain, and (ii) a plurality of synaptic connections between pairs of neurons in the brain; generating data defining a graph representing synaptic connectivity between the neurons in the brain; determining an artificial neural network architecture corresponding to the graph representing the synaptic connectivity between the neurons in the brain; and processing a network input using an artificial neural network having the artificial neural network architecture to generate a network output.

Abstract: In one aspect, there is provided a method performed by one or more data processing apparatus for training a neural network, the method including: obtaining a set of training examples, where each training example includes: (i) a training input, and (ii) a target output, and training the neural network on the set of training examples. Training the neural network can include, for each training example: processing the training input using the neural network to generate a corresponding training output, updating current values of at least a set of encoder sub-network parameters and a set of decoder sub-network parameters by a supervised update, and updating current values of at least a set of brain emulation sub-network parameters by an unsupervised update based on correlations between activation values generated by artificial neurons of the neural network during processing of the training input by the neural network.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing defect detection using brain emulation neural networks.

Abstract: In one aspect, there is provided a method performed by one or more data processing apparatus, the method includes: obtaining a network input including a respective data element at each input position in a sequence of input positions, and processing the network input using a neural network to generate a network output that defines a prediction related to the network input, where the neural network includes a sequence of encoder blocks and a decoder block, where each encoder block has a respective set of encoder block parameters, and where the set of encoder block parameters includes multiple brain emulation parameters that, when initialized, represent biological connectivity between multiple biological neuronal elements in a brain of a biological organism.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing remote sensing data using brain emulation neural networks. One of the methods includes obtaining an aerial image of a plurality of agricultural plots; processing the aerial image using an encoder subnetwork of a segmentation neural network to generate an encoder subnetwork output; processing the encoder subnetwork output using a brain emulation subnetwork of the segmentation neural network to generate a brain emulation subnetwork output; processing the brain emulation subnetwork output using a decoder subnetwork of the segmentation neural network to generate a network output that defines a segmentation of the aerial image into a plurality of categories including at least one agricultural plot category; and identifying at least one of the plurality of agricultural plots in the aerial image from the segmentation of the aerial image.

Abstract: In one aspect, there is provided a method performed by one or more data processing apparatus that includes obtaining a network input and processing the network input using a neural network to generate a network output that defines a prediction for the network input. The method further includes processing the network input using an encoding sub-network of the neural network to generate an embedding of the network input, processing the embedding of the network input using a brain hybridization sub-network of the neural network to generate an alternative embedding of the network input, and processing the alternative embedding of the network input using a decoding sub-network of the neural network to generate the network output that defines the prediction for the network input.

Abstract: A sensor device includes a sensor housing defining a channel extending along a channel axis through the housing from a first side of the sensor housing to a second side of the sensor housing opposite the first side, at least one contact electrode extending from the first side of the housing, an electrically-conducting lead attached to the housing in electrical communication with the at least one contact electrode, and a locking mechanism located in the channel permitting one-way axial motion of a thread threaded through the channel from the first side to the second side.

Abstract: In one aspect, there is provided a method performed by one or more data processing apparatus, the method including: obtaining data defining a connectivity graph that represents synaptic connectivity between multiple biological neuronal elements in a brain of a biological organism, where the connectivity graph includes: multiple nodes, and multiple edges that each connect a respective pair of nodes, determining a partition of the connectivity graph into multiple community sub-graphs by performing an optimization that encourages a higher measure of connectedness between nodes included within each community sub-graph relative to nodes included in different community sub-graphs, and selecting a neural network architecture for performing a machine learning task using multiple community sub-graphs determined by the optimization that encourages the higher measure of connectedness between nodes included within each community sub-graph relative to nodes included in different community sub-graphs.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying one or more regions of a brain of a biological organism that are predicted to be functionally-specialized for performing a task. In one aspect, a method comprises: obtaining data defining a synaptic connectivity graph representing synaptic connectivity between neurons in the brain of the biological organism; identifying a plurality of sub-graphs of the synaptic connectivity graph; determining, for each sub-graph of the plurality of sub-graphs, a performance measure characterizing a performance of a neural network having a neural network architecture that is specified by the sub-graph in accomplishing the task; and determining, based on the performance measures, that one or more sub-graphs of the plurality of sub-graphs correspond to regions of the brain of the biological organism that are predicted to be functionally-specialized for performing the task.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a student neural network. In one aspect, there is provided a method comprising: processing a training input using the student neural network to generate a student neural network output comprising a respective score for each of a plurality of classes; processing the training input using a brain emulation neural network to generate a brain emulation neural network output comprising a respective score for each of the plurality of classes; and adjusting current values of the student neural network parameters using gradients of an objective function that characterizes a similarity between: (i) the student neural network output for the training input, and (ii) the brain emulation neural network output for the training input.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a student neural network. In one aspect, there is provided a method comprising: processing a training input using the student neural network to generate an output for the training input; processing the student neural network output using a discriminative neural network to generate a discriminative score for the student neural network output, wherein the discriminative score characterizes a prediction for whether the network input was generated using: (i) the student neural network, or (ii) a brain emulation neural network; and adjusting current values of the student neural network parameters using gradients of an objective function that depends on the discriminative score for the student neural network output.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting a neural network architecture for performing a machine learning task. In one aspect, a method comprises: obtaining data defining a synaptic connectivity graph representing synaptic connectivity between neurons in a brain of a biological organism; generating data defining a plurality of candidate graphs based on the synaptic connectivity graph; determining, for each candidate graph, a performance measure on a machine learning task of a neural network having a neural network architecture that is specified by the candidate graph; and selecting a final neural network architecture for performing the machine learning task based on the performance measures.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing a reservoir computing neural network. In one aspect there is provided a reservoir computing neural network comprising: (i) a brain emulation sub-network, and (ii) a prediction sub-network. The brain emulation sub-network is configured to process the network input in accordance with values of a plurality of brain emulation sub-network parameters to generate an alternative representation of the network input. The prediction sub-network is configured to process the alternative representation of the network input in accordance with values of a plurality of prediction sub-network parameters to generate the network output. The values of the brain emulation sub-network parameters are determined before the reservoir computing neural network is trained and are not adjusting during training of the reservoir computing neural network.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an artificial neural network architecture based on a synaptic connectivity graph. According to one aspect, there is provided a method comprising: obtaining a synaptic resolution image of at least a portion of a brain of a biological organism; processing the image to identify: (i) a plurality of neurons in the brain, and (ii) a plurality of synaptic connections between pairs of neurons in the brain; generating data defining a graph representing synaptic connectivity between the neurons in the brain; determining an artificial neural network architecture corresponding to the graph representing the synaptic connectivity between the neurons in the brain; and processing a network input using an artificial neural network having the artificial neural network architecture to generate a network output.

Abstract: Methods, systems, and computer programs encoded on a computer storage medium, for improving EEG measurements by identifying artifacts present in EEG measurements and providing a real-time indication to a user of likely artifacts in EEG measurements are described. EEG measurements of a patient can be obtained by placing a wearable device or EEG cap on a patient's head. Sensors in the cap provide EEG data to a computing device that processes the data to identify one or more artifacts in the EEG data. The artifacts can be identified by conducting one or more operations of determining the signal to noise ratio of the line noise, calculating mutual information between sensor pairs, and applying the p-welch method. Based on the types of artifacts identified, the computing device can output an indicator that provides feedback to the technician performing an EEG test to make adjustments to the test setup.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining an artificial neural network architecture corresponding to a sub-graph of a synaptic connectivity graph. In one aspect, there is provided a method comprising: obtaining data defining a graph representing synaptic connectivity between neurons in a brain of a biological organism; determining, for each node in the graph, a respective set of one or more node features characterizing a structure of the graph relative to the node; identifying a sub-graph of the graph, comprising selecting a proper subset of the nodes in the graph for inclusion in the sub-graph based on the node features of the nodes in the graph; and determining an artificial neural network architecture corresponding to the sub-graph of the graph.

Abstract: In one aspect, there is provided a method performed by one or more data processing apparatus for selecting actions to be performed by an agent interacting with an environment, the method including, at each of multiple time steps, receiving an observation characterizing a current state of the environment at the time step, providing an input including the observation to an action selection neural network having a brain emulation sub-network with an architecture that is based on synaptic connectivity between biological neurons in a brain of a biological organism, processing the input including the observation characterizing the current state of the environment at the time step using the action selection neural network having the brain emulation sub-network to generate an action selection output, and selecting an action to be performed by the agent at the time step based on the action selection output.