Abstract: A method that is implemented by one or more data processing devices can include receiving a training set that includes a plurality of representations of topological structures in patterns of activity in a source neural network and training a neural network using the representations either as an input to the neural network or as a target answer vector. The activity is responsive to an input into the source neural network.

Abstract: A brain-inspired cognitive learning method can obtain good learning results in various environments and tasks by selecting the most suitable algorithm models and parameters based on the environments and tasks, and can correct wrong behavior. The framework includes four main modules: a cognitive feature extraction module, a cognitive control module, a learning network module, and a memory module. The memory module includes a data base, a cognitive case base, and an algorithm and hyper-parameter base, which store data of dynamic environments and tasks, cognitive cases, and concrete algorithms and hyper-parameter values, respectively. For dynamic environments and tasks, the most suitable algorithm model and hyper-parameter combination can be flexibly selected. In addition, with “good money drives out bad”, mislabeled data is corrected using correctly labeled data, to achieve robustness of training data.

Abstract: Some embodiments provide a compiler for optimizing the implementation of a machine-trained network (e.g., a neural network) on an integrated circuit (IC). The compiler of some embodiments receives a specification of a machine-trained network including multiple layers of computation nodes and generates a graph representing options for implementing the machine-trained network in the IC. The compiler, as part of generating the graph, in some embodiments, determines whether any set of channels contains no non-zero values (i.e., contains only zero values). For sets of channels that include no non-zero values, some embodiments perform a zero channel removal operation to remove all-zero channels wherever possible. In some embodiments, zero channel removal operations include removing input channels, removing output channels, forward propagation, and backward propagation of channels and constants.

Abstract: A feedforward generative neural network that generates an output example that includes multiple output samples of a particular type in a single neural network inference. Optionally, the generation may be conditioned on a context input. For example, the feedforward generative neural network may generate a speech waveform that is a verbalization of an input text segment conditioned on linguistic features of the text segment.

Abstract: A generator network of a variational autoencoder can be trained to approximate a simulator and generate a first result. The simulator is associated with input data, based on which the simulator outputs output data. A training data set for the generator network can include the simulator's input data and output data. Based on the simulator's output data and the first result of the generator network, an inference network of the variational autoencoder can be trained to generate a second result. The second result of the trained inference network inverts the first result of the generator and approximates the simulator's input data. The trained inference network can function as an inverted simulator.

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors for distributed training of a neural network. One of the methods includes receiving, at each of the plurality of devices, a respective batch; performing, by each device, a forward pass comprising, for each batch normalization layer: generating, by each of the devices, a respective output of the corresponding other layer for each training example in the batch, determining, by each of the devices, a per-replica mean and a per-replica variance; determining, for each sub-group, a distributed mean and a distributed variance from the per-replica means and the per-replica variances for the devices in the sub-group; and applying, by each device, batch normalization to the respective outputs of the corresponding other layer generated by the device using the distributed mean and the distributed variance for the sub-group to which the device belongs.

Abstract: An electronic apparatus for performing machine learning a method of machine learning, and a non-transitory computer-readable recording medium are provided. The electronic apparatus includes an operation module configured to include a plurality of processing elements arranged in a predetermined pattern and share data between the plurality of processing elements which are adjacent to each other to perform an operation; and a processor configured to control the operation module to perform a convolution operation by applying a filter to input data, wherein the processor controls the operation module to perform the convolution operation by inputting each of a plurality of elements configuring a two-dimensional filter to the plurality of processing elements in a predetermined order and sequentially applying the plurality of elements to the input data.

Abstract: A classification device includes a generation unit, a learning unit, a classification unit, and an output control unit. The generation unit generates pseudo data having a feature similar to a feature of training data. The learning unit learns, by using the training data and the pseudo data, a classification model that classifies data into one of a pseudo class for classifying the pseudo data and a plurality of classification classes other than the pseudo class and that is constructed by a neural network. The classification unit classifies, by using the classification model, input data as a target for classification into one of the pseudo class and the plurality of classification classes. The output control unit outputs information indicating that the input data classified into the pseudo class is data not belonging to any of the plurality of classification classes.

Abstract: A multiply-accumulate method and architecture are disclosed. The architecture includes a plurality of networks of non-volatile memory elements arranged in tiled columns. Logic digitally modulates the equivalent conductance of individual networks among the plurality of networks to map the equivalent conductance of each individual network to a single weight within the neural network. A first partial selection of weights within the neural network is mapped into the equivalent conductances of the networks in the columns to enable the computation of multiply-and-accumulate operations by mixed-signal computation. The logic updates the mappings to select a second partial selection of weights to compute additional multiply-and-accumulate operations and repeats the mapping and computation operations until all computations for the neural network are completed.

Abstract: Embodiments of the present invention provide the use of a conditional Generative Adversarial Network (GAN) to simultaneously correct and downscale (super-resolve) global ensemble weather or climate forecasts. Specifically, a generator deep neural network (G-DNN) in the cGAN comprises a corrector DNN (C-DNN) followed by a super-resolver DNN (SR-DNN). The C-DNN bias-corrects coarse, global meteorological forecasts, taking into account other relevant contextual meteorological fields. The SR-DNN downscales bias-corrected C-DNN output into G-DNN output at a higher target spatial resolution. The GAN is trained in three stages: C-DNN training, SR-DNN training, and overall GAN training, each using separate loss functions. Embodiments of the present invention significantly outperform an interpolation baseline, and approach the performance of operational regional high-resolution forecast models across an array of established probabilistic metrics.

Abstract: A meta-learning system includes an inner function computation module, adapted to compute output data from applied input data according to an inner model function, depending on model parameters; an error computation module, adapted to compute errors indicating mismatches between the computed output data and target values; a state update module, adapted to update the model parameters of the inner model function according to an updated state, updated based on a current state of the state update module, in response to an error received from the error computation module. The state update module is learned to adjust the model parameters of the inner model function, such that a following training of the inner model function with training data is improved.

Abstract: Provided are a computer program product, system, and method for ranking action sets comprised of actions for an event to optimize action set selection. Information is maintained on actions for a plurality of events. Each action indicates an action value of the action to the user and event weights of the action with respect to a plurality of the events. A determination is made of actions sets having at least one action to perform for the event. For each determined action set, a rank of the action set is calculated as a function of the action value for each action in the action set and an event weight of the action with respect to the event. At least one action set is presented to the user for consideration. In response to receiving user feedback, an adjusted rank is set for at least one of the presented action sets.

Abstract: In a computer system that includes a trained recurrent neural network (RNN), a computer-based method includes: producing a copy of the trained RNN; producing a version of the RNN prior to any training; trying to solve a control task for the RNN with the copy of the trained RNN and with the untrained version of the RNN; and in response to the copy of the trained RNN or the untrained version of the RNN solving the task sufficiently well: retraining the trained RNN with one or more traces (sequences of inputs and outputs) from the solution; and retraining the trained RNN based on one or more traces associated with other prior control task solutions, as well as retraining the RNN based on previously observed traces to predict environmental inputs and other data (which maybe consequences of executed control actions).

Abstract: A processor-implemented neural network method includes acquiring connection weight of an analog neural network (ANN) node of a pre-trained ANN; and determining, a firing rate of a spiking neural network (SNN) node of an SNN, corresponding to the ANN node, based on an activation of the ANN node which is determined based on the connection weight. and the firing rate is also determined based on information indicating a timing at which the SNN node initially fires.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating document vector representations. One of the methods includes obtaining a new document; and determining a vector representation for the new document using a trained neural network system, wherein the trained neural network system has been trained to receive an input document and a sequence of words from the input document and to generate a respective word score for each word in a set of words, wherein each of the respective word scores represents a predicted likelihood that the corresponding word follows a last word in the sequence in the input document, and wherein determining the vector representation for the new document using the trained neural network system comprises iteratively providing each of the plurality of sequences of words to the trained neural network system to determine the vector representation for the new document using gradient descent.

Abstract: Neural network processing hardware using parallel computational architectures with reconfigurable core-level and vector-level parallelism is provided. In various embodiments, a neural network model memory is adapted to store a neural network model comprising a plurality of layers. Each layer has at least one dimension and comprises a plurality of synaptic weights. A plurality of neural cores is provided. Each neural core includes a computation unit and an activation memory. The computation unit is adapted to apply a plurality of synaptic weights to a plurality of input activations to produce a plurality of output activations. The computation unit has a plurality of vector units. The activation memory is adapted to store the input activations and the output activations. The system is adapted to partition the plurality of cores into a plurality of partitions based on dimensions of the layer and the vector units.

Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.

Abstract: Systems, computer program products, and computer-implemented methods for determining relationships between one or more outputs of a first model and one or more inputs of a second model that collectively represent a real world system, and chaining the models together. For example, the system described herein may determine how to chain a plurality of models by training an artificial intelligence system using the nodes of the models such that the trained artificial intelligence system predicts related output and input node connections. The system may then link related nodes to chain the models together. The systems, computer program products, and computer-implemented methods may thus, according to various embodiments, enable a plurality of discrete models to be optimally chained.

Abstract: Systems, methods, articles of manufacture, and computer program products to: train a prediction model using a machine learning process, the prediction model configured to estimate whether further application of a hyperparameter tuning technique will cause an improvement in at least one of the hyperparameters; select the hyperparameters using the tuning technique; apply the prediction model to determine if further adjustment of the hyperparameters is likely to improve the success metric; and terminate the tuning technique when: accuracy of the prediction model in predicting improvement in a hyperparameter is above a predetermined accuracy threshold, and the prediction model predicts that further application of the tuning technique will not result in an improvement to the hyperparameter; or the accuracy of the prediction model in predicting improvement in the parameter is below the predetermined accuracy threshold, and an accuracy of hyperparameter adjustment is determined to be below a predetermined adjustment

Abstract: A system includes a determination component that determines output for successively larger neural networks of a set; and a consensus component that determines consensus between a first neural network and a second neural network of the set. A linear chain of increasingly complex neural networks trained on progressively larger inputs is utilized (e.g., increasingly complex neural networks is generally representative of increased accuracy). Outputs of progressively networks are computed until a consensus point is reached—where two or more successive large networks yield a same inference output. At such point of consensus the larger neural network of the set reaching consensus can be deemed appropriately sized (or of sufficient complexity) for a classification task at hand.