Abstract: In a computer system that includes a trained recurrent neural network (RNN), a computer-based method includes: producing a copy of the trained RNN; producing a version of the RNN prior to any training; trying to solve a control task for the RNN with the copy of the trained RNN and with the untrained version of the RNN; and in response to the copy of the trained RNN or the untrained version of the RNN solving the task sufficiently well: retraining the trained RNN with one or more traces (sequences of inputs and outputs) from the solution; and retraining the trained RNN based on one or more traces associated with other prior control task solutions, as well as retraining the RNN based on previously observed traces to predict environmental inputs and other data (which maybe consequences of executed control actions).

Abstract: A computer-based method includes receiving an input signal at a neuron in a computer-based neural network that includes a plurality of neuron layers, applying a first non-linear transform to the input signal at the neuron to produce a plain signal, and calculating a weighted sum of a first component of the input signal and the plain signal at the neuron. In a typical implementation, the first non-linear transform is a function of the first component of the input signal and at least a second component of the input signal.