Abstract: Prediction and training methods using recurrent neural networks are disclosed. In one aspect, the prediction method provides a sequence of input data applicable to a plurality of input connections, a plurality of hidden layer connections, and an ordered sequence of hidden layers comprising at least one recurrent hidden layer. Each hidden layer connection has an associated binary-valued hidden layer weight and each input connection is associated with a binary-valued input weight. For each time step, a derived hidden state vector is binarized in each hidden layer and a binary-valued vector representation of a next input datum of the input data sequence is applied to the input connections. For each hidden unit input, a first sum of connected weighted hidden states and a second sum of connected weighted input data is computed and linearly combined to determine an input state.