Abstract: A method for training a recurrent network represented by x(k+1)=f(W x(k)), where W is a weight matrix, x is the output of the network, and K is a time index includes (a) determining the weight matrix at a first time increment, (b) incrementing the time increment associated with a received data point, and (c) determining a change in the weight matrix at the incremented time interval according to the formula: ? ? ? ? W ? ( K ) = ? ? ? ? W ? ( K - 1 ) + ? ? ? ? ? ? ( K ) ? x T ? ( K - 1 ) ? ? ? V - 1 ? ( K - 1 ) - B ? ( K - 1 ) ? ? ? V - 1 ? ( K - 1 ) ? ? ? x ? ? ? ( K - 1 ) ? [ V - 1 ? ( K - 1 ) ? ? ? x ? ? ? ( K - 1 ) ] T 1 + x T ? ( K - 1 ) ? ? ? V - 1 ? ( K - 1 ) ? ? ? x ? ? ? ( K - 1 )

Abstract: The present invention is a fully connected feed forward network that includes at least one hidden layer 16. The hidden layer 16 includes nodes 20 in which the output of the node is fed back to that node as an input with a unit delay produced by a delay device 24 occurring in the feedback path 22 (local feedback). Each node within each layer also receives a delayed output (crosstalk) produced by a delay unit 36 from all the other nodes within the same layer 16. The node performs a transfer function operation based on the inputs from the previous layer and the delayed outputs. The network can be implemented as analog or digital or within a general purpose processor. Two teaching methods can be used: (1) back propagation of weight calculation that includes the local feedback and the crosstalk or (2) more preferably a feed forward gradient decent which immediately follows the output computations and which also includes the local feedback and the crosstalk.