Abstract: The present invention relates to methods and systems for improving the efficiency of artificial neural networks by configuring them to implement linear dynamical systems that compute state updates in linear time. More specifically, the present invention specifies methods and systems for setting the weights of least one linear artificial neural network layer by (a) selecting an set of basis vectors that define a desired impulse response for the layer, (b) deriving a matrix that produces this desired impulse response over a target temporal window, (c) deriving a matrix that dampens the impulse response to zero outside of the target window, and (d) combining these two matrices together through addition to obtain the layer's recurrent weights. Systems composed of at least one such linear layer are applied to a time series of input data elements to produce outputs that encode the results of pattern classification, signal processing, data representation, and data generation tasks.