Abstract: An accelerator for training deep neural networks is provided. The accelerator includes a baseline architecture having an input buffer, a weight buffer, an output buffer, a buffer controller, and a two-dimensional array of processing elements. The array of processing elements is used in both convolutional and fully connected layers. The convolutional layer includes multiple filters. The output of each said filter in said convolutional layers is achieved by a weighted summation. In a preferred embodiment, each convolutional and fully connected layer is equipped with input/output buffers that fetch/store the input/output data. In a particularly preferred embodiment, each processing element can access the weight buffer that holds the weight vector.

Abstract: The disclosed invention provides a novel architecture that reduces the computation time of stochastic computing-based multiplications in the convolutional layers of convolutional neural networks (CNNs). Each convolution in a CNN is composed of numerous multiplications where each input value is multiplied by a weight vector. Subsequent multiplications are performed by multiplying the input and differences of the successive weights. Leveraging this property, disclosed is a differential Multiply-and-Accumulate unit to reduce the time consumed by convolutions in the architecture. The disclosed architecture offers 1.2× increase in speed and 2.7× increase in energy efficiency compared to known convolutional neural networks.

Abstract: Inaccuracy of computations is an important challenge with the Stochastic Computing (SC) paradigm. Recently, deterministic approaches to SC are proposed to produce completely accurate results with SC circuits. Instead of random bit-streams, the computations are performed on structured deterministic bit-streams. However, current deterministic methods take a large number of clock cycles to produce correct result. This long processing time directly translates to very high energy consumption. This invention proposes a design methodology based on the Residue Number Systems (RNS) to mitigate the long processing time of the deterministic methods. Compared to the state-of-the-art deterministic methods of SC, the proposed approach delivers improvements in terms of processing time and energy consumption.