Abstract: A neural network IC 31 includes n dedicated processing elements (PEs) 62, an output register 66 for storing the PEs' outputs so that they are immediately accessible to all of the PEs, a number of output circuits 78 that are connected to selected PEs to provide binary outputs, and a timing circuit 74. Each of the PEs includes a weight memory 90 for storing input, output and bias weight arrays, a first in first out (FIFO) memory 88 for storing input data, a dot product circuit 92 and an activation circuit 94. The dot product circuit computes a dot product of the input weight array and the contents of the FIFO memory, a dot product of the output weight array and the contents of the output register, a dot product of the bias value and a constant, and sums the three results. The activation circuit maps the output of the dot product circuit through an activation function to produce the PE's output.