Abstract: A neural network includes one layer of neurons including neurons having neuron connections to neurons in the layer and input connections to a network input. The neuron connections and the input connections have respective neuron connection weights and input connection weights. The neurons have neuron responses set by an activation function with activation values and include activation function computing circuits configured for computing current activation values of the activation function as a function of previous activation values of the activation function and current network input values.