Abstract: Neural network inference may be performed by configuration of a device including an accumulation memory, a plurality of convolution modules configured to perform mathematical operations on input values, a plurality of adder modules configured to sum values output from the plurality of convolution modules, and a plurality of convolution output interconnects connecting the plurality of convolution modules, the plurality of adder modules, and the accumulation memory. The accumulation memory is an accumulation memory allocation of a writable memory block having a reconfigurable bank width, and each bank of the accumulation memory allocation is a virtual combination of consecutive banks of the writable memory block.

Abstract: Neural network inference may be performed by configuration of a device including an accumulation memory, a plurality of convolution modules configured to perform mathematical operations on input values, a plurality of adder modules configured to sum values output from the plurality of convolution modules, and a plurality of convolution output interconnects connecting the plurality of convolution modules, the plurality of adder modules, and the accumulation memory. The accumulation memory is an accumulation memory allocation of a writable memory block having a reconfigurable bank width, and each bank of the accumulation memory allocation is a virtual combination of consecutive banks of the writable memory block.

Abstract: Neural network inference may be performed by an apparatus or integrated circuit configured to perform mathematical operations on activation data stored in an activation data memory and weight values stored in a weight memory, to store values resulting from the mathematical operations onto an accumulation memory, to perform activation operations on the values stored in the accumulation memory, to store resulting activation data onto the activation data memory, and to perform inference of a neural network by feeding and synchronizing instructions from an external memory.

Abstract: Neural network accelerator hardware-specific division of inference may be performed by operations including obtaining a computational graph and a hardware chip configuration. The operations also include dividing inference of the plurality of layers into a plurality of groups. Each group includes a number of sequential layers based on an estimate of duration and energy consumption by the hardware chip to perform inference of the neural network by performing the mathematical operations on activation data, sequentially by layer, of corresponding portions of layers of each group. The operations further include generating instructions for the hardware chip to perform inference of the neural network, sequentially by group, of the plurality of groups.

Abstract: Neural network accelerator hardware-specific division of inference may be performed by operations including obtaining a computational graph and a hardware chip configuration. The operations also include dividing inference of the plurality of layers into a plurality of groups. Each group includes a number of sequential layers based on an estimate of duration and energy consumption by the hardware chip to perform inference of the neural network by performing the mathematical operations on activation data, sequentially by layer, of corresponding portions of layers of each group. The operations further include generating instructions for the hardware chip to perform inference of the neural network, sequentially by group, of the plurality of groups.

Abstract: Neural network inference may be performed by configuration of a device including a plurality of convolution modules, a plurality of adder modules, an accumulation memory, and a convolution output interconnect control module configured to open and close convolution output interconnects among a plurality of convolution output interconnects connecting the plurality of convolution modules, the plurality of adder modules, and the accumulation memory. Inference may be performed while the device is configured according to at least one convolution output connection scheme whereby each convolution module has no more than one open direct connection through the plurality of convolution output interconnects to the accumulation memory or one of the plurality of adder modules. The device includes a convolution output interconnect control module to configure the plurality of convolution output interconnects according to the at least one convolution output connection scheme.