Abstract: Embodiments describe throttling cores in an IC based on the datatype. Government regulations or export control may indicate the maximum number of operations that can be performed in the IC. However, the IC may perform operations at different rates. For example, the maximum number of operations that an IC can perform in a given time period can be different depending on the datatypes—e.g., 200 Tera Operations per Second (TOPS) for INT4 datatypes but only 100 TOPS for INT8 datatypes. Embodiments herein perform throttling based on particular datatypes so the IC is not unnecessarily throttled.

Abstract: Embodiments herein describe a system including a direct memory access (DMA) engine configured to receive first data stored in a local memory and an artificial intelligence (AI) engine configured to receive second data bypassing the DMA engine. The first data includes matrix-matrix instructions and the second data includes vector-matrix instructions. The second data is configured to be directly sent to the AI engine to avoid additional overhead of DMA data transfer latency, reduce memory bandwidth usage, and minimize power consumption.

Abstract: A device may include an array of data processing engines (DPEs) on a die and an event broadcast network. Each of the DPEs includes a core, a memory module, event logic in at least one of the core or the memory module, and an event broadcast circuitry coupled to the event logic. The event logic is capable of detecting an occurrence of one or more events in the core or the memory module. The event broadcast circuitry is capable of receiving an indication of a detected event detected by the event logic. The event broadcast network includes interconnections between the event broadcast circuitry of the DPEs. Detected events can trigger or initiate various responses, such as debugging, tracing, and profiling.

Abstract: Examples herein describe techniques for reducing the amount of memory used during weight sparsity. When decompressing the weights, the uncompressed weight data typically has many zero values. By knowing the location of these zero values (e.g., their indices in a weight matrix), the processor core can prune some of the activations (e.g., logically reduce the size of the activation matrix) which improves the efficiency of the processor core. In embodiments herein, the processor core includes logic for identifying the indices of the non-zero value after decompressing the compressed weights. These indices can then be used to prune the activations to improve the efficiency of the processor core.

Abstract: Embodiments herein describe a hardware accelerator with an array of data processing engines (DPEs) which includes a controller (e.g., a microcontroller) for multiple columns of the array. The controllers can be hardened circuitry that executes software code (or firmware) that controls the hardware accelerator. In one embodiment, the task of the controller is to control and orchestrate the functions performed by the hardware accelerator.

Abstract: Examples herein describe techniques for reducing the amount of memory used during weight sparsity. When decompressing the weights, the uncompressed weight data typically has many zero values. By knowing the location of these zero values (e.g., their indices in a weight matrix), the processor core can prune some of the activations (e.g., logically reduce the size of the activation matrix) which improves the efficiency of the processor core. In embodiments herein, the processor core includes logic for identifying the indices of the non-zero value after decompressing the compressed weights. These indices can then be used to prune the activations to improve the efficiency of the processor core.

Abstract: Examples herein describe an array of controllers. The array includes a first controller having a first memory and a first processor and a second controller having a second memory and a second processor. The first controller is configured to execute a first segment of control code. The control code is compiled based on one or more instructions included in a user defined application. The second controller is configured to execute a second segment of the control code. The one or more instructions included in the user defined application are executable by executing the first segment of the control code and the second segment of the control code.

Abstract: Embodiments herein describe a hardware accelerator with an array of data processing engines (DPEs) which includes a controller (e.g., a microcontroller) for multiple columns of the array. The controllers can be hardened circuitry that executes software code (or firmware) that controls the hardware accelerator. In one embodiment, the task of the controller is to control and orchestrate the functions performed by the hardware accelerator.

Abstract: Embodiments herein describe using DMA circuitry in multiple tiles in a hardware accelerator array to program the DMA operations within the array. For example, a system on a chip (SoC) may include a controller that is external to the hardware accelerator array. While the controller can be used to program the DMA circuitry within the array, this can be slow since the controller may be compute limited. Instead, the embodiments herein describe techniques where the controller is provided pointers to the register read and write corresponding to the DMA operations. The controller can provide these pointers to multiple DMA engines in the hardware accelerator array (e.g., DMA circuitry in interface tiles) which fetch the DMA operations and program themselves, as well as other DMA circuitry in the array.

Abstract: An integrated circuit (IC) can include a data processing array including a plurality of compute tiles arranged in a grid. The IC can include an array interface coupled to the data processing array. The array interface includes a plurality of interface tiles. Each interface tile includes a plurality of direct memory access circuits. The IC can include a network-on-chip (NoC) coupled to the array interface. Each direct memory access circuit is communicatively linked to the NoC via an independent communication channel.

Abstract: An integrated circuit (IC) can include a data processing array including a plurality of compute tiles arranged in a grid. The IC can include an array interface coupled to the data processing array. The array interface includes a plurality of interface tiles. Each interface tile includes a plurality of direct memory access circuits. The IC can include a network-on-chip (NoC) coupled to the array interface. Each direct memory access circuit is communicatively linked to the NoC via an independent communication channel.

Abstract: Examples herein describe techniques for reducing the amount of memory used during weight sparsity. When decompressing the weights, the uncompressed weight data typically has many zero values. By knowing the location of these zero values (e.g., their indices in a weight matrix), the processor core can prune some of the activations (e.g., logically reduce the size of the activation matrix) which improves the efficiency of the processor core. In embodiments herein, the processor core includes logic for identifying the indices of the non-zero value after decompressing the compressed weights. These indices can then be used to prune the activations to improve the efficiency of the processor core.