Abstract: Fine-grained preemption of a data flow architecture based neural processing unit (NPU) includes executing, by a controller, control-code that implements a first context in the NPU. In response to the controller detecting a preemption opcode in the control-code, detecting, by the controller, a second context awaiting execution by the neural processing unit. The second context has a priority that is greater than a priority of the first context. In response to detecting the second context, the NPU switches from executing the first context to implementing the second context.

Abstract: Command stream stitching for hardware acceleration includes generating, by a host processor, a stitched block representing a plurality of commands for a hardware accelerator. The host processor generates a stitched command from the plurality of commands. The stitched command references the stitched block. The hardware accelerator executes the stitched block in response to invoking the stitched command. The hardware accelerator generates a single notification directed to the host processor for the stitched command.

Abstract: Examples herein describe an array of controllers. The array includes a first controller having a first memory and a first processor and a second controller having a second memory and a second processor. The first controller is configured to execute a first segment of control code. The control code is compiled based on one or more instructions included in a user defined application. The second controller is configured to execute a second segment of the control code. The one or more instructions included in the user defined application are executable by executing the first segment of the control code and the second segment of the control code.

Abstract: Embodiments herein describe partitioning an acceleration device based on the needs of each user application executing in a host. In one embodiment, a flexible queue provisioning method allows the acceleration device to be dynamically partitioned by pushing the configuration through a control command queue to the device by management software running in a trusted zone. The new configuration is parsed and verified by trusted firmware, which, then, creates isolated IO command queues on the acceleration device. These IO command queues can be directly mapped to a user application, VM, or other PCIe devices. In one embodiment, each IO command queue exposes only the compute resource assigned by the trusted firmware in the acceleration device.

Abstract: An integrated circuit can include a communication endpoint configured to maintain a communication link with a host computer, a queue configured to receive a plurality of host commands from the host computer via the communication link, and a processor configured to execute a device runtime. The processor, responsive to executing the device runtime, is configured to perform validation of the host commands read from the queue and selectively execute the host commands based on a result of the validation on a per host command basis. The host commands are executable by the processor to manage functions of the integrated circuit. The queue is implemented in a region of memory that is shared by the integrated circuit and the host computer.

Abstract: Embodiments herein describe partitioning an acceleration device based on the needs of each user application executing in a host. In one embodiment, a flexible queue provisioning method allows the acceleration device to be dynamically partitioned by pushing the configuration through a control command queue to the device by management software running in a trusted zone. The new configuration is parsed and verified by trusted firmware, which, then, creates isolated IO command queues on the acceleration device. These IO command queues can be directly mapped to a user application, VM, or other PCIe devices. In one embodiment, each IO command queue exposes only the compute resource assigned by the trusted firmware in the acceleration device.

Abstract: An integrated circuit can include a communication endpoint configured to maintain a communication link with a host computer, a queue configured to receive a plurality of host commands from the host computer via the communication link, and a processor configured to execute a device runtime. The processor, responsive to executing the device runtime, is configured to perform validation of the host commands read from the queue and selectively execute the host commands based on a result of the validation on a per host command basis. The host commands are executable by the processor to manage functions of the integrated circuit. The queue is implemented in a region of memory that is shared by the integrated circuit and the host computer.

Abstract: A hardware acceleration device can include a switch communicatively linked to a host central processing unit (CPU), an adapter coupled to the switch via a control bus, wherein the control bus is configured to convey addresses of descriptors from the host central CPU to the adapter, and a random-access memory (RAM) coupled to the switch through a data bus. The RAM is configured to store descriptors received from the host CPU via the data bus. The hardware acceleration device can include a compute unit coupled to the adapter and configured to perform operations specified by the descriptors. The adapter may be configured to retrieve the descriptors from the RAM via the data bus, provide arguments from the descriptors to the compute unit, and provide control signals to the compute unit to initiate the operations using the arguments.

Abstract: A hardware acceleration device can include a switch communicatively linked to a host central processing unit (CPU), an adapter coupled to the switch via a control bus, wherein the control bus is configured to convey addresses of descriptors from the host central CPU to the adapter, and a random-access memory (RAM) coupled to the switch through a data bus. The RAM is configured to store descriptors received from the host CPU via the data bus. The hardware acceleration device can include a compute unit coupled to the adapter and configured to perform operations specified by the descriptors. The adapter may be configured to retrieve the descriptors from the RAM via the data bus, provide arguments from the descriptors to the compute unit, and provide control signals to the compute unit to initiate the operations using the arguments.

Abstract: Examples herein describe techniques for launching and executing a pipeline formed by heterogeneous processing units. A system on a chip (SoC) can include different hardware elements which form a collection of heterogeneous processing units, such as general purpose processor, programmable logic array, and specialized processors. These processing units are heterogeneous meaning their underlying hardware and techniques for processing data are different, in contrast to a system that using homogeneous processing units. In the embodiments herein, the heterogeneous processing units can be arranged into a pipeline where each stage of the pipeline is performed by one of the processing units.

Abstract: Supporting multiple clients on a single programmable integrated circuit (IC) can include implementing a first image within the programmable IC in response to a first request for processing to be performed by the programmable IC, wherein the request is from a first process executing in a host data processing system coupled to the programmable IC, receiving, using a processor of the host data processing system, a second request for processing to be performed on the programmable IC from a second and different process executing in the host data processing system while the programmable IC still implements the first image, comparing, using the processor, a second image specified by the second request to the first image, and, in response to determining that the second image matches the first image based on the comparing, granting, using the processor, the second request for processing to be performed by the programmable IC.