Abstract: The disclosed embodiments provide a system for managing a counting use case. During operation, the system matches, to a first counting use case, a first parameter of a first unified request over an application programming interface (API) provided by a unified counting platform. Next, the system identifies, based on metadata for configuring the first counting use case in the unified counting platform, a first counting solution assigned to the first counting use case. The system then formats a first set of parameters in the first unified request into a first adapted request that is transmitted to the first counting solution. The system also formats a first response to the first adapted request from the first counting solution into a first unified response to the first unified request. Finally, the system transmits the first unified response to a first source of the first unified request.

Abstract: A method of performing a reshape operation specified in a reshape layer of a neural network model is described. The reshape operation reshapes an input tensor with an input tensor shape to an output tensor with an output tensor shape. The tensor data that has to be reshaped is directly routed between tile memories of the hardware accelerator in an efficient manner. This advantageously optimizes usage of memory space and allows any number and type of neural network models to be run on the hardware accelerator.

Abstract: A compiler of a computing device is described that identifies a sequence of neural network models frequently invoked by an application of the computing device, compiles the models in that sequence, and loads a static random access memory (SRAM) of a hardware accelerator with the compiled models only when the same compiled models—from another, but same, sequence that was previously invoked—are not already present in the SRAM. This prevents unnecessary reloading of compiled models into the SRAM, thereby increasing runtime speed and conserving computational energy.

Abstract: The disclosed embodiments provide a system for managing a counting use case. During operation, the system matches, to a first counting use case, a first parameter of a first unified request over an application programming interface (API) provided by a unified counting platform. Next, the system identifies, based on metadata for configuring the first counting use case in the unified counting platform, a first counting solution assigned to the first counting use case. The system then formats a first set of parameters in the first unified request into a first adapted request that is transmitted to the first counting solution. The system also formats a first response to the first adapted request from the first counting solution into a first unified response to the first unified request. Finally, the system transmits the first unified response to a first source of the first unified request.