Abstract: A method for mixed precision quantization of an artificial intelligence (AI) model by an electronic device is included. The method includes performing, by the electronic device, perturbation in weights of each layer of a plurality of layers of the AI model for a pre-defined number of times, determining, by the electronic device, a change in an output of each layer of a plurality of layers of the AI model based on a perturbation in weights of each layer of the plurality of layers, determining, by the electronic device, a sensitivity metric for each layer of the plurality of layers of the AI model as a measure of the change in the output of each layer, assigning, by the electronic device, a bit-precision to each layer of the plurality of layers of the AI model based on the determined sensitivity metric, and performing, by the electronic device, the mixed precision quantization of the AI model using the bit-precision assigned to each layer of the plurality of layers of the AI model.

Abstract: The disclosed embodiments provide a system for managing a counting use case. During operation, the system matches, to a first counting use case, a first parameter of a first unified request over an application programming interface (API) provided by a unified counting platform. Next, the system identifies, based on metadata for configuring the first counting use case in the unified counting platform, a first counting solution assigned to the first counting use case. The system then formats a first set of parameters in the first unified request into a first adapted request that is transmitted to the first counting solution. The system also formats a first response to the first adapted request from the first counting solution into a first unified response to the first unified request. Finally, the system transmits the first unified response to a first source of the first unified request.

Abstract: A method of performing a reshape operation specified in a reshape layer of a neural network model is described. The reshape operation reshapes an input tensor with an input tensor shape to an output tensor with an output tensor shape. The tensor data that has to be reshaped is directly routed between tile memories of the hardware accelerator in an efficient manner. This advantageously optimizes usage of memory space and allows any number and type of neural network models to be run on the hardware accelerator.

Abstract: A compiler of a computing device is described that identifies a sequence of neural network models frequently invoked by an application of the computing device, compiles the models in that sequence, and loads a static random access memory (SRAM) of a hardware accelerator with the compiled models only when the same compiled models—from another, but same, sequence that was previously invoked—are not already present in the SRAM. This prevents unnecessary reloading of compiled models into the SRAM, thereby increasing runtime speed and conserving computational energy.

Abstract: The disclosed embodiments provide a system for managing a counting use case. During operation, the system matches, to a first counting use case, a first parameter of a first unified request over an application programming interface (API) provided by a unified counting platform. Next, the system identifies, based on metadata for configuring the first counting use case in the unified counting platform, a first counting solution assigned to the first counting use case. The system then formats a first set of parameters in the first unified request into a first adapted request that is transmitted to the first counting solution. The system also formats a first response to the first adapted request from the first counting solution into a first unified response to the first unified request. Finally, the system transmits the first unified response to a first source of the first unified request.