Abstract: An example apparatus is to clip a value of an activation associated with a layer of a floating-point version of a machine learning model to determine a clipped value of the activation, the value of the activation based on calibration data applied to the floating-point version of the machine learning model. The example apparatus is also to determine, using the clipped value of the activation, a quantization factor to quantize activations associated with a corresponding layer of a fixed-point version of the machine learning model. The example apparatus is further to configure the fixed-point version of the machine learning model on a device using the quantization factor.

Abstract: Various embodiments of the present disclosure relate to performing layer normalization within the context of neural networks, and in particular, to optimizing the operations required to perform layer normalization. In one example embodiment a technique for performing layer normalization is provided. The technique first includes generating a first input matrix and a second input matrix using a plurality of values stored by a feature vector. Next, the technique includes matrix multiplying the first input matrix with the second input matrix to generate an output matrix, such that the output matrix stores a plurality of result values. Finally, the technique includes performing layer normalization for the feature vector using the plurality of result values stored by the output matrix.

Abstract: Various embodiments of the present disclosure relate to optimizing the execution of a transformer network, and in particular, to optimizing the execution of non-linear operations within the transformer network. In one example embodiment, a technique for executing a transformer network within the context of an encoder is provided. The technique first includes generating embedding data based on sensor data, and generating key data, query data, and value data based on the embedding data. Next the technique includes producing a first result by performing a first matrix multiplication operation with respect to the key data and transpose-read query data. Next, the technique includes performing a SoftMax operation on the first result to produce a second result, and transpose-writing the second result to memory. Finally, the technique includes producing a third result by performing a second matrix multiplication operation with respect to the value data and transpose-written second result.

Abstract: Various embodiments of the present disclosure relate to pruning the data of a neural network, and in particular, to removing the unnecessary weights from the various channels of the neural network. In one example embodiment, a technique for pruning the weights of a neural network is provided. The technique first includes identifying weights to prune from a channel of the neural network based on a sparsity target and a weight threshold. Once identified, the technique includes determining a pruning factor for pruning the identified weights based on a current training epoch, an initial training epoch, a final training epoch, and a desired pruning pace. Next, the technique includes, over multiple training epochs, reducing each of the identified weights by multiplying the identified weights by the pruning factor. Finally, the technique includes removing the identified weights from the channel that have been reduced to below a threshold value.

Abstract: Disclosed herein are systems and methods for performing post training quantization. A processor obtains fixed-point output values from a layer of an artificial neural network (ANN) wherein the layer includes fixed-point weights determined based on floating-point weights and a weight scaling factor determined based on an output scaling factor. Next, the processor converts the fixed-point output values to floating-point output values based on the output scaling factor. Then, the processor expands a range of floating-point values. Next, the processor calculates a new output scaling factor based on the expanded range of floating-point output values. Finally, the processor stores the new output scaling factor in an associated memory.

Abstract: Disclosed herein are systems and methods for providing on-the-fly padding to feature maps of convolutional neural networks (CNNs). In an implementation, a processor first identifies a padding schema for a feature map based on a type of convolution to be performed on the feature map. Next the processor identifies a feature vector from the feature map currently in an associated memory. Then, the processor determines a padding for the feature vector based on the padding schema. Finally, the processor applies the padding to the feature vector while the feature vector is transferred from the associated memory to registers of the suitable computer.

Abstract: Disclosed herein are improved systems and methods for accelerated 2D dilated convolution. A processor determines an offset based on a dilation factor of the 2D dilated convolution. The processor selects rows of data from the 2D input in phases based on the offset and loads an input feature panel without overwriting data that has not yet been consumed by the 2D dilated convolution processor. As the 2D dilated convolution processor performs the convolution iterations, the processor continues to load additional data for the convolution. As the convolution iterations are completed, the processor spaces result of the 2D dilated convolution into a matrix such that results of each phase are spaced based on the offset.