Abstract: Systems and methods for modifying a structure of an artificial neural network (ANN) are provided. An example method comprises receiving, by one or more processing units, a plurality of arrays of weights associated with the ANN, modifying, by the processing units, the plurality of arrays of weights to generate a further plurality of further arrays of weights, where after the modification the following conditions are satisfied: an amount of operations required for computing neurons of the ANN using the further plurality of further arrays of weights is less than an amount of operations required for computing same neurons of the ANN using the plurality of arrays of weights; and outputs of the neurons of the ANN computed using the plurality of arrays of weights are substantially equal to further outputs of the neurons of the ANN using the further plurality of further arrays of weights.

Abstract: An example method for accelerating neuron computations in an artificial neural network (ANN) comprises receiving a plurality of pairs of first values and second values associated with a neuron of an ANN, selecting pairs from the plurality of pairs, wherein a count of the selected pairs is less than a count of all pairs in the plurality of pairs, performing mathematical operations on the selected pairs to obtain a result, determining that the result does not satisfy a criterion, and, until the result satisfies the criterion, selecting further pairs from the plurality, performing the mathematical operations on the selected further pairs to obtain further results, and determining, based on the result and the further results, an output of the neuron.

Abstract: Systems and methods for quality monitoring and hidden quantization in artificial neural network (ANN) computations are provided. An example method may include receiving a description of an ANN and input data associated with the ANN, performing, based on a quantization scheme, quantization of the ANN to obtain a quantized ANN, performing, based on the set of input data, ANN computations of the quantized ANN to obtain a result of the ANN computation for the input data, while performing the ANN computations, monitoring, a measure of quality of the ANN computations of the quantized ANN, determining that the measure of quality does not satisfy quality requirements, and in response to the determination, informing a user of an external system of the measure of quality, and adjusting, based on the measure of quality, the quantization scheme to be used in the ANN computations for further input data.

Abstract: Systems and methods for accelerating computation of an artificial neural network (ANN) are provided. An example method comprises receiving, by processing units coupled with arithmetic units and accumulation units, a first plurality of first values and a second plurality of second values associated with one or more neurons of the ANN, generating, by the processing units, a plurality of pairs, wherein each pair of the plurality of pairs has a first value of the first plurality and a second value of the second plurality and the first value and the second value satisfy criteria, performing, by the arithmetic units, mathematical operations on pairs of the plurality of pairs to obtain results; accumulating, by the accumulation units, the results to obtain accumulated results, and determining, by the processing units and based on the accumulated results, an output of the neurons.

Abstract: Systems and methods for optimizing operations in artificial neural network computations are disclosed. An example method may include selecting a first input value from a set of input values to a neuron, selecting, based on a criterion, a second input value from the set of input values, acquiring a first weight from a set of weights, acquiring a second weight from a set of weights, performing, in parallel, a first mathematical operation on the first input value and the first weight to obtain a first result, a second mathematical operation based on the second input value and the second weight to obtain a second result, the second mathematical operation requiring less number of bits than the first mathematical operation, the second number of bits being less than the first number of bits, and computing an output of the neuron based on the first result and the second result.

Abstract: Systems and methods for modifying structure of an artificial neural network (ANN) are provided. An example method comprises receiving, by one or more processing units, a plurality of arrays of weights associated with the ANN, modifying, by the processing units, the plurality of arrays of weights to generate a further plurality of further arrays of weights, wherein after the modification the following conditions are satisfied: an amount of operations required for computing neurons of the ANN using the further plurality of further arrays of weights is less than an amount of operations required for computing same neurons of the ANN using the plurality of arrays of weights; and outputs of the neurons of the ANN computed using the plurality of arrays of weights are substantially equal to further outputs of the neurons of the ANN using the further plurality of further arrays of weights.

Abstract: Systems and methods for realigning streams of neuron outputs are provided. An example method may include generating, by a processing unit, neuron outputs including at least a first neuron output and a second neuron output, generating, by at least one further processing unit, further neuron outputs including at least a further first neuron output and a further second neuron output, receiving, by a synchronization module communicatively coupled to the processing unit and the further processing unit, the neuron outputs, wherein the neuron outputs and the further neuron outputs are received in an arbitrary order, and ordering, by the synchronization module, the first neuron output, the further first neuron output, the second neuron output and the further second neuron output according to a further order, the further order being different from the arbitrary order.

Abstract: Systems and methods for performing a quantization of artificial neural networks (ANNs) are provided. An example method may include receiving a description of an ANN and input data associated with the ANN, wherein the input data are represented according to a first data type; selecting a first value interval of the first data type to be mapped to a second value interval of a second data type; performing, based on the input data and the description of the ANN, the computations of one or more neurons of the ANN, wherein the computations are performed for at least one value within the second value interval, the value being a result of mapping a value of the first value interval to a value of the second value interval; determining, a measure of saturations in neurons of the ANN, and adjusting, based on the measure of saturations, the value intervals.

Abstract: Systems and methods for performing multiple locally stored artificial neural network (ANN) computations are provided. An example method comprises receiving, by one or more processing units, an ANN dataset associated with at least one ANN of a plurality of ANNs; storing, by processing units, the ANN dataset in a memory coupled to the processing units; associating, by the processing units, a base address with the at least one ANN, wherein the base address is to be used to locate the ANN dataset in the memory; keeping, by the processing units, the ANN dataset in the memory; receiving, by the processing units, an input dataset and the base address; determining, by the processing units and based on the base address, a location of the ANN dataset in the memory; and performing, by the processing units, ANN computation using the ANN dataset and input dataset.

Abstract: Systems and methods for accelerating computation of an artificial neural network (ANN) are provided. An example method comprises receiving, by processing units coupled with arithmetic units and accumulation units, a first plurality of first values and a second plurality of second values associated with one or more neurons of the ANN, generating, by the processing units, a plurality of pairs, wherein each pair of the plurality of pairs has a first value of the first plurality and a second value of the second plurality and the first value and the second value satisfy criteria, performing, by the arithmetic units, mathematical operations on pairs of the plurality of pairs to obtain results; accumulating, by the accumulation units, the results to obtain accumulated results, and determining, by the processing units and based on the accumulated results, an output of the neurons.

Abstract: An example method for accelerating neuron computations in an artificial neural network (ANN) comprises receiving a plurality of pairs of first values and second values associated with a neuron of an ANN, selecting pairs from the plurality of pairs, wherein a count of the selected pairs is less than a count of all pairs in the plurality of pairs, performing mathematical operations on the selected pairs to obtain a result, determining that the result does not satisfy a criterion, and, until the result satisfies the criterion, selecting further pairs from the plurality, performing the mathematical operations on the selected further pairs to obtain further results, and determining, based on the result and the further results, an output of the neuron.

Abstract: Systems and methods for accelerating neuron computations in artificial neural network (ANN) are provided. An example method may comprise receiving, for calculation of a neuron of an ANN, a plurality of first values represented by A bits and a plurality of second values represented by B bits, splitting each value of the plurality of the first values into a set of parts, a count of bits of each of set of parts being less than A, to obtain a set of pluralities of parts, selectively performing mathematical operations on a first plurality of the set of pluralities and the plurality of the second values to obtain a first result, selectively performing further mathematical operations on further pluralities of the set of pluralities and the plurality of the second values to obtain further results, and determining, based on the first result and the further results, an output of the neuron.

Abstract: Systems and methods for accelerating artificial neural network computation are disclosed. An example may comprise selecting, by a controller communicatively coupled to a selector and an arithmetic unit and based on a criterion, an input value from the stream of input values of a neuron, configuring, by the controller, the selector to provide, dynamically, the selected input value to the arithmetic unit, providing, by the controller to the arithmetic unit, an information of the selected input value, acquiring, by the arithmetic unit and based on the information, a weight from a set of weights, and performing, by the arithmetic unit a mathematical operation on the selected input value and the weight to obtain a result, wherein the result is to be used to compute an output of the neuron. The criterion may include a comparison between the input value and a reference value. The reference value may include zero.

Abstract: Systems and methods for quality monitoring and hidden quantization in artificial neural network (ANN) computations are provided. An example method may include receiving a description of an ANN and input data associated with the ANN, performing, based on a quantization scheme, quantization of the ANN to obtain a quantized ANN, performing, based on the set of input data, ANN computations of the quantized ANN to obtain a result of the ANN computation for the input data, while performing the ANN computations, monitoring, a measure of quality of the ANN computations of the quantized ANN, determining that the measure of quality does not satisfy quality requirements, and in response to the determination, informing a user of an external system of the measure of quality, and adjusting, based on the measure of quality, the quantization scheme to be used in the ANN computations for further input data.

Abstract: Systems and methods for performing a quantization of artificial neural networks (ANNs) are provided. An example method may include receiving a description of an ANN and input data associated with the ANN, wherein the input data are represented according to a first data type; selecting a first value interval of the first data type to be mapped to a second value interval of a second data type; performing, based on the input data and the description of the ANN, the computations of one or more neurons of the ANN, wherein the computations are performed for at least one value within the second value interval, the value being a result of mapping a value of the first value interval to a value of the second value interval; determining, a measure of saturations in neurons of the ANN, and adjusting, based on the measure of saturations, the value intervals.

Abstract: Systems and methods for optimizing artificial neural network (ANN) computations based on automatic determination of a batch size are disclosed. An example method may comprise receiving, by an optimization module, an ANN structure associated with the ANN, and generating, based on the ANN structure, a configuration for a computation engine capable of performing computation of the layers of the ANN. The configuration may include information concerning a batch size of one or more layers of the ANN. The batch size of a layer can be determined based on a bandwidth required to read data related to layer, a number of parameters associated with the layer, and a time the layer processes one input dataset from the batch. The batch size of the layer can differ from the batch size of the ANN. The batch size of the layer may differ from a batch size of another layer of ANN.

Abstract: Systems and methods for realigning streams of neuron outputs are provided. An example method may include generating, by a processing unit, neuron outputs including at least a first neuron output and a second neuron output, generating, by at least one further processing unit, further neuron outputs including at least a further first neuron output and a further second neuron output, receiving, by a synchronization module communicatively coupled to the processing unit and the further processing unit, the neuron outputs, wherein the neuron outputs and the further neuron outputs are received in an arbitrary order, and ordering, by the synchronization module, the first neuron output, the further first neuron output, the second neuron output and the further second neuron output according to a further order, the further order being different from the arbitrary order.

Abstract: Systems and methods for accelerating artificial neural network computation are disclosed. An example may comprise selecting, by a controller communicatively coupled to a selector and an arithmetic unit and based on a criterion, an input value from the stream of input values of a neuron, configuring, by the controller, the selector to provide, dynamically, the selected input value to the arithmetic unit, providing, by the controller to the arithmetic unit, an information of the selected input value, acquiring, by the arithmetic unit and based on the information, a weight from a set of weights, and performing, by the arithmetic unit a mathematical operation on the selected input value and the weight to obtain a result, wherein the result is to be used to compute an output of the neuron. The criterion may include a comparison between the input value and a reference value. The reference value may include zero.