Abstract: A probability distribution corresponding to the kernel function is determined and weights are sampled from the determined probability distribution corresponding to the given kernel function. Memristive devices of an analog crossbar are programmed based on the sampled weights, where each memristive device of the analog crossbar is configured to represent a corresponding weight. Two matrix-vector multiplication operations are performed on an analog input x and an analog input y using the programmed crossbar and a dot product is computed on results of the matrix-vector multiplication operations.

Abstract: A 3D compute-in-memory accelerator system and method for efficient inference of Mixture of Expert (MoE) neural network models. The system includes a plurality of compute-in-memory cores, each in-memory core including multiple tiers of in-memory compute cells. One or more tiers of in-memory compute cells correspond to an expert sub-model of the MoE model. One or more expert sub-models are selected for activation propagation based on a function-based routing, the tiers of the corresponding experts being activated based on this function. In one embodiment, this function is a hash-based tier selection function used for dynamic routing of inputs and output activations. In embodiments, the function is applied to select a single expert or multiple experts with input data-based or with layer-activation-based MoEs for single tier activation. Further, the system is configured as a multi-model system with single expert model selection or with a multi-model system with multi-expert selection.

Abstract: A method of operating a neuromorphic system is provided. The method includes applying voltage signals across input lines of a crossbar array structure, the crossbar array structure including rows and columns interconnected at junctions via programmable electronic devices, the rows including the input lines for applying voltage signals across the electronic devices and the columns including output lines for outputting currents. The method also includes correcting, via a correction unit connected to the output lines, each of the output currents obtained at the output lines according to an affine transformation to compensate for temporal conductance variations in the electronic devices.

Abstract: The present disclosure relates to an electronic system for computing items of an outer product matrix, for each item of at least part of the items of the matrix. The system is configured to receive a pair of real numbers of two vectors, the pair corresponding to the item. The system is further configured to compute a stochastic representation of the real numbers resulting in two sets of bits, the set of bits comprising a subset of bits representing the real number and a sign bit indicative of the sign of the real number. The system is further configured to perform a sequence of digital operations using the two sets of bits to provide a representation of the item.

Abstract: Methods and apparatus are provided for training an artificial neural network, having a succession of neuron layers with interposed synaptic layers each storing a respective set of weights {w} for weighting signals propagated between its adjacent neuron layers, via an iterative cycle of signal propagation and weight-update calculation operations. Such a method includes, for at least one of the synaptic layers, providing a plurality Pl of arrays of memristive devices, each array storing the set of weights of that synaptic layer Sl in respective memristive devices, and, in a signal propagation operation, supplying respective subsets of the signals to be weighted by the synaptic layer Sl in parallel to the Pl arrays. The method also includes, in a weight-update calculation operation, calculating updates to respective weights stored in each of the Pl arrays in dependence on signals propagated by the neuron layers.

Abstract: Methods and apparatus are provided for training an artificial neural network having a succession of layers of neurons interposed with layers of synapses. A set of crossbar arrays of memristive devices, connected between row and column lines, implements the layers of synapses. Each memristive device stores a weight for a synapse interconnecting a respective pair of neurons in successive neuron layers. The training method includes performing forward propagation, backpropagation and weight-update operations of an iterative training scheme by applying input signals, associated with respective neurons, to row or column lines of the set of arrays to obtain output signals on the other of the row or column lines, and storing digital signal values corresponding to the input and output signals. The weight-update operation is performed by calculating digital weight-correction values for respective memristive devices, and applying programming signals to those devices to update the stored weights.

Abstract: A computer-implemented method of mixed-precision deep learning with multi-memristive synapses may be provided. The method comprises representing, each synapse of an artificial neural network by a combination of a plurality of memristive devices, wherein each of the plurality of memristive devices of each of the synapses contributes to an overall synaptic weight with a related device significance, accumulating a weight gradient ?W for each synapse in a high-precision variable, and performing a weight update to one of the synapses using an arbitration scheme for selecting a respective memristive device, according to which a threshold value related to the high-precision variable for performing the weight update is set according to the device significance of the respective memristive device selected by the arbitration schema.

Abstract: A method of operating a neuromorphic system is provided. The method includes applying voltage signals across input lines of a crossbar array structure, the crossbar array structure including rows and columns interconnected at junctions via programmable electronic devices, the rows including the input lines for applying voltage signals across the electronic devices and the columns including output lines for outputting currents. The method also includes correcting, via a correction unit connected to the output lines, each of the output currents obtained at the output lines according to an affine transformation to compensate for temporal conductance variations in the electronic devices.

Abstract: A method and system providing a multi-memristive synaptic element for a cognitive computing system. The multi-memristive synaptic element comprises an array of memristive devices. The method comprises arbitrating a synaptic weight allocation, a related synaptic weight being represented by a synaptic weight variable of said multi-memristive synaptic element, updating said synaptic weight variable by a delta amount, and assigning said memristive devices to elements of a clock-like ordered circular list for selecting a particular memristor of said memristive devices requiring to be updated by a deterministic, periodic global clock that points to a different memristor at every clock tick, such that said multi-memristive synaptic element has a larger dynamic range and a more linear conductance response than a single memristor synaptic element.

Abstract: The present disclosure relates to an electronic system for computing items of an outer product matrix, for each item of at least part of the items of the matrix. The system is configured to receive a pair of real numbers of two vectors, the pair corresponding to said item. The system is further configured to compute a stochastic representation of the real numbers resulting in two sets of bits, the set of bits comprising a subset of bits representing the real number and a sign bit indicative of the sign of the real number. The system is further configured to perform a sequence of digital operations using the two sets of bits to provide a representation of said item.

Abstract: Methods and apparatus are provided for training an artificial neural network, having a succession of neuron layers with interposed synaptic layers each storing a respective set of weights {w} for weighting signals propagated between its adjacent neuron layers, via an iterative cycle of signal propagation and weight-update calculation operations. Such a method includes, for at least one of the synaptic layers, providing a plurality P1 of arrays of memristive devices, each array storing the set of weights of that synaptic layer S1 in respective memristive devices, and, in a signal propagation operation, supplying respective subsets of the signals to be weighted by the synaptic layer S1 in parallel to the P1 arrays. The method also includes, in a weight-update calculation operation, calculating updates to respective weights stored in each of the P1 arrays in dependence on signals propagated by the neuron layers.

Abstract: A computer-implemented method of mixed-precision deep learning with multi-memristive synapses may be provided. The method comprises representing, each synapse of an artificial neural network by a combination of a plurality of memristive devices, wherein each of the plurality of memristive devices of each of the synapses contributes to an overall synaptic weight with a related device significance, accumulating a weight gradient ?W for each synapse in a high-precision variable, and performing a weight update to one of the synapses using an arbitration scheme for selecting a respective memristive device, according to which a threshold value related to the high-precision variable for performing the weight update is set according to the device significance of the respective memristive device selected by the arbitration schema.

Abstract: Artificial neural networks (ANNs) are a distributed computing model in which computation is accomplished using many simple processing units (called neurons) and the data embodied by the connections between neurons (called synapses) and the strength of these connections (called synaptic weights). An attractive implementation of ANNs uses the conductance of non-volatile memory (NVM) elements to code the synaptic weight. In this application, the non-idealities in the response of the NVM (such as nonlinearity, saturation, stochasticity and asymmetry in response to programming pulses) lead to reduced network performance compared to an ideal network implementation. Disclosed is a method that improves performance by implementing a learning rate parameter that is local to each synaptic connection, a method for tuning this local learning rate, and an implementation that does not compromise the ability to train many synaptic weights in parallel during learning.

Abstract: Methods and apparatus are provided for training an artificial neural network having a succession of layers of neurons interposed with layers of synapses. A set of crossbar arrays of memristive devices, connected between row and column lines, implements the layers of synapses. Each memristive device stores a weight for a synapse interconnecting a respective pair of neurons in successive neuron layers. The training method includes performing forward propagation, backpropagation and weight-update operations of an iterative training scheme by applying input signals, associated with respective neurons, to row or column lines of the set of arrays to obtain output signals on the other of the row or column lines, and storing digital signal values corresponding to the input and output signals. The weight-update operation is performed by calculating digital weight-correction values for respective memristive devices, and applying programming signals to those devices to update the stored weights.

Abstract: A method and system providing a multi-memristive synaptic element for a cognitive computing system. The multi-memristive synaptic element comprises an array of memristive devices. The method comprises arbitrating a synaptic weight allocation, a related synaptic weight being represented by a synaptic weight variable of said multi-memristive synaptic element, updating said synaptic weight variable by a delta amount, and assigning said memristive devices to elements of a clock-like ordered circular list for selecting a particular memristor of said memristive devices requiring to be updated by a deterministic, periodic global clock that points to a different memristor at every clock tick, such that said multi-memristive synaptic element has a larger dynamic range and a more linear conductance response than a single memristor synaptic element.

Abstract: A system, method and computer program product for achieving a collective task. The system comprises a plurality of elements representative of a first hierarchy level, each element comprises a plurality of sub-elements. The system comprises also an arbitration module for selecting one of the sub-elements of each element at a point in time based on a global clock, wherein each sub-element relates to one list element of an ordered circular list, and a combination module adapted for a combination of sub-actions performed by a portion of the sub-elements of one of the elements over a predefined period of time, wherein each sub-element performs one of the sub-actions.

Abstract: A method and system providing a multi-memristive synaptic element for a cognitive computing system. The multi-memristive synaptic element comprises an array of memristive devices. The method comprises arbitrating a synaptic weight allocation, a related synaptic weight being represented by a synaptic weight variable of said multi-memristive synaptic element, updating said synaptic weight variable by a delta amount, and assigning said memristive devices to elements of a clock-like ordered circular list for selecting a particular memristor of said memristive devices requiring to be updated by a deterministic, periodic global clock that points to a different memristor at every clock tick, such that said multi-memristive synaptic element has a larger dynamic range and a more linear conductance response than a single memristor synaptic element.