Abstract: Aspects of the invention include performing a stochastic update for a crossbar array by generating a set of stochastic pulses for a crossbar array, the crossbar array including a plurality of row wires and a plurality of column wires, the plurality of row wires including a first row wire and the plurality of column wires including a first column wire, wherein a three terminal device is coupled to the first row wire and the first column wire at a crosspoint of the first row wire and the first column wire, and wherein a resistivity of the three terminal device is modified responsive to a coincidence of pulses from the set of stochastic pulses at the crosspoint of the first row and the first column, and wherein at least one terminal in the three terminal device is floating.

Abstract: Aspects of the invention include a first matrix resistive processing unit (“RPU”) array receives a first input vector along the rows of the first matrix RPU. A second matrix RPU array receives a second input vector along the rows of the second matrix RPU. A reference matrix RPU array receives an inverse of the first input vector along the rows of the reference matrix RPU and an inverse of the second input vector along the rows of the reference matrix RPU. A plurality of analog to digital converters are coupled to respective outputs of a plurality of summing junctions that receive respective column outputs of the first matrix RPU array, the second matrix RPU array, and the reference RPU array and provides a digital value of the output of the plurality of summing junctions.

Abstract: Embodiments of the present invention include a crossbar array that includes a resistive processing unit (RPU) device at each crosspoint in the crossbar array. The RPU device includes a single weight storage element, and multiple weight reader elements. A first weight reader element is coupled with a first row wire to compute a first matrix product value using a first value and a stored value, the first value being transmitted via the first row wire and the stored value being stored in the single weight storage element. A second weight reader element is coupled with a second row wire to compute a second matrix product value of a second value and said stored value, the second value being transmitted via the second row wire.

Abstract: A computer-implemented method and computer processing system are provided for update management for a neural network. The method includes performing an isotropic update process on the neural network using a Resistive Processing Unit. The isotropic update process uses a multiplicand and a multiplier from a multiplication operation. The performing step includes scaling the multiplicand and the multiplier to have a same order of magnitude.

Abstract: A processing unit, including a first circuit, and a first circuit element connected to the first circuit. The first circuit element is at least charged by the first circuit.

Abstract: Embodiments are directed to forming and training a resistive processing unit (RPU) system. The RPU system is formed from a plurality of RPU tiles, whereby the RPU tiles are the atomic building block of the RPU system. The plurality of RPU tiles is configured as a plurality of RPU chips. The plurality of RPU compute nodes is formed from the plurality of RPU chips. The plurality of RPU compute nodes can further be connected by a low latency, high speed network. The RPU system is trained for an artificial neural network model using the atomic matrix operations of a forward cycle, backward cycle, and matrix update.

Abstract: A processor includes an array of resistive processing units connected between row and column lines with a resistive element. A first single instruction, multiple data processing unit (SIMD) is connected to the row lines. A second SIMD is connected to the column lines. A first instruction issuer is connected to the first SIMD to issue instructions to the first SIMD, and a second instruction issuer is connected to the second SIMD to issue instructions to the second SIMD such that the processor is programmable and configurable for specific operations depending on an issued instruction set.

Abstract: Machine learning is enhanced by efficiently updating a weight that is represented as a conductivity of a resistive processing unit (RPU) that is connected between a row wire and a column wire. The weight is updated by the RPU interacting with bit streams carried on the row and column wires. Efficiency of the update is enhanced by calculating a bit length for the bit streams as a function of factors that include learning rate ?, maximum activity xmax, maximum error differential ?max, and minimum weight update increment ?wmin.

Abstract: Technical solutions are described for improving efficiency of training a resistive processing unit (RPU) array using a neural network training methodology. An example method includes reducing asymmetric RPUs from the RPU array by determining an asymmetric value of an RPU from the RPU array, and burning the RPU in response to the asymmetry value being above a predetermined threshold. The RPU can be burned by causing an electric voltage across the RPU to be above a predetermined limit. The method further includes initiating the training methodology for the RPU array after the asymmetric RPUs from the RPU array are reduced.

Abstract: A processing unit, including a first circuit part, and a capacitor connected to the first circuit part. The capacitor is charged or discharged by the first circuit part.

Abstract: A processor includes an array of resistive processing units connected between row and column lines with a resistive element. A first single instruction, multiple data processing unit (SIMD) is connected to the row lines. A second SIMD is connected to the column lines. A first instruction issuer is connected to the first SIMD to issue instructions to the first SIMD, and a second instruction issuer is connected to the second SIMD to issue instructions to the second SIMD such that the processor is programmable and configurable for specific operations depending on an issued instruction set.

Abstract: Cross-point arrays and methods of updating values of the same include input resistive processing units (RPUs), each having a settable resistance, each connected to a common node. Output RPUs each have a settable resistance and are each connected to the common node. An update switch is configured to connect an update voltage to the common node.

Abstract: Methods and systems for copying weight values between weight arrays includes reading outputs from a first array and reading outputs from a second array. Differences between respective outputs of the first array and the second array are determined. Values of the second array are adjusted in accordance with the determined differences.

Abstract: An apparatus and method are provided for saturation prevention of a current integrator in a Resistive Processing Unit-based (RPU-based) accelerator. The apparatus includes a set of hardware switches. The apparatus further includes a voltage generator, operatively coupled between an input terminal and an output terminal of the current integrator, reducing a magnitude of an output voltage at the output terminal of the current integrator during a current integration operation by selectively applying a non-zero initial voltage to the current integrator prior to the current integration operation, responsive to an operating state of the set of hardware switches.

Abstract: Techniques for auto weight scaling a bounded weight range of RPU devices with the size of the array during ANN training are provided. In one aspect, a method of ANN training includes: initializing weight values winit in the array to a random value, wherein the array represents a weight matrix W with m rows and n columns; calculating a scaling factor ? based on a size of the weight matrix W; providing digital inputs x to the array; dividing the digital inputs x by a noise and bound management factor ? to obtain adjusted digital inputs x?; performing a matrix-vector multiplication of the adjusted digital inputs x? with the array to obtain digital outputs y?; multiplying the digital outputs y? by the noise and bound management factor ?; and multiplying the digital outputs y? by the scaling factor ? to provide digital outputs y of the array.

Abstract: Advanced noise and signal management techniques for RPU arrays during ANN training are provided. In one aspect of the invention, a method for ANN training includes: providing an array of RPU devices with pre-normalizers and post-normalizers; computing and pre-normalizing a mean and standard deviation of all elements of an input vector x to the array that belong to the set group of each of the pre-normalizers; and computing and post-normalizing the mean p and the standard deviation a of all elements of an output vector y that belong to the set group of each of the post-normalizers.

Abstract: A cross-point array and method for forming the same are provided. The cross-point array includes an array of Resistive Processing Unit (RPU) devices having rows and columns interconnected at cross-points. The cross-point array further includes a plurality of series resistors. Each respective one of the plurality of series resistors has a first end connected in series with a respective one of the RPU devices and a second end connected to a respective one of the cross-points to compensate for a parasitic voltage drop associated with each of the RPU devices.

Abstract: A neural network system comprises a plurality of neurons, comprising a layer of input neurons, one or more layers of hidden neurons, and a layer of output neurons. The system further comprises a plurality of arrays of weights, each array of weights being configured to receive a plurality of discrete data points from a first layer of neurons and to produce a corresponding discrete data point to a second layer of neurons during a feed forward operation, each array of weights comprising a plurality of resistive processing units (RPU) having respective settable resistances. The system includes a neuron control system configured to control an operation mode of each of the plurality of neurons, wherein the operation mode comprises: a feed forward mode, a back propagation mode, and a weight update mode.

Abstract: Zero-shifting techniques in analog crosspoint arrays are provided. In one aspect, an analog array-based vector-matrix multiplication includes: a weight array connected to a reference array, each including a crossbar array having a set of conductive row wires and a set of conductive column wires intersecting the set of conductive row wires, and optimizable crosspoint devices at intersections of the set of conductive column wires and the set of conductive row wires. A method for analog array-based vector-matrix computing is also provided that includes: applying repeated voltage pulses to the crosspoint devices in the weight array until all of the crosspoint devices in the weight array converge to their own symmetry point; and copying conductance values for each crosspoint device from the weight array to the reference array.

Abstract: A system includes an analog memory architecture for performing differential reading. The analog memory architecture includes a weight array including first cross-point devices located at intersections of a first set of conductive column wires and a first set of conductive row wires, and a reference array operatively coupled to the weight array and including second cross-point devices located at intersections of a second set of conductive column wires and a second set of conductive row wires. The second cross-point devices include differential unipolar switching memory devices configured to enable zero-value shifting of the outputs of the first cross-point devices.