Abstract: Certain aspects of the present disclosure provide techniques for performing compute in memory (CIM) computations. A device comprises a CIM module configured to apply a plurality of analog weights to data using multiply-accumulate operations to generate an output. The device further comprises a digital weight storage unit configured to store digital weight references, wherein a digital weight reference corresponds to an analog weight of the plurality of analog weights. The device also comprises a device controller configured to program the plurality of analog weights to the CIM module based on the digital weight references and determine degradation of one or more analog weights. The digital weight references in the digital weight storage unit are populated with values from a host device. Degraded analog weights in the CIM module are replaced with corresponding digital weight references from the digital weight storage unit without reference to the host device.

Abstract: Non-volatile memory structures for performing compute in memory inferencing for neural networks are presented. To improve performance, both in terms of speed and energy consumption, weight matrices are replaced with their singular value decomposition (SVD) and use of a low rank approximations (LRAs). The decomposition matrices can be stored in a single array, with the resultant LRA matrices requiring fewer weight values to be stored. The reduced sizes of the LRA matrices allow for inferencing to be performed more quickly and with less power. In a high performance and energy efficiency mode, a reduced rank for the SVD matrices stored on a memory die is determined and used to increase performance and reduce power needed for an inferencing operation.

Abstract: Techniques are presented for performing in-memory matrix multiplication operations for binary input, binary weight valued convolution neural network (CNN) inferencing. The weights of a filter are stored in pairs of memory cells of a storage class memory device, such as a ReRAM or phase change memory based devices. To reduce current consumption, the binary valued filters are transformed into ternary valued filters by taking sums and differences of binary valued filter pairs. The zero valued weights of the transformed filters are stored as a pair of high resistance state memory cells, reducing current consumption during convolution. The results of the in-memory multiplications are pair-wise combined to compensate for the filter transformations. To compensate for zero valued weights, a zero weight register stores the number of zero weights along each bit line and is used to initialize counter values for accumulating the multiplication operations.

Abstract: A non-volatile memory device is configured for in-memory computation of discrete Fourier transformations and their inverses. The real and imaginary components of the twiddle factors are stored as conductance values of memory cells in non-volatile memory arrays having a cross-point structure. The real and imaginary components of inputs are encoded as word line voltages applied to the arrays. Positive and negative valued components of the twiddle factors are stored separately and positive and negative of the inputs are separately applied to the arrays. Real and imaginary parts of the outputs for the discrete Fourier transformation are determined from combinations of the output currents from the arrays.

Abstract: Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter. The arrangement extends to ternary inputs to realize a ternary-binary network (TBN) by adding a circuit to detect 0 input values and adjust the accumulated count accordingly. The arrangement further extends to a ternary-ternary network (TTN) by allowing 0 weight values in a unit synapse, maintaining the number of 0 weights in a register, and adjusting the count accordingly.

Abstract: A non-volatile memory device includes arrays of non-volatile memory cells that are configured to the store weights for a recurrent neural network (RNN) inference engine with a gated recurrent unit (GRU) cell. A set three non-volatile memory arrays, such as formed of storage class memory, store a corresponding three sets of weights and are used to perform compute-in-memory inferencing. The hidden state of a previous iteration and an external input are applied to the weights of the first and the of second of the arrays, with the output of the first array used to generate an input to the third array, which also receives the external input. The hidden state of the current generation is generated from the outputs of the second and third arrays.

Abstract: Techniques are presented for accelerating in-memory matrix multiplication operations for a convolution neural network (CNN) inference in which the weights of a filter are stored in the memory of a storage class memory device, such as a ReRAM or phase change memory based device. To improve performance for inference operations when filters exhibit sparsity, a zero column index and a zero row index are introduced to account for columns and rows having all zero weight values. These indices can be saved in a register on the memory device and when performing a column/row oriented matrix multiplication, if the zero row/column index indicates that the column/row contains all zero weights, the access of the corresponding bit/word line is skipped as the result will be zero regardless of the input.

Abstract: Certain aspects of the present disclosure provide techniques for performing compute in memory (CIM) computations. A device comprises a CIM module configured to apply a plurality of analog weights to data using multiply-accumulate operations to generate an output. The device further comprises a digital weight storage unit configured to store digital weight references, wherein a digital weight reference corresponds to an analog weight of the plurality of analog weights. The device also comprises a device controller configured to program the plurality of analog weights to the CIM module based on the digital weight references and determine degradation of one or more analog weights. The digital weight references in the digital weight storage unit are populated with values from a host device. Degraded analog weights in the CIM module are replaced with corresponding digital weight references from the digital weight storage unit without reference to the host device.

Abstract: Certain aspects of the present disclosure provide techniques for performing compute in memory (CIM) computations. A device comprises a CIM module configured to apply analog weights to input data using multiply-accumulate operations to generate an output. The device further comprises a digital weight storage unit configured to store digital weight references, wherein a digital weight reference corresponds to an analog weight of the analog weights. The device also comprises a device controller configured to program the analog weights to the CIM module, cause the CIM module to process the input data, and reprogram one or more analog weights that are degraded. The digital weight references in the digital weight storage unit are populated with values from a host processing device. Degraded analog weights in the CIM module are reprogrammed based on the corresponding digital weight references from the digital weight storage unit without reference to the host processing device.

Abstract: A non-volatile memory device is configured for in-memory computation of layers of a neural network by storing weight values as conductance values in memory cells formed of a series combination of a threshold switching selector, such as an ovonic threshold switch, and a programmable resistive element, such as a ReRAM element. By scaling the input voltages (representing inputs for the layer of the neural network) relative to the threshold values of the threshold switching selectors, dropout for inputs can be implemented to reduce overfitting by the neural network.

Abstract: To improve efficiencies for inferencing operations of neural networks, ensemble neural networks are used for compute-in-memory inferencing. In an ensemble neural network, the layers of a neural network are replaced by an ensemble of multiple smaller neural network generated from subsets of the same training data as would be used for the layers of the full neural network. Although the individual smaller network layers are “weak classifiers” that will be less accurate than the full neural network, by combining their outputs, such as in majority voting or averaging, the ensembles can have accuracies approaching that of the full neural network. Ensemble neural networks for compute-in-memory operations can have their efficiency further improved by implementations based binary memory cells, such as by binary neural networks using binary valued MRAM memory cells. The size of an ensemble can be increased or decreased to optimize the system according to error requirements.

Abstract: Non-volatile memory structures are presented for a content addressable memory (CAM) that can perform in-memory search operations for both ternary and binary valued key values. Each ternary or binary valued key bit is stored in a pair of memory cells along a bit line of a NAND memory array, with the stored keys searched by applying each ternary or binary valued bit of an input key as voltage levels on a pair of word lines. The system is highly scalable. The system can also be used to perform nearest neighbor searches between stored vectors and an input vector to find stored vectors withing a specified Hamming distance of the input vector.

Abstract: A non-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Abstract: A non-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Abstract: Non-volatile memory structures for performing compute in memory inferencing for neural networks are presented. To improve performance, both in terms of speed and energy consumption, weight matrices are replaced with their singular value decomposition (SVD) and use of a low rank approximations (LRAs). The decomposition matrices can be stored in a single array, with the resultant LRA matrices requiring fewer weight values to be stored. The reduced sizes of the LRA matrices allow for inferencing to be performed more quickly and with less power. In a high performance and energy efficiency mode, a reduced rank for the SVD matrices stored on a memory die is determined and used to increase performance and reduce power needed for an inferencing operation.

Abstract: A non-volatile memory device includes arrays of non-volatile memory cells that are configured to the store weights for a recurrent neural network (RNN) inference engine with a gated recurrent unit (GRU) cell. A set three non-volatile memory arrays, such as formed of storage class memory, store a corresponding three sets of weights and are used to perform compute-in-memory inferencing. The hidden state of a previous iteration and an external input are applied to the weights of the first and the of second of the arrays, with the output of the first array used to generate an input to the third array, which also receives the external input. The hidden state of the current generation is generated from the outputs of the second and third arrays.

Abstract: A device includes arrays of Non-Volatile Memory (NVM) cells. Reference sequences representing portions of a genome are stored in respective groups of NVM cells. Exact matching phase substring sequences representing portions of at least one sample read are loaded into groups of NVM cells. One or more groups of NVM cells are identified where the stored reference sequence matches the loaded exact matching phase substring sequence using the arrays at Content Addressable Memories (CAMs). Approximate matching phase substring sequences are loaded into groups of NVM cells. One or more groups of NVM cells are identified where the stored reference sequence approximately matches the loaded approximate matching phase substring sequence using the arrays as Ternary CAMs (TCAMs). At least one of the reference sequence and the approximate matching phase substring sequence for each group of NVM cells includes at least one wildcard value when the arrays are used as TCAMs.

Abstract: Anon-volatile memory structure capable of storing weights for layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. An in-array multiplication can be performed between multi-bit valued inputs, or activations, for a layer of the DNN and multi-bit valued weights of the layer. Each bit of a weight value is stored in a binary valued memory cell of the memory array and each bit of the input is applied as a binary input to a word line of the array for the multiplication of the input with the weight. To perform a multiply and accumulate operation, the results of the multiplications are accumulated by adders connected to sense amplifiers along the bit lines of the array. The adders can be configured to multiple levels of precision, so that the same structure can accommodate weights and activations of 8-bit, 4-bit, and 2-bit precision.

Abstract: Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied as a pattern of voltage values on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight by determining whether or not the unit synapse conducts. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter. The arrangement can be extended to ternary inputs to realize a ternary-binary network (TBN) by adding a circuit to detect 0 input values and adjust the accumulated count accordingly.

Abstract: Anon-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.