Abstract: Anon-volatile memory structure capable of storing weights for layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. An in-array multiplication can be performed between multi-bit valued inputs, or activations, for a layer of the DNN and multi-bit valued weights of the layer. Each bit of a weight value is stored in a binary valued memory cell of the memory array and each bit of the input is applied as a binary input to a word line of the array for the multiplication of the input with the weight. To perform a multiply and accumulate operation, the results of the multiplications are accumulated by adders connected to sense amplifiers along the bit lines of the array. The adders can be configured to multiple levels of precision, so that the same structure can accommodate weights and activations of 8-bit, 4-bit, and 2-bit precision.

Abstract: A non-volatile memory device includes arrays of non-volatile memory cells that are configured to the store weights for a recurrent neural network (RNN) inference engine with a gated recurrent unit (GRU) cell. A set three non-volatile memory arrays, such as formed of storage class memory, store a corresponding three sets of weights and are used to perform compute-in-memory inferencing. The hidden state of a previous iteration and an external input are applied to the weights of the first and the of second of the arrays, with the output of the first array used to generate an input to the third array, which also receives the external input. The hidden state of the current generation is generated from the outputs of the second and third arrays.

Abstract: A non-volatile memory device includes an array of non-volatile memory cells that are configured to store weights of a neural network. Associated with the array is a data latch structure that includes a page buffer, which can store weights for a layer of the neural network that is read out of the array, and a transfer buffer, that can store inputs for the neural network. The memory device can perform multiply and accumulate operations between inputs and weight of the neural network within the latch structure, avoiding the need to transfer data out of the array and associated latch structure for portions of an inference operation. By using binary weights and inputs, multiplication can be performed by bit-wise XNOR operations. The results can then be summed and activation applied, all within the latch structure.

Abstract: Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied as a pattern of voltage values on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight by determining whether or not the unit synapse conducts. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter. The arrangement can be extended to ternary inputs to realize a ternary-binary network (TBN) by adding a circuit to detect 0 input values and adjust the accumulated count accordingly.

Abstract: A non-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Abstract: Anon-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Abstract: An innovative low-bit-width device may include a first digital-to-analog converter (DAC), a second DAC, a plurality of non-volatile memory (NVM) weight arrays, one or more analog-to-digital converters (ADCs), and a neural circuit. The first DAC is configured to convert a digital input signal into an analog input signal. The second DAC is configured to convert a digital previous hidden state (PHS) signal into an analog PHS signal. NVM weight arrays are configured to compute vector matrix multiplication (VMM) arrays based on the analog input signal and the analog PHS signal. The NVM weight arrays are coupled to the first DAC and the second DAC. The one or more ADCs are coupled to the plurality of NVM weight arrays and are configured to convert the VMM arrays into digital VMM values. The neural circuit is configured to process the digital VMM values into a new hidden state.

Abstract: Technology for reconfigurable input precision in-memory computing is disclosed herein. Reconfigurable input precision allows the bit resolution of input data to be changed to meet the requirements of in-memory computing operations. Voltage sources (that may include DACs) provide voltages that represent input data to memory cell nodes. The resolution of the voltage sources may be reconfigured to change the precision of the input data. In one parallel mode, the number of DACs in a DAC node is used to configure the resolution. In one serial mode, the number of cycles over which a DAC provides voltages is used to configure the resolution. The memory system may include relatively low resolution voltage sources, which avoids the need to have complex high resolution voltage sources (e.g., high resolution DACs). Lower resolution voltage sources can take up less area and/or use less power than higher resolution voltage sources.

Abstract: Apparatuses and techniques are described for reading crosspoint arrays of memory cells with high bandwidth and a relatively small page buffer. Multiple crosspoint arrays (XPAs) are read in parallel, with one memory cell per XPA being read, in a bank of XPAs. To reduce the read time, a row can be selected for the XPAs, after which memory cells in different columns are read, one column at a time, while the same row is selected. This avoids the need to transmit commands and a row address for re-selecting the row in each successive read operation. The XPAs may be ungrouped, or one XPA may be accessible at a time in a group. In one option, the XPAs are arranged in sets, either individually or in groups, and one set is accessible at a time.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a non-volatile memory cell and multi-bit input circuitry that simultaneously receives a plurality of bits, receives a supply voltage, converts the plurality of bits and the supply voltage into a multiply voltage, and applies the multiply voltage to the non-volatile memory cell. The non-volatile memory cell may pass a memory cell current in response to the multiply voltage. A magnitude of the multiply voltage may represent a multiplier. The memory cell current may represent a product of the multiplier and a multiplicand stored in the non-volatile memory cell.

Abstract: Systems and methods for dynamically assigning memory array die to CMOS die of a plurality of stacked die during memory operations are described. The plurality of stacked die may be vertically stacked and connected together via one or more vertical through-silicon via (TSV) connections. The memory array die may only comprise memory cell structures (e.g., vertical NAND strings) without column decoders, row decoders, charge pumps, sense amplifiers, control circuitry, page registers, or state machines. The CMOS die may contain support circuitry necessary for performing the memory operations, such as read and write memory operations. The one or more vertical TSV connections may allow each memory array die of the plurality of stacked die to communicate with or be electrically connected to one or more CMOS die of the plurality of stacked die.

Abstract: An innovative low-bit-width device may include a first digital-to-analog converter (DAC), a second DAC, a plurality of non-volatile memory (NVM) weight arrays, one or more analog-to-digital converters (ADCs), and a neural circuit. The first DAC is configured to convert a digital input signal into an analog input signal. The second DAC is configured to convert a digital previous hidden state (PHS) signal into an analog PHS signal. NVM weight arrays are configured to compute vector matrix multiplication (VMM) arrays based on the analog input signal and the analog PHS signal. The NVM weight arrays are coupled to the first DAC and the second DAC. The one or more ADCs are coupled to the plurality of NVM weight arrays and are configured to convert the VMM arrays into digital VMM values. The neural circuit is configured to process the digital VMM values into a new hidden state.

Abstract: Techniques are presented for performing in-memory matrix multiplication operations for binary input, binary weight valued convolution neural network (CNN) inferencing. The weights of a filter are stored in pairs of memory cells of a storage class memory device, such as a ReRAM or phase change memory based devices. To reduce current consumption, the binary valued filters are transformed into ternary valued filters by taking sums and differences of binary valued filter pairs. The zero valued weights of the transformed filters are stored as a pair of high resistance state memory cells, reducing current consumption during convolution. The results of the in-memory multiplications are pair-wise combined to compensate for the filter transformations. To compensate for zero valued weights, a zero weight register stores the number of zero weights along each bit line and is used to initialize counter values for accumulating the multiplication operations.

Abstract: Apparatuses and techniques are described for reading crosspoint arrays of memory cells with high bandwidth and a relatively small page buffer. Multiple crosspoint arrays (XPAs) are read in parallel, with one memory cell per XPA being read, in a bank of XPAs. To reduce the read time, a row can be selected for the XPAs, after which memory cells in different columns are read, one column at a time, while the same row is selected. This avoids the need to transmit commands and a row address for re-selecting the row in each successive read operation. The XPAs may be ungrouped, or one XPA may be accessible at a time in a group. In one option, the XPAs are arranged in sets, either individually or in groups, and one set is accessible at a time.

Abstract: Methods and apparatus are disclosed for implementing principal component analysis (PCA) within a non-volatile memory (NVM) die of solid state drive (SSD) to reduce the dimensionality of machine learning data before the data is transferred to other components of the SSD, such as to a data storage controller equipped with a machine learning engine. The machine learning data may include, for example, training images for training an image recognition system in which the SSD is installed. In some examples, the on-chip PCA components of the NVM die are configured as under-the-array or next-to-the-array components. In other examples, one or more arrays of the NVM die are configured as multiplication cores for performing PCA matrix multiplication. In still other aspects, multiple NVM dies are arranged in parallel, each with on-chip PCA components to permit parallel concurrent on-chip processing of machine learning data.

Abstract: Techniques are presented for accelerating in-memory matrix multiplication operations for a convolution neural network (CNN) inference in which the weights of a filter are stored in the memory of a storage class memory device, such as a ReRAM or phase change memory based device. To improve performance for inference operations when filters exhibit sparsity, a zero column index and a zero row index are introduced to account for columns and rows having all zero weight values. These indices can be saved in a register on the memory device and when performing a column/row oriented matrix multiplication, if the zero row/column index indicates that the column/row contains all zero weights, the access of the corresponding bit/word line is skipped as the result will be zero regardless of the input.

Abstract: Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter. The arrangement extends to ternary inputs to realize a ternary-binary network (TBN) by adding a circuit to detect 0 input values and adjust the accumulated count accordingly. The arrangement further extends to a ternary-ternary network (TTN) by allowing 0 weight values in a unit synapse, maintaining the number of 0 weights in a register, and adjusting the count accordingly.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a non-volatile memory cell and multi-bit input circuitry that simultaneously receives a plurality of bits, receives a supply voltage, converts the plurality of bits and the supply voltage into a multiply voltage, and applies the multiply voltage to the non-volatile memory cell. The non-volatile memory cell may pass a memory cell current in response to the multiply voltage. A magnitude of the multiply voltage may represent a multiplier. The memory cell current may represent a product of the multiplier and a multiplicand stored in the non-volatile memory cell.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a non-volatile memory cell and multi-bit input circuitry that simultaneously receives a plurality of bits, receives a supply voltage, converts the plurality of bits and the supply voltage into a multiply voltage, and applies the multiply voltage to the non-volatile memory cell. The non-volatile memory cell may pass a memory cell current in response to the multiply voltage. A magnitude of the multiply voltage may represent a multiplier. The memory cell current may represent a product of the multiplier and a multiplicand stored in the non-volatile memory cell.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a non-volatile memory cell and multi-bit input circuitry that simultaneously receives a plurality of bits, receives a supply voltage, converts the plurality of bits and the supply voltage into a multiply voltage, and applies the multiply voltage to the non-volatile memory cell. The non-volatile memory cell may pass a memory cell current in response to the multiply voltage. A magnitude of the multiply voltage may represent a multiplier. The memory cell current may represent a product of the multiplier and a multiplicand stored in the non-volatile memory cell.