Abstract: Non-volatile memory structures are presented for a content addressable memory (CAM) that can perform in-memory search operations for both ternary and binary valued key values. Each ternary or binary valued key bit is stored in a pair of memory cells along a bit line of a NAND memory array, with the stored keys searched by applying each ternary or binary valued bit of an input key as voltage levels on a pair of word lines. The system is highly scalable. The system can also be used to perform nearest neighbor searches between stored vectors and an input vector to find stored vectors withing a specified Hamming distance of the input vector.

Abstract: A non-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Abstract: A non-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Abstract: Exemplary methods and apparatus are disclosed that implement super-sparse image/video compression by storing image dictionary elements within a cross-bar resistive random access memory (ReRAM) array (or other suitable cross-bar NVM array). In illustrative examples, each column of the cross-bar ReRAM array stores the values for one dictionary element (such as one 4×4 dictionary element). Methods and apparatus are described for training (configuring) the cross-bar ReRAM array to generate and store the dictionary elements by sequentially applying patches from training images to the array using an unstructured Hebbian training procedure. Additionally, methods and apparatus are described for compressing an input image by applying patches from the input image to the ReRAM array to read out cross-bar column indices identifying the columns storing the various dictionary elements that best fit the image. This may be done in parallel using a set of ReRAM arrays.

Abstract: Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied as a pattern of voltage values on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight by determining whether or not the unit synapse conducts. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter.

Abstract: Methods and apparatus are disclosed for implementing principal component analysis (PCA) within a non-volatile memory (NVM) die of solid state drive (SSD) to reduce the dimensionality of machine learning data before the data is transferred to other components of the SSD, such as to a data storage controller equipped with a machine learning engine. The machine learning data may include, for example, training images for training an image recognition system in which the SSD is installed. In some examples, the on-chip PCA components of the NVM die are configured as under-the-array or next-to-the-array components. In other examples, one or more arrays of the NVM die are configured as multiplication cores for performing PCA matrix multiplication. In still other aspects, multiple NVM dies are arranged in parallel, each with on-chip PCA components to permit parallel concurrent on-chip processing of machine learning data.

Abstract: Exemplary methods and apparatus are disclosed that implement super-sparse image/video compression by storing image dictionary elements within a cross-bar resistive random access memory (ReRAM) array (or other suitable cross-bar NVM array). In illustrative examples, each column of the cross-bar ReRAM array stores the values for one dictionary element (such as one 4×4 dictionary element). Methods and apparatus are described for training (configuring) the cross-bar ReRAM array to generate and store the dictionary elements by sequentially applying patches from training images to the array using an unstructured Hebbian training procedure. Additionally, methods and apparatus are described for compressing an input image by applying patches from the input image to the ReRAM array to read out cross-bar column indices identifying the columns storing the various dictionary elements that best fit the image. This may be done in parallel using a set of ReRAM arrays.

Abstract: Methods and apparatus are disclosed for implementing principal component analysis (PCA) within a non-volatile memory (NVM) die of solid state drive (SSD) to reduce the dimensionality of machine learning data before the data is transferred to other components of the SSD, such as to a data storage controller equipped with a machine learning engine. The machine learning data may include, for example, training images for training an image recognition system in which the SSD is installed. In some examples, the on-chip PCA components of the NVM die are configured as under-the-array or next-to-the-array components. In other examples, one or more arrays of the NVM die are configured as multiplication cores for performing PCA matrix multiplication. In still other aspects, multiple NVM dies are arranged in parallel, each with on-chip PCA components to permit parallel concurrent on-chip processing of machine learning data.

Abstract: Non-volatile memory structures for performing compute in memory inferencing for neural networks are presented. To improve performance, both in terms of speed and energy consumption, weight matrices are replaced with their singular value decomposition (SVD) and use of a low rank approximations (LRAs). The decomposition matrices can be stored in a single array, with the resultant LRA matrices requiring fewer weight values to be stored. The reduced sizes of the LRA matrices allow for inferencing to be performed more quickly and with less power. In a high performance and energy efficiency mode, a reduced rank for the SVD matrices stored on a memory die is determined and used to increase performance and reduce power needed for an inferencing operation.

Abstract: A non-volatile memory device includes arrays of non-volatile memory cells that are configured to the store weights for a recurrent neural network (RNN) inference engine with a gated recurrent unit (GRU) cell. A set three non-volatile memory arrays, such as formed of storage class memory, store a corresponding three sets of weights and are used to perform compute-in-memory inferencing. The hidden state of a previous iteration and an external input are applied to the weights of the first and the of second of the arrays, with the output of the first array used to generate an input to the third array, which also receives the external input. The hidden state of the current generation is generated from the outputs of the second and third arrays.

Abstract: Anon-volatile memory structure capable of storing weights for layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. An in-array multiplication can be performed between multi-bit valued inputs, or activations, for a layer of the DNN and multi-bit valued weights of the layer. Each bit of a weight value is stored in a binary valued memory cell of the memory array and each bit of the input is applied as a binary input to a word line of the array for the multiplication of the input with the weight. To perform a multiply and accumulate operation, the results of the multiplications are accumulated by adders connected to sense amplifiers along the bit lines of the array. The adders can be configured to multiple levels of precision, so that the same structure can accommodate weights and activations of 8-bit, 4-bit, and 2-bit precision.

Abstract: A non-volatile memory device includes an array of non-volatile memory cells that are configured to store weights of a neural network. Associated with the array is a data latch structure that includes a page buffer, which can store weights for a layer of the neural network that is read out of the array, and a transfer buffer, that can store inputs for the neural network. The memory device can perform multiply and accumulate operations between inputs and weight of the neural network within the latch structure, avoiding the need to transfer data out of the array and associated latch structure for portions of an inference operation. By using binary weights and inputs, multiplication can be performed by bit-wise XNOR operations. The results can then be summed and activation applied, all within the latch structure.

Abstract: Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied as a pattern of voltage values on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight by determining whether or not the unit synapse conducts. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter. The arrangement can be extended to ternary inputs to realize a ternary-binary network (TBN) by adding a circuit to detect 0 input values and adjust the accumulated count accordingly.

Abstract: A non-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Abstract: Anon-volatile memory structure capable of storing layers of a deep neural network (DNN) and perform an inferencing operation within the structure is presented. A stack of bonded die pairs is connected by through silicon vias. Each bonded die pair includes a memory die, having one or more memory arrays onto which layers of the neural network are mapped, and a peripheral circuitry die, including the control circuits for performing the convolution or multiplication for the bonded die pair. The multiplications can either be done in-array on the memory die or in-logic on the peripheral circuitry die. The arrays can be formed into columns along the vias, allowing an inferencing operation to be performed by propagating an input up and down the columns, with the output of one level being the input of the subsequent layer.

Abstract: An innovative low-bit-width device may include a first digital-to-analog converter (DAC), a second DAC, a plurality of non-volatile memory (NVM) weight arrays, one or more analog-to-digital converters (ADCs), and a neural circuit. The first DAC is configured to convert a digital input signal into an analog input signal. The second DAC is configured to convert a digital previous hidden state (PHS) signal into an analog PHS signal. NVM weight arrays are configured to compute vector matrix multiplication (VMM) arrays based on the analog input signal and the analog PHS signal. The NVM weight arrays are coupled to the first DAC and the second DAC. The one or more ADCs are coupled to the plurality of NVM weight arrays and are configured to convert the VMM arrays into digital VMM values. The neural circuit is configured to process the digital VMM values into a new hidden state.

Abstract: Technology for reconfigurable input precision in-memory computing is disclosed herein. Reconfigurable input precision allows the bit resolution of input data to be changed to meet the requirements of in-memory computing operations. Voltage sources (that may include DACs) provide voltages that represent input data to memory cell nodes. The resolution of the voltage sources may be reconfigured to change the precision of the input data. In one parallel mode, the number of DACs in a DAC node is used to configure the resolution. In one serial mode, the number of cycles over which a DAC provides voltages is used to configure the resolution. The memory system may include relatively low resolution voltage sources, which avoids the need to have complex high resolution voltage sources (e.g., high resolution DACs). Lower resolution voltage sources can take up less area and/or use less power than higher resolution voltage sources.

Abstract: Apparatuses and techniques are described for reading crosspoint arrays of memory cells with high bandwidth and a relatively small page buffer. Multiple crosspoint arrays (XPAs) are read in parallel, with one memory cell per XPA being read, in a bank of XPAs. To reduce the read time, a row can be selected for the XPAs, after which memory cells in different columns are read, one column at a time, while the same row is selected. This avoids the need to transmit commands and a row address for re-selecting the row in each successive read operation. The XPAs may be ungrouped, or one XPA may be accessible at a time in a group. In one option, the XPAs are arranged in sets, either individually or in groups, and one set is accessible at a time.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a non-volatile memory cell and multi-bit input circuitry that simultaneously receives a plurality of bits, receives a supply voltage, converts the plurality of bits and the supply voltage into a multiply voltage, and applies the multiply voltage to the non-volatile memory cell. The non-volatile memory cell may pass a memory cell current in response to the multiply voltage. A magnitude of the multiply voltage may represent a multiplier. The memory cell current may represent a product of the multiplier and a multiplicand stored in the non-volatile memory cell.

Abstract: Systems and methods for dynamically assigning memory array die to CMOS die of a plurality of stacked die during memory operations are described. The plurality of stacked die may be vertically stacked and connected together via one or more vertical through-silicon via (TSV) connections. The memory array die may only comprise memory cell structures (e.g., vertical NAND strings) without column decoders, row decoders, charge pumps, sense amplifiers, control circuitry, page registers, or state machines. The CMOS die may contain support circuitry necessary for performing the memory operations, such as read and write memory operations. The one or more vertical TSV connections may allow each memory array die of the plurality of stacked die to communicate with or be electrically connected to one or more CMOS die of the plurality of stacked die.