Abstract: A storage device and method of controlling a storage device are disclosed. The storage device includes a host, a logic die, and a high bandwidth memory stack including a memory die. A computation lookup table is stored on a memory array of the memory die. The host sends a command to perform an operation utilizing a kernel and a plurality of input feature maps, includes finding the product of a weight of the kernel and values of multiple input feature maps. The computation lookup table includes a row corresponding to a weight of the kernel, and a column corresponding to a value of the input feature maps. A result value stored at a position corresponding to a row and a column is the product of the weight corresponding to the row and the value corresponding to the column.

Abstract: A flash memory device includes a plurality of flash memory cell arrays, wherein: a flash memory cell array in the plurality of flash memory cell arrays comprises a plurality of layers of flash memory cell planes; and a flash memory cell plane includes a plurality of flash memory cells. The flash memory device further includes a logic circuitry coupled to the plurality of flash memory cell arrays, configured to perform operations using the plurality of flash memory cell arrays; and a sensing circuitry configured to access a corresponding flash memory cell plane among the plurality of flash memory cell planes.

Abstract: This application describes an accelerator, a computer system, and a method for accelerating Graph Neural Network (GNN) node attribute fetching. The accelerator comprises a graph structure processor configured to obtain, according to a root node in a graph, a plurality of candidate nodes for GNN processing on the root node; a GNN sampler configured to generate sampled graph nodes based on the plurality of candidate nodes from the graph structure processor; and a GNN attribute processor configured to fetch attribute data of the sampled graph nodes received from the GNN sampler for the GNN processing on the root node.

Abstract: The disclosed embodiments relate to a computer system with a cache memory that supports tagless addressing. During operation, the system receives a request to perform a memory access, wherein the request includes a virtual address. In response to the request, the system performs an address-translation operation, which translates the virtual address into both a physical address and a cache address. Next, the system uses the physical address to access one or more levels of physically addressed cache memory, wherein accessing a given level of physically addressed cache memory involves performing a tag-checking operation based on the physical address. If the access to the one or more levels of physically addressed cache memory fails to hit on a cache line for the memory access, the system uses the cache address to directly index a cache memory, wherein directly indexing the cache memory does not involve performing a tag-checking operation and eliminates the tag storage overhead.

Abstract: This specification describes methods and systems for accelerating attribute data access for graph neural network (GNN) processing. An example method includes: receiving a root node identifier corresponding to a node in a graph for GNN processing; determining one or more candidate node identifiers according to the root node identifier, wherein attribute data corresponding to the one or more candidate node identifiers are sequentially stored in a memory; and sampling one or more graph node identifiers at least from the one or more candidate node identifiers for the GNN processing.

Abstract: The present application discloses a computing system for implementing an artificial neural network model. The artificial neural network model has a structure of multiple layers. The computing system comprises a first processing unit, a second processing unit, and a third processing unit. The first processing unit performs computations of the first layer based on a first part of input data of the first layer to generate a first part of output data. The second processing unit performs computations of the first layer based on a second part of the input data of the first layer so as to generate a second part of the output data. The third processing unit performs computations of the second layer based on the first part and the second part of the output data. The first processing unit, the second processing unit, and the third processing unit have the same structure.

Abstract: The total memory space that is logically available to a processor in a general-purpose graphics processing unit (GPGPU) module is increased to accommodate terabyte-sized amounts of data by utilizing the memory space in an external memory module, and by further utilizing a portion of the memory space in a number of other external memory modules.

Abstract: The present application discloses a graph neural network processing method and associated machine and system. The graph neural network method is used for a master, wherein the master, a first worker and a second worker train the graph neural network in a distributed environment. The method includes: receiving a first request from the first worker and a second request from the second worker, wherein the first worker sends the first request to the master to obtain at least an attribute of a first requested node, and the second worker sends a second request to the master to obtain at least an attribute of a second requested node; determining whether the first requested node and the second requested node are the same nodes and generating a determination result accordingly; and selectively performing broadcast or unicast to the first worker and the second worker, at least based on the determination result.

Abstract: A cache memory includes cache lines to store information. The stored information is associated with physical addresses that include first, second, and third distinct portions. The cache lines are indexed by the second portions of respective physical addresses associated with the stored information. The cache memory also includes one or more tables, each of which includes respective table entries that are indexed by the first portions of the respective physical addresses. The respective table entries in each of the one or more tables are to store indications of the second portions of respective physical addresses associated with the stored information.

Abstract: A programmable device receives commands from a processor and, based on the commands: identifies a root node in a graph; identifies nodes in the graph that are neighbors of the root node; identifies nodes in the graph that are neighbors of the neighbors; retrieves data associated with the root node; retrieves data associated with at least a subset of the nodes that are neighbors of the root node and that are neighbors of the neighbor nodes; and writes the data that is retrieved into a memory.

Abstract: This specification describes methods and systems for accessing attribute data in graph neural network (GNN) processing. An example system includes: a plurality of cores, each of the plurality of cores comprises a key-value fetcher and a filter, and is programmable using a software interface to support a plurality of data formats of the GNN attribute data, wherein: the key-value fetcher is programmable using the software interface to perform key-value fetching associated with accessing the GNN attribute data, and the filter of at least one of the plurality of cores is programmable using the software interface to sample node identifiers associated with accessing the GNN attribute data; and a first memory communicatively coupled with the plurality of cores, wherein the first memory is configured to store data shared by the plurality of cores.

Abstract: A method of processing in-memory commands in a high-bandwidth memory (HBM) system includes sending a function-in-HBM instruction to the HBM by a HBM memory controller of a GPU. A logic component of the HBM receives the FIM instruction and coordinates the instructions execution using the controller, an ALU, and a SRAM located on the logic component.

Abstract: The systems and methods are configured to efficiently and effectively access memory. In one embodiment, a memory controller comprises a request queue, a buffer, a control component, and a data path system. The request queue receives memory access requests. The control component is configured to process information associated with access requests via a first narrow memory channel and a second narrow memory channel. The first narrow memory channel and the second narrow memory channel can have a portion of command/control communication lines and address communication lines that are included in and shared between the first narrow memory channel and the second narrow memory channel. The data path system can include a first data module and one set of unshared data lines associated with the first memory channel and a second data module and another set of unshared data lines associated with second memory channel.

Abstract: A method of correcting a memory error of a dynamic random-access memory module (DRAM) using a double data rate (DDR) interface, the method includes conducting a memory transaction including multiple bursts with a memory controller to send data from data chips of the DRAM to the memory controller, detecting one or more errors using an ECC chip of the DRAM, determining a number of the bursts having the errors using the ECC chip of the DRAM, determining whether the number of the bursts having the errors is greater than a threshold number, determining a type of the errors, and directing the memory controller based on the determined type of the errors, wherein the DRAM includes a single ECC chip per memory channel.

Abstract: A system for pruning weights during training of a neural network includes a configurable pruning hardware unit that is configured to: receive, from a neural network training engine, inputs including the weights, gradients associated with the weights, and a prune indicator per weight; select unpruned weights for pruning; prune the unpruned weights selected for pruning; update the prune indicator per weight for the weights that are selected and pruned; and provide the updated prune indicator to the training engine for the next iteration or epoch. The pruning hardware unit can be configured to perform incremental pruning or non-incremental pruning.

Abstract: According to some example embodiments of the present disclosure, in a method for a memory lookup mechanism in a high-bandwidth memory system, the method includes: using a memory die to conduct a multiplication operation using a lookup table (LUT) methodology by accessing a LUT, which includes floating point operation results, stored on the memory die; sending, by the memory die, a result of the multiplication operation to a logic die including a processor and a buffer; and conducting, by the logic die, a matrix multiplication operation using computation units.

Abstract: The maximum capacity of a very fast memory in a system that requires very fast memory access times is increased by adding a memory with remote access times that are slower than required, and then moving infrequently accessed data from the memory with the very fast access times to the memory with the slow access times.

Abstract: A high bandwidth memory system. In some embodiments, the system includes: a memory stack having a plurality of memory dies and eight 128-bit channels; and a logic die, the memory dies being stacked on, and connected to, the logic die; wherein the logic die may be configured to operate a first channel of the 128-bit channels in: a first mode, in which a first 64 bits operate in pseudo-channel mode, and a second 64 bits operate as two 32-bit fine-grain channels, or a second mode, in which the first 64 bits operate as two 32-bit fine-grain channels, and the second 64 bits operate as two 32-bit fine-grain channels.

Abstract: A dual-model memory interface of a computing system is provided, configurable to present memory interfaces having differently-graded bandwidth capacity to different processors of the computing system. A mode switch controller of the memory interface controller, based on at least an arbitration rule written to a configuration register, switches the memory interface controller between a narrow-band mode and a wide-band mode. In each mode, the memory interface controller disables either a plurality of narrow-band memory interfaces of the memory interface controller according to a first bus standard, or a wide-band memory interface of the memory interface controller according to a second bus standard. The memory interface controller virtualizes a plurality of system memory units of the computing system as a virtual wide-band memory unit according to the second bus standard, or virtualizes a system memory unit of the computing system as a virtual narrow-band memory unit according to the first bus standard.

Abstract: This application describes a hardware accelerator, a computer system, and a method for accelerating Graph Neural Network (GNN) node attribute fetching. The hardware accelerator comprises a GNN attribute processor; and a first memory, wherein the GNN attribute processor is configured to: receive a graph node identifier; determine a target memory address within the first memory based on the graph node identifier; determine, based on the received graph node identifier, whether attribute data corresponding to the received graph node identifier is cached in the first memory at the target memory address; and in response to determining that the attribute data is not cached in the first memory: fetch the attribute data from a second memory, and write the fetched attribute data into the first memory at the target memory address.