Patents by Inventor Krishna T. MALLADI
Krishna T. MALLADI has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11947961Abstract: According to some example embodiments of the present disclosure, in a method for a memory lookup mechanism in a high-bandwidth memory system, the method includes: using a memory die to conduct a multiplication operation using a lookup table (LUT) methodology by accessing a LUT, which includes floating point operation results, stored on the memory die; sending, by the memory die, a result of the multiplication operation to a logic die including a processor and a buffer; and conducting, by the logic die, a matrix multiplication operation using computation units.Type: GrantFiled: November 30, 2022Date of Patent: April 2, 2024Assignee: Samsung Electronics Co., Ltd.Inventors: Peng Gu, Krishna T. Malladi, Hongzhong Zheng
-
Patent number: 11940922Abstract: A method of processing in-memory commands in a high-bandwidth memory (HBM) system includes sending a function-in-HBM instruction to the HBM by a HBM memory controller of a GPU. A logic component of the HBM receives the FIM instruction and coordinates the instructions execution using the controller, an ALU, and a SRAM located on the logic component.Type: GrantFiled: December 14, 2022Date of Patent: March 26, 2024Assignee: Samsung Electronics Co., Ltd.Inventors: Mu-Tien Chang, Krishna T. Malladi, Dimin Niu, Hongzhong Zheng
-
Patent number: 11934669Abstract: A processor includes a plurality of memory units, each of the memory units including a plurality of memory cells, wherein each of the memory units is configurable to operate as memory, as a computation unit, or as a hybrid memory-computation unit.Type: GrantFiled: July 29, 2020Date of Patent: March 19, 2024Assignee: Samsung Electronics Co., Ltd.Inventors: Dimin Niu, Shuangchen Li, Bob Brennan, Krishna T. Malladi, Hongzhong Zheng
-
Publication number: 20240086292Abstract: A method for computing. In some embodiments, the method includes: calculating an advantage score of a first computing task, the advantage score being a measure of an extent to which a plurality of function in memory circuits is capable of executing the first computing task more efficiently by than one or more extra-memory processing circuits, the first computing task including instructions and data; in response to determining that the advantage score of the first computing task is less than a first threshold, executing the first computing task by the one or more extra-memory processing circuits; and in response to determining that the first computing task is at least equal to the first threshold: compiling the instructions for execution by the function in memory circuits; formatting the data for the function in memory circuits; and executing the first computing task, by the function in memory circuits.Type: ApplicationFiled: November 17, 2023Publication date: March 14, 2024Inventors: Krishna T. Malladi, Wenqin Huangfu
-
Patent number: 11921638Abstract: According to some embodiments of the present invention, there is provided a hybrid cache memory for a processing device having a host processor, the hybrid cache memory comprising: a high bandwidth memory (HBM) configured to store host data; a non-volatile memory (NVM) physically integrated with the HBM in a same package and configured to store a copy of the host data at the HBM; and a cache controller configured to be in bi-directional communication with the host processor, and to manage data transfer between the HBM and NVM and, in response to a command received from the host processor, to manage data transfer between the hybrid cache memory and the host processor.Type: GrantFiled: June 6, 2022Date of Patent: March 5, 2024Assignee: Samsung Electronics Co., Ltd.Inventors: Krishna T. Malladi, Hongzhong Zheng
-
Patent number: 11921656Abstract: An apparatus may include a heterogeneous computing environment that may be controlled, at least in part, by a task scheduler in which the heterogeneous computing environment may include a processing unit having fixed logical circuits configured to execute instructions; a reprogrammable processing unit having reprogrammable logical circuits configured to execute instructions that include instructions to control processing-in-memory functionality; and a stack of high-bandwidth memory dies in which each may be configured to store data and to provide processing-in-memory functionality controllable by the reprogrammable processing unit such that the reprogrammable processing unit is at least partially stacked with the high-bandwidth memory dies. The task scheduler may be configured to schedule computational tasks between the processing unit, and the reprogrammable processing unit.Type: GrantFiled: January 17, 2022Date of Patent: March 5, 2024Assignee: SAMSUNG ELECTRONICS CO., LTD.Inventors: Krishna T. Malladi, Hongzhong Zheng
-
Patent number: 11914903Abstract: A device may include an interconnect interface, a memory system including one or more first type memory devices to receive first data, one or more second type memory devices to receive second data, and an accelerator configured to perform an operation using the first data and the second data. The memory system may further include a cache configured to cache the second data for the one or more second type memory devices. A device may include an interconnect interface, a memory system coupled to the interconnect interface to receive data, an accelerator coupled to the memory system, and virtualization logic configured to partition one or more resources of the accelerator into one or more virtual accelerators, wherein a first one of the one or more virtual accelerators may be configured to perform a first operation on a first portion of the data.Type: GrantFiled: October 8, 2021Date of Patent: February 27, 2024Assignee: SAMSUNG ELECTRONICS CO., LTD.Inventors: Yang Seok Ki, Krishna T. Malladi, Rekha Pitchumani
-
Patent number: 11893239Abstract: A high-bandwidth memory (HBM) system includes an HBM device and a logic circuit. The logic circuit includes a first interface coupled to a host device and a second interface coupled to the HBM device. The logic circuit receives a first command from the host device through the first interface and converts the received first command to a first processing-in-memory (PIM) command that is sent to the HBM device through the second interface. The first PIM command has a deterministic latency for completion. The logic circuit further receives a second command from the host device through the first interface and converting the received second command to a second PIM command that is sent to the HBM device through the second interface. The second PIM command has a non-deterministic latency for completion.Type: GrantFiled: February 10, 2020Date of Patent: February 6, 2024Inventors: Krishna T. Malladi, Hongzhong Zheng
-
Publication number: 20240028332Abstract: According to some example embodiments of the present disclosure, in a method for a memory lookup mechanism in a high-bandwidth memory system, the method includes: using a memory die to conduct a multiplication operation using a lookup table (LUT) methodology by accessing a LUT, which includes floating point operation results, stored on the memory die; sending, by the memory die, a result of the multiplication operation to a logic die including a processor and a buffer; and conducting, by the logic die, a matrix multiplication operation using computation units.Type: ApplicationFiled: October 2, 2023Publication date: January 25, 2024Inventors: Peng Gu, Krishna T. Malladi, Hongzhong Zheng
-
Publication number: 20240004646Abstract: According to one embodiment, a memory module includes: a memory die including a dynamic random access memory (DRAM) banks, each including: an array of DRAM cells arranged in pages; a row buffer to store values of one of the pages; an input/output (IO) module; and an in-memory compute (IMC) module including: an arithmetic logic unit (ALU) to receive operands from the row buffer or the IO module and to compute an output based on the operands and one of a plurality of ALU operations; and a result register to store the output of the ALU; and a controller to: receive, from a host processor, operands and an instruction; determine, based on the instruction, a data layout; supply the operands to the DRAM banks in accordance with the data layout; and control an IMC module to perform one of the ALU operations on the operands in accordance with the instruction.Type: ApplicationFiled: September 14, 2023Publication date: January 4, 2024Inventors: Krishna T. Malladi, Wenqin Huangfu
-
Patent number: 11853186Abstract: A method for computing. In some embodiments, the method includes: calculating an advantage score of a first computing task, the advantage score being a measure of an extent to which a plurality of function in memory circuits is capable of executing the first computing task more efficiently by than one or more extra-memory processing circuits, the first computing task including instructions and data; in response to determining that the advantage score of the first computing task is less than a first threshold, executing the first computing task by the one or more extra-memory processing circuits; and in response to determining that the first computing task is at least equal to the first threshold: compiling the instructions for execution by the function in memory circuits; formatting the data for the function in memory circuits; and executing the first computing task, by the function in memory circuits.Type: GrantFiled: March 21, 2022Date of Patent: December 26, 2023Assignee: Samsung Electronics Co., Ltd.Inventors: Krishna T. Malladi, Wenqin Huangfu
-
Patent number: 11853210Abstract: Provided are systems, methods, and apparatuses for providing a storage resource. The method can include: operating a first controller coupled to a network interface in accordance with a cache coherent protocol; performing at least one operation on data associated with a cache using a second controller coupled to the first controller and coupled to a first memory; and storing the data on a second memory coupled to one of the first controller or the second controller.Type: GrantFiled: April 30, 2021Date of Patent: December 26, 2023Assignee: Samsung Electronics Co., Ltd.Inventors: Krishna T. Malladi, Andrew Chang, Ehsan Najafabadi
-
Publication number: 20230367711Abstract: Provided are systems, methods, and apparatuses for providing a storage resource. The method can include: operating a first controller coupled to a network interface in accordance with a cache coherent protocol; performing at least one operation on data associated with a cache using a second controller coupled to the first controller and coupled to a first memory; and storing the data on a second memory coupled to one of the first controller or the second controller.Type: ApplicationFiled: July 24, 2023Publication date: November 16, 2023Inventors: Krishna T. Malladi, Andrew Chang, Ehsan Najafabadi
-
Patent number: 11782707Abstract: According to one embodiment, a memory module includes: a memory die including a dynamic random access memory (DRAM) banks, each including: an array of DRAM cells arranged in pages; a row buffer to store values of one of the pages; an input/output (IO) module; and an in-memory compute (IMC) module including: an arithmetic logic unit (ALU) to receive operands from the row buffer or the IO module and to compute an output based on the operands and one of a plurality of ALU operations; and a result register to store the output of the ALU; and a controller to: receive, from a host processor, operands and an instruction; determine, based on the instruction, a data layout; supply the operands to the DRAM banks in accordance with the data layout; and control an IMC module to perform one of the ALU operations on the operands in accordance with the instruction.Type: GrantFiled: December 10, 2021Date of Patent: October 10, 2023Assignee: Samsung Electronics Co., Ltd.Inventors: Krishna T. Malladi, Wenqin Huangfu
-
Patent number: 11775294Abstract: According to some example embodiments of the present disclosure, in a method for a memory lookup mechanism in a high-bandwidth memory system, the method includes: using a memory die to conduct a multiplication operation using a lookup table (LUT) methodology by accessing a LUT, which includes floating point operation results, stored on the memory die; sending, by the memory die, a result of the multiplication operation to a logic die including a processor and a buffer; and conducting, by the logic die, a matrix multiplication operation using computation units.Type: GrantFiled: November 30, 2021Date of Patent: October 3, 2023Assignee: Samsung Electronics Co., Ltd.Inventors: Peng Gu, Krishna T. Malladi, Hongzhong Zheng
-
Publication number: 20230289081Abstract: A storage device and method of controlling a storage device are disclosed. The storage device includes a host, a logic die, and a high bandwidth memory stack including a memory die. A computation lookup table is stored on a memory array of the memory die. The host sends a command to perform an operation utilizing a kernel and a plurality of input feature maps, includes finding the product of a weight of the kernel and values of multiple input feature maps. The computation lookup table includes a row corresponding to a weight of the kernel, and a column corresponding to a value of the input feature maps. A result value stored at a position corresponding to a row and a column is the product of the weight corresponding to the row and the value corresponding to the column.Type: ApplicationFiled: May 11, 2023Publication date: September 14, 2023Inventors: Peng Gu, Krishna T. Malladi, Hongzhong Zheng
-
Publication number: 20230281128Abstract: A memory system is disclosed. The memory system may include a first cache-coherent interconnect memory module and a second cache-coherent interconnect memory module. A cache-coherent interconnect switch may connect the first cache-coherent interconnect memory module, the second cache-coherent interconnect memory module, and a processor. A processing element may process a data stored on at least one of the first cache-coherent interconnect memory module and the second cache-coherent interconnect memory module.Type: ApplicationFiled: June 7, 2022Publication date: September 7, 2023Inventors: Wenqin HUANGFU, Krishna T. MALLADI, Andrew CHANG
-
Patent number: 11681451Abstract: A storage device and method of controlling a storage device are disclosed. The storage device includes a host, a logic die, and a high bandwidth memory stack including a memory die. A computation lookup table is stored on a memory array of the memory die. The host sends a command to perform an operation utilizing a kernel and a plurality of input feature maps, includes finding the product of a weight of the kernel and values of multiple input feature maps. The computation lookup table includes a row corresponding to a weight of the kernel, and a column corresponding to a value of the input feature maps. A result value stored at a position corresponding to a row and a column is the product of the weight corresponding to the row and the value corresponding to the column.Type: GrantFiled: September 13, 2021Date of Patent: June 20, 2023Assignee: Samsung Electronics Co., Ltd.Inventors: Peng Gu, Krishna T. Malladi, Hongzhong Zheng
-
Publication number: 20230146611Abstract: A system and method for training a neural network. In some embodiments, the system includes a computational storage device including a backing store. The computational storage device may be configured to: store, in the backing store, an embedding table for a neural network embedding operation; receive a first index vector including a first index and a second index; retrieve, from the backing store: a first row of the embedding table, corresponding to the first index, and a second row of the embedding table, corresponding to the second index; and calculate a first embedded vector based on the first row and the second row.Type: ApplicationFiled: February 9, 2022Publication date: May 11, 2023Inventors: Shiyu LI, Krishna T. MALLADI, Andrew CHANG, Yang Seok KI
-
Publication number: 20230147472Abstract: A system and method for training a neural network. In some embodiments, the system includes: a graphics processing unit cluster; and a computational storage cluster connected to the graphics processing unit cluster by a cache-coherent system interconnect. The graphics processing unit cluster may include one or more graphics processing units. The computational storage cluster may include one or more computational storage devices. A first computational storage device of the one or more computational storage devices may be configured to (i) store an embedding table, (ii) receive an index vector including a first index and a second index; and (iii) calculate an embedded vector based on: a first row of the embedding table, corresponding to the first index, and a second row of the embedding table, corresponding to the second index.Type: ApplicationFiled: February 11, 2022Publication date: May 11, 2023Inventors: Shiyu LI, Krishna T. MALLADI, Andrew CHANG, Yang Seok KI