Patents by Inventor Siddharth SHETH
Siddharth SHETH has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20250209017Abstract: A digital in-memory compute (DIMC) accelerator system using a chiplet architecture. The system includes a host device is configured to compile computational workload data for a target application obtained from data gathering devices into an instruction set architecture (ISA) graph to be executed by a plurality of accelerator apparatuses. Each such accelerator includes a plurality of chiplets, each of which includes a plurality of tiles, and each such tile includes a plurality of slices, a central processing unit (CPU), and a DIMC device configured to perform high throughput computations using the ISA graph to process the computational workload. The target application can include natural language processing (NLP), autonomous reasoning/decision-making, video/image processing, cybersecurity/fraud detection, manufacturing/industrial processes, agentic artificial intelligence (AI), or smart cities/Internet of Things (IoT).Type: ApplicationFiled: March 11, 2025Publication date: June 26, 2025Inventors: Sudeep Bhoja, Siddharth Sheth
-
Patent number: 12271321Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.Type: GrantFiled: October 24, 2023Date of Patent: April 8, 2025Assignee: d-MATRIX CORPORATIONInventors: Sudeep Bhoja, Siddharth Sheth
-
Patent number: 12260223Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications, including generative AI. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions.Type: GrantFiled: November 23, 2022Date of Patent: March 25, 2025Assignee: d-MATRIX CORPORATIONInventors: Sudeep Bhoja, Siddharth Sheth
-
Publication number: 20250061319Abstract: A server system using switch linking between groups of AI processing systems. The system includes at least a first central processing unit (CPU) coupled to a first group of AI processing systems via a first switch and a second CPU coupled to a second group of AI processing systems via a second switch. Each of the CPUs is also coupled to a separate group of memory devices and a communication link is configured between the first switch and the second switch to communicate information between the first group of AI processing systems and the second group of AI processing systems. Each of these AI processing system groups include a plurality of AI processing modules, and each of these modules include a plurality of chiplet devices. Each chiplet device is configured with a plurality of in-memory compute (IMC) devices for processing neural network model workloads.Type: ApplicationFiled: November 1, 2024Publication date: February 20, 2025Inventors: Keith NELLIS, Sudeep Bhoja, Siddharth Sheth
-
Publication number: 20250036582Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.Type: ApplicationFiled: October 16, 2024Publication date: January 30, 2025Inventors: Sudeep BHOJA, Siddharth SHETH
-
Patent number: 12147359Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.Type: GrantFiled: January 25, 2024Date of Patent: November 19, 2024Assignee: d-MATRIX CORPORATIONInventors: Sudeep Bhoja, Siddharth Sheth
-
Publication number: 20240241841Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.Type: ApplicationFiled: October 24, 2023Publication date: July 18, 2024Inventors: Sudeep BHOJA, Siddharth SHETH
-
Publication number: 20240160586Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.Type: ApplicationFiled: January 25, 2024Publication date: May 16, 2024Inventors: Sudeep BHOJA, Siddharth SHETH
-
Patent number: 11886359Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.Type: GrantFiled: October 17, 2022Date of Patent: January 30, 2024Assignee: d-MATRIX CORPORATIONInventors: Sudeep Bhoja, Siddharth Sheth
-
Patent number: 11847072Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.Type: GrantFiled: November 30, 2021Date of Patent: December 19, 2023Assignee: d-MATRIX CORPORATIONInventors: Sudeep Bhoja, Siddharth Sheth
-
Publication number: 20230169021Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.Type: ApplicationFiled: November 30, 2021Publication date: June 1, 2023Inventors: Sudeep BHOJA, Siddharth SHETH
-
Publication number: 20230168899Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications, including generative AI. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions.Type: ApplicationFiled: November 23, 2022Publication date: June 1, 2023Inventors: Sudeep BHOJA, Siddharth SHETH
-
Patent number: 10409758Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.Type: GrantFiled: May 3, 2019Date of Patent: September 10, 2019Assignee: INPHI CORPORATIONInventors: Siddharth Sheth, Radhakrishnan L. Nagarajan
-
Publication number: 20190258598Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.Type: ApplicationFiled: May 3, 2019Publication date: August 22, 2019Inventors: Siddharth SHETH, Radhakrishnan L. NAGARAJAN
-
Patent number: 10289595Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.Type: GrantFiled: July 20, 2018Date of Patent: May 14, 2019Assignee: INPHI CORPORATIONInventors: Siddharth Sheth, Radhakrishnan L. Nagarajan
-
Publication number: 20180329852Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.Type: ApplicationFiled: July 20, 2018Publication date: November 15, 2018Inventors: Siddharth SHETH, Radhakrishnan L. NAGARAJAN
-
Patent number: 10120826Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.Type: GrantFiled: November 13, 2017Date of Patent: November 6, 2018Assignee: INPHI CORPORATIONInventors: Siddharth Sheth, Radhakrishnan L. Nagarajan
-
Patent number: 10120825Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.Type: GrantFiled: October 11, 2017Date of Patent: November 6, 2018Assignee: INPHI CORPORATIONInventors: Siddharth Sheth, Radhakrishnan L. Nagarajan
-
Patent number: 10055375Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.Type: GrantFiled: October 11, 2017Date of Patent: August 21, 2018Assignee: INPHI CORPORATIONInventors: Siddharth Sheth, Radhakrishnan L. Nagarajan
-
Publication number: 20180067888Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.Type: ApplicationFiled: November 13, 2017Publication date: March 8, 2018Inventors: Siddharth SHETH, Radhakrishnan L. NAGARAJAN