Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications, including generative AI. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions.

Abstract: A server system using switch linking between groups of AI processing systems. The system includes at least a first central processing unit (CPU) coupled to a first group of AI processing systems via a first switch and a second CPU coupled to a second group of AI processing systems via a second switch. Each of the CPUs is also coupled to a separate group of memory devices and a communication link is configured between the first switch and the second switch to communicate information between the first group of AI processing systems and the second group of AI processing systems. Each of these AI processing system groups include a plurality of AI processing modules, and each of these modules include a plurality of chiplet devices. Each chiplet device is configured with a plurality of in-memory compute (IMC) devices for processing neural network model workloads.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications, including generative AI. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.