Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications, including generative AI. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions.

Abstract: An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device. Each slice can include a digital in-memory compute (DIMC) device configured to perform high throughput computations. In particular, the DIMC device can be configured to accelerate the computations of attention functions for transformer-based models (a.k.a. transformers) applied to machine learning applications. A single input multiple data (SIMD) device configured to further process the DIMC output and compute softmax functions for the attention functions. The chiplet can also include die-to-die (D2D) interconnects, a peripheral component interconnect express (PCIe) bus, a dynamic random access memory (DRAM) interface, and a global CPU interface to facilitate communication between the chiplets, memory and a server or host system.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.

Abstract: The present invention provides an integrated system-on-chip device. The device is configured on a single silicon substrate member. The device has a data input/output interface provided on the substrate member. The device has an input/output block provided on the substrate member and coupled to the data input/output interface. The device has a signal processing block provided on the substrate member and coupled to the input/output block. The device has a driver module provided on the substrate member and coupled to the signal processing block. The device further includes a driver interface and coupled to the driver module and configured to be coupled to a silicon photonics device. In an example, a control block is configured to receive and send instruction(s) in a digital format to the communication block and is configured to receive and send signals in an analog format to communicate with the silicon photonics device.