Abstract: Building on the models-on-silicon (model-on-chip or model-on-die) architecture and design, multiple models-on-silicon chips/dies can be arranged in a stacked formation to form a single cube, referred to herein as AI cube. Each of these chips or dies can embed one or more transformer blocks, such as one or more consecutive transformer blocks of a transformer-based neural network. This stacked configuration enables processing of data in a feedforward manner, effectively performing processing for an inference task of a transformer-based neural network, e.g., an entire large language model, within one compact semiconductor integrated circuit package. For example, a 70 billion parameter LLM can be arranged and implemented onto an AI cube, where different groups of transformer blocks are distributed to different chips in the AI cube in a feedforward manner.

Abstract: A memory device may include one or more semiconductor structures having a frontside and a backside, one or more gate electrodes, and metal layers at both the frontside and backside. A frontside metal layer may include metal lines that are used as bit lines of the memory device. A backside metal layer may include metal lines that are used as write bit lines of the memory device. A write bit line at the backside may be parallel to a bit line at the frontside. Another backside metal layer may include metal lines that are used as word lines of the memory device. A word line at the backside may be parallel to a gate electrode. A switch may be between a bit line and a write bit line. The bit line is electrically coupled to the write bit line when the switch is closed.