Abstract: Modern datacenters require efficient mechanisms for memory resource sharing between accelerators and host processors to support AI/ML workloads, HPC applications, and distributed computing environments. Embodiments herein disclose systems incorporating RPUs that enable entities to access host memory through UALink-based interconnects. The processor utilizes a coherent interconnect coupling processing cores to memory controllers, with an MMU mapping virtual addresses to physical addresses within the processor's physical address space. The RPU performs hardware-accelerated physical address translations between UALink-associated addresses and the processor's physical address space, enabling entities to access memory via the UALink port, coherent interconnect, and memory controllers.

Abstract: Datacenter workloads demand flexible memory architectures spanning from rack-level to pod-scale deployments. Embodiments herein disclose systems enabling CXL memory semantics over physical layers based on IEEE 802.3 PMA, facilitating memory disaggregation across datacenter fabric infrastructures. The embodiments comprise processing cores with coherent interconnects, MMUs for address translation, and memory channels supporting substantial memory capacities. Resource Provisioning Units (RPUs) translate between CXL data, optionally encapsulated, transmitted via physical layers based on IEEE 802.3 PMA, and CXL requests, enabling external entities to access memory across different physical address spaces.

Abstract: Modern datacenters require efficient mechanisms for memory resource sharing and utilization across distributed computing environments. Some of the disclosed embodiments introduce systems and methods incorporating a Resource Provisioning Unit (RPU) that performs host-to-host physical address translations, enabling external hosts to access memory resources utilizing CXL protocols. The system includes a processor coupled to DRAM, an MMU for virtual-to-physical address mapping, and a CXL device for host communication. The RPU enables hosts to access the DRAM utilizing messages conforming to CXL protocols, including CXL.mem with Host-managed Device Memory (HDM) regions and CXL.io with Transaction Layer Packets. Some embodiments support multiple hosts and CXL memory expanders utilizing additional CXL devices and root ports.

Abstract: Large-scale compute environments for Artificial Intelligence (AI) and High-Performance Computing (HPC) may utilize memory fabric infrastructures to achieve higher workload performance compared to conventional networking-only infrastructures. However, the role of memory fabrics extends beyond high-end use cases into general compute scenarios in public cloud, private cloud, and hybrid enterprise environments, where Memory-as-a-Service (Memory-aaS) augments and complements the portfolio of services provided to tenants in the datacenter. Embodiments herein disclose a streamlined CXL memory fabric utilizing a Resource Provisioning Unit (RPU), enabling lightweight, scalable provisioning of memory to CPUs, GPUs, and accelerators via CXL.io, CXL.mem, and CXL.cache. These embodiments support translation between asymmetric and symmetric memory transactions and operate in both switch-enabled CXL environments, such as CXL 2.0 and CXL 3.x, and non-switched CXL environments, such as CXL 1.1.

Abstract: Modern datacenters require flexible interconnect solutions that bridge diverse protocol domains while maintaining compatibility with existing infrastructure. Embodiments herein disclose semiconductor devices that implement protocol translations and physical address translations within IC packages conforming to the PCIe Retimer Supplemental Features and Standard BGA Footprint Specification. The devices comprise first and second interfaces communicating according to first and second protocols respectively, with an embedded computer that extracts physical addresses from messages received via the first interface, translates these addresses, and generates messages carrying the translated addresses for transmission via the second interface. This retimer-compatible form factor essentially enables drop-in deployment within existing PCIe and cabling infrastructures while providing protocol bridging and address translation capabilities.

Abstract: Datacenter workloads demand flexible memory architectures spanning from rack-level to pod-scale deployments. Embodiments herein disclose systems enabling CXL memory semantics over physical layers based on IEEE 802.3 PMA, facilitating memory disaggregation across datacenter fabric infrastructures. The embodiments comprise processing cores with coherent interconnects, MMUs for address translation, and memory channels supporting substantial memory capacities. Resource Provisioning Units (RPUs) translate between CXL data, optionally encapsulated, transmitted via physical layers based on IEEE 802.3 PMA, and CXL requests, enabling external entities to access memory across different physical address spaces.

Abstract: Modem datacenters require flexible interconnect solutions that bridge diverse protocol domains while maintaining compatibility with existing infrastructure. Embodiments herein disclose semiconductor devices that implement protocol translations and physical address translations within IC packages conforming to the PCIe Retimer Supplemental Features and Standard BGA Footprint Specification. The devices comprise first and second interfaces communicating according to first and second protocols respectively, with an embedded computer that extracts physical addresses from messages received via the first interface, translates these addresses, and generates messages carrying the translated addresses for transmission via the second interface. This retimer-compatible form factor essentially enables drop-in deployment within existing PCIe and cabling infrastructures while providing protocol bridging and address translation capabilities.

Abstract: Modern AI/ML workloads demand efficient integration of GPUs and accelerators with ARM-based server architectures in datacenters and edge computing environments. Embodiments herein disclose systems incorporating RPUs that enable NVLink-connected accelerators to access memory resources within ARM CHI-based coherent interconnect fabrics. One embodiment comprises a CHI-based coherent interconnect with interconnect components routing CHI messages between processing cores and memory controllers supporting substantial memory capacities. The RPU bridges NVLink and CHI protocols, while performing protocol translation between NVLink and CHI messaging, enabling GPUs and accelerators to access system memory through the coherent fabric. Multiple RPUs optionally support scalable configurations with multiple NVLink-connected devices accessing shared memory resources.

Abstract: Modern datacenters require efficient mechanisms for memory resource sharing and utilization across distributed computing environments. Some of the disclosed embodiments introduce systems and methods incorporating a Resource Provisioning Unit (RPU) that performs host-to-host physical address translations, enabling external hosts to access memory resources utilizing CXL protocols. The system includes a processor coupled to DRAM, an MMU for virtual-to-physical address mapping, and a CXL device for host communication. The RPU enables hosts to access the DRAM utilizing messages conforming to CXL protocols, including CXL.mem with Host-managed Device Memory (HDM) regions and CXL.io with Transaction Layer Packets. Some embodiments support multiple hosts and CXL memory expanders utilizing additional CXL devices and root ports.

Abstract: Modem datacenters require efficient mechanisms for memory resource sharing between accelerators and host processors to support AI/ML workloads, HPC applications, and distributed computing environments. Embodiments herein disclose systems incorporating RPUs that enable entities to access host memory through UALink-based interconnects. The processor utilizes a coherent interconnect coupling processing cores to memory controllers, with an MMU mapping virtual addresses to physical addresses within the processor's physical address space. The RPU performs hardware-accelerated physical address translations between UALink-associated addresses and the processor's physical address space, enabling entities to access memory via the UALink port, coherent interconnect, and memory controllers.

Abstract: Modem datacenters require efficient mechanisms for memory resource sharing across heterogeneous computing environments to support AI workloads, LLM inference, and high-performance computing applications. Some of the disclosed embodiments introduce systems and methods incorporating an RPU that performs address translations between NVLink-based protocols and host physical address spaces, enabling GPUs, accelerators, and other NVLink-capable devices to access host memory resources. The system includes processing cores with MMUs, a coherent interconnect coupling the cores to memory controllers supporting more than 64 GB of memory, and an RPU with an NVLink-based interface. The RPU translates physical addresses associated with the NVLink-based protocol to physical addresses within the host's physical address space, enabling entities to access host memory via the NVLink-based interface.

Abstract: Protocol translation embodiments enabling Memory as a Service (MaaS) and symmetric memory access across heterogeneous computing infrastructures comprising CPUs, GPUs, accelerators, fabric attached memory, and interconnects such as CXL, UALink, NVLink, and Ethernet. The embodiments facilitate NTB or Host-to-Host memory borrowing and symmetric memory flows by translating between CXL and non-CXL protocols, enabling seamless memory sharing between CXL-native and legacy systems, such as PCIe. The embodiments receive non-CXL.cache requests from a first entity, translate them to CXL.cache Device-to-Host Requests for transmission to a second entity, and conversely translate CXL.mem requests to non-CXL.mem requests. This protocol translation enables memory pooling across diverse data center architectures where CPUs, GPUs, and accelerators may share memory resources through Host-to-Host memory borrowing. The embodiments support composable infrastructure for AI/ML workloads, cloud-native applications.

Abstract: Embodiments of a switch enabling multiple hosts to concurrently access and share resources of a Single Logical Device (SLD). The switch includes first and second upstream switch ports (USPs) coupled to first and second hosts, and a downstream switch port (DSP) coupled to the SLD. The first USP communicates with the first host according to a first Compute Express Link (CXL) protocol, the second USP communicates with the second host according to a second CXL protocol, and the DSP communicates with a first CXL SLD component according to a third CXL protocol. A Resource Provisioning Unit (RPU) terminates the first, second, and third CXL protocols, and exposes second and third virtualized SLDs, which utilize resources of the first SLD component, to the first and second hosts, respectively. Optionally, the first, second, and third CXL protocols are CXL.mem protocols.

Abstract: A Memory as a Service (MaaS) system enabling on-demand memory provisioning across distributed computing infrastructure comprising CPUs, GPUs, and accelerators. The system comprises a first host, a second host, and a computer interconnected via Compute Express Link (CXL). Each host runs packaged computing environments (PCEs) comprising containers or virtual machines. The computer monitors page table values from processes running in PCEs, identifies underutilized DRAM regions that remain unused for predetermined durations, and dynamically reallocates these memory resources as a service to memory-demanding processes on different hosts. This CXL-enabled MaaS architecture transforms static memory allocation into a flexible, consumption-based model, reducing data center memory waste while enabling real-time memory elasticity for cloud-native applications, AI/ML workloads, and multi-tenant environments.

Abstract: Embodiments for communicating using a switch having multiple sets of switch ports, which can generate abstractions for compute nodes and accelerators in a switched Compute Express Link (CXL) interconnect, enabling any-to-any connectivity between hosts and devices. A first set of switch ports couples hosts and/or devices with a first virtual to physical binding. A second set couples a Resource Provisioning Unit (RPU) with the first virtual to physical binding. The first binding binds at least some of the first and/or second sets of switch ports to a first set of Virtual CXL Switches (VCSs). A third set couples the RPU to a second virtual to physical binding. A fourth set couples additional hosts and/or devices with the second virtual to physical binding. And the second virtual to physical binding binds at least some of the third and/or fourth sets of switch ports to a second set of VCSs.

Abstract: This invention pertains to a system optimized for reutilizing allocated underutilized or unused allocated DRAM, comprising a first host, a second host, and a resource composer, interconnected via CXL. Both hosts run packaged computing environments (PCEs), which may be containers or virtual machines, and are equipped to handle respective processes, P1 and P2. The resource composer is tasked with receiving data related to P1's memory usage from a kernel module on the first host, identifying underutilized DRAM mapped to P1, and subsequently remapping it to P2's address space on the second host. This process involves the use of CXL.mem commands, which are then translated into appropriate CXL.cache or CXL.io commands for DRAM access based on the mapping.

Abstract: Embodiments for communicating using a switch configured to establish multiple types of communication routes. First and second upstream switch ports (USPs) communicate with first and second hosts according to first and second Compute Express Link (CXL) protocols, respectively. A downstream switch port (DSP) communicates with a device according to a third CXL protocol. The switch couples the first USP to the DSP via a first route traversing a single Virtual CXL Switch (VCS), and couples the first USP to the second USP via a second route traversing two VCSs. Optionally, the switch includes a Resource Provisioning Unit (RPU) coupling the two VCSs of the second route, terminating the first and second CXL protocols, and translating between CXL messages conforming to the first and second CXL protocols.

Abstract: Embodiments for communicating using a switch. A first Virtual CXL Switch (VCS) routes messages, conforming to a first CXL protocol, from a first switch port to a Resource Provisioning Unit (RPU). A second VCS routes messages, conforming to a second CXL protocol, from the RPU to the second switch port. The RPU terminates the first and second CXL protocols and translates at least some of the messages conforming to the first CXL protocol to at least some of the messages conforming to the second CXL protocol. Optionally, the first CXL protocol comprises CXL.mem, the second CXL protocol comprises CXL.cache, and the RPU manages snoop and invalidation message flows and maintains transaction order requirements.

Abstract: Embodiments for communicating between a Compute Express Link (CXL) host and a CXL device. A CXL Endpoint receives, from the host, first messages conforming to a first CXL.io protocol and second messages conforming to a first non-CXL.io protocol. A CXL Root Port receives, from the device, third messages conforming to a second CXL.io protocol and fourth messages conforming to a second non-CXL.io protocol. A computer terminates the first CXL.io protocol, processes at least some of the first messages without translating and sending corresponding translated first messages to the device, terminates the first non-CXL.io protocol, translates at least a quarter of the second messages, and makes the translated second messages available to the CXL Root Port for communication with the device. Optionally, each of the first and second non-CXL.io protocols is selected from at least one of CXL.mem or CXL.cache.

Abstract: Embodiments for communicating between Compute Express Link (CXL) hosts. A first CXL Endpoint receives, from a first host, first messages conforming to a first CXL.io protocol and second messages conforming to a first non-CXL.io protocol. A second CXL Endpoint receives, from a second host, third messages conforming to a second CXL.io protocol and fourth messages conforming to a second non-CXL.io protocol. A computer terminates the first CXL.io protocol, processes at least some of the first messages without translating and sending corresponding translated first messages to the second host, terminates the first non-CXL.io protocol, translates at least a quarter of the second messages, and makes the translated second messages available to the second CXL Endpoint for communication with the second host. Optionally, each of the first and second non-CXL.io protocols is selected from at least one of CXL.mem or CXL.cache.