Abstract: In some implementations, a switch comprises a plurality of UALink-based ports, each port supporting a plurality of virtual channels, and a traffic classification circuit configured to determine whether a request is a collective request or a unicast request based on a command type field, and assign the request to a corresponding virtual channel. In some implementations, a switch comprises a credit monitoring circuit configured to monitor credit utilization for each virtual channel on each port, and a virtual channel allocation circuit configured to adjust virtual channel assignment criteria when credit utilization of a virtual channel indicates congestion. In some implementations, a method comprises monitoring credit utilization for virtual channels, detecting congestion, adjusting virtual channel assignment criteria, and assigning subsequent traffic to a different virtual channel based on the adjusted criteria.

Abstract: Implementations for harvesting underutilized memory at compute entities to serve as storage substrate for an NVMe device. A logical device exposes an NVMe device to a first entity, the NVMe device comprising an NVMe namespace exposing a Logical Block Address (LBA) space. DRAM at a second entity and at a third entity is coupled via CXL to a memory back-end of the logical device. A memory traffic analyzer receives telemetry of accesses to the DRAM, identifies memory regions that are underutilized, and grades them by actual use. The memory back-end maps portions of the LBA space to identified and graded memory regions based at least in part on the grading, such that storage operations of the NVMe device are mapped at least in part to underutilized DRAM harvested across entities.

Abstract: Implementations for characterizing memory access patterns in systems that handle CXL memory transactions. A memory traffic analyzer of the system receives, from CXL.mem handling logic of the system, digests of CXL.mem requests handled by the CXL.mem handling logic. The digests comprise subsets of fields of the CXL.mem requests, the subsets comprising at least addresses carried by the CXL.mem requests or identifiers of consumers associated with the CXL.mem requests. The memory traffic analyzer characterizes, based on the digests, memory access patterns to memory regions reachable through the CXL.mem handling logic. The system identifies, based on the characterization, candidate memory addresses for promotion or demotion among memory resources reachable through the CXL.mem handling logic.

Abstract: Some implementations address the memory bandwidth and capacity pressure of memory-intensive workloads, such as AI and machine-learning acceleration and in-memory analytics, executing on multi-processor systems with disaggregated and pooled memory. A processor interleaves a single memory region across local memory of the processor, remote memory of a second processor reached over an inter-socket link (ISoL), and a CXL-attached memory reached over a CXL port as byte-addressable memory, at a granularity of an integer number of cache lines per allocation step. An interleaving controller distributes consecutive allocation steps to the local memory, the remote memory, and the CXL-attached memory according to a ratio of cache lines drawn from each, so that the workload draws bandwidth and capacity from the three memories together. The ratio may be adjusted during operation based on a measured memory-bandwidth metric, and the CXL-attached memory may comprise multiple CXL memory devices.

Abstract: Modern AI and HPC systems distribute workloads across many GPUs and accelerators, where moving data between accelerators quickly and without saturating external interconnect bandwidth is a central scaling challenge. Some implementations disclose a processor that carries two CXL ports operating according to the same CXL protocol, comprising at least one of CXL.mem or CXL.cache, and that exposes a shared memory region located within the processor's own local DRAM. A first external entity reaches the shared memory region through a first CXL port, and a second external entity reaches it through a second CXL port, so the two entities communicate with each other through the host-local shared memory region. The processor may serve as a coherency agent, may expose the region as host-managed device memory, and may integrate the ports with an on-package coherent interconnect for scale-up accelerator clusters.

Abstract: In some implementations, a system comprises at least a first, a second, and a third NVMe storage element, and memory resources distributed across a CXL fabric and contributed by at least a first entity coupled via a first CXL link and a second entity coupled via a second CXL link. A memory traffic analyzer receives telemetry of memory accesses to the memory resources and grades the memory resources, based on the telemetry, into a higher-grade subset and a lower-grade subset. A resource composer couples the higher-grade subset as cache to the first NVMe storage element, and couples the lower-grade subset as cache to the second NVMe storage element. The memory resources may be drawn from underutilized DRAM at hosts and from CXL memory expanders, supporting differentiated cache performance among the NVMe storage elements.

Abstract: Some implementations address the mapping of shared data in systems where a CPU and multiple GPUs jointly operate on common data over a coherent interconnect. As machine learning and other accelerated workloads scale across multiple GPUs, a CPU and the GPUs may coherently share a shared memory region over CXL.cache, with each accessing the shared memory region as a separate consumer. A memory tiering engine determines mapping of portions of the shared memory region across at least two memory tiers and performs per-consumer attribution that distinguishes accesses originated by the CPU, the first GPU, and the second GPU as separate consumers. Mapping of a portion in a memory tier is based on which consumer the per-consumer attribution identifies as originating accesses to the portion, so that portions may be mapped to the consumer that accesses them across multi-GPU, memory-tiered datacenter and high-performance-computing deployments.

Abstract: Some implementations address memory-management challenges of modern AI and high-performance-computing (HPC) platforms, in which a processor simultaneously expands its memory over CXL.mem to a CXL memory device, and shares memory coherently over CXL.cache with an accelerator such as a GPU or AI accelerator. As memory pooling and heterogeneous memory tiering become central to datacenter scale-out, a single memory region may be accessed over either sub-protocol, leaving conventional per-domain monitoring unable to form a unified view. A memory traffic analyzer receives telemetry of CXL.mem traffic and telemetry of CXL.cache traffic and produces a combined memory-access tracking output that represents, for memory regions, an access intensity attributed to the CXL.mem traffic and an access intensity attributed to the CXL.cache traffic. The processor may use the combined output to place frequently-accessed memory regions in lower-latency memory, improving performance across disaggregated and coherent memory tiers.

Abstract: Implementations for producing a unified representation of memory access tracking across a CXL fabric. A first analyzer associated with a first portion of the CXL fabric produces first telemetry of memory accesses at a first resolution comprising a multi-level heat-map of accesses to memory regions in the first portion. A second analyzer associated with a second portion of the CXL fabric produces second telemetry of memory accesses at a second resolution, lower than the first resolution, comprising a binary hot-or-cold classification of memory regions in the second portion. A computer receives the first telemetry and the second telemetry and produces the representation of memory access tracking across the CXL fabric comprising values calculated based on the first telemetry and from the second telemetry, the representation usable by a memory tiering algorithm to determine mapping of memory regions across the CXL fabric.

Abstract: Some modern datacenters running AI/ML training, in-memory analytics, and HPC workloads demand memory capacity that scales independently of compute, driving rack-level memory disaggregation and pooling. Various implementations herein disclose address translation positioned between an entity and a memory pool that aggregates memory contributed by two or more sources across a rack. An apparatus, processor, or method receives CXL.mem requests carrying physical addresses within a host physical address space utilized by the entity, translates those addresses into addresses within the address space utilized by the memory pool, and forwards memory access requests carrying the translated addresses toward the pool.

Abstract: Implementations for exposing, to an entity, a remote memory address region backed by DRAM of multiple back-end entities, with mapping of portions of the remote memory address region managed using telemetry of memory accesses. A memory pool or a processor is coupled to the entity via a CXL link configured to support CXL.mem. First and second back-end entities, each having processing cores and locally-attached DRAM, are coupled to the memory pool via respective back-end CXL links. A resource composer multiplexes the first DRAM and the second DRAM onto the CXL link such that different portions of the remote memory address region are mapped to different back-end DRAMs. And a memory traffic analyzer provides telemetry of memory accesses to the resource composer for mapping management.

Abstract: Implementations for distributed memory provisioning to virtualized software entities over CXL. A memory pool comprising first DRAM is coupled via CXL to a first entity executing a virtualized software entity, to a second entity comprising second DRAM, and to a third entity comprising third DRAM. The memory pool exposes a remote memory address region of the virtualized software entity to the first entity. A memory tiering engine maps distinct sub-regions of the remote memory address region to distinct ones of the first, second, and third DRAM based on a memory utilization characteristic of the virtualized software entity, such that at a given time during execution the at least three sub-regions are mapped to the at least three DRAMs concurrently.

Abstract: Some modern AI and LLM workloads need memory that exceeds the capacity attached to a single processor. Implementations enable an external entity, such as another host or an AI accelerator, to access memory belonging to a CPU through a CXL port. CXL requests received from the entity carry physical addresses, which are translated to physical addresses within the CPU's own physical address space, enabling the entity to access the CPU's memory through the port. The translation may be performed by the CPU itself, by an RPU coupled to the CPU's interconnect, or by a designated CPU in a multi-CPU cluster, in which case the entity accesses a memory pool spanning the cluster. The CPU may also consume memory from a CXL device while exposing both its local memory and the CXL device's memory to the entity.

Abstract: A centralized controller classifies attributes of devices comprising an Ultra Accelerator Link (UALink)-based ports into classifications including identity attributes, configuration attributes representing intended state, operational attributes representing current observed state, and telemetry attributes. The centralized controller communicates the attributes via multiple management protocols over an out-of-band management network, each protocol mapping the classifications to protocol-specific data representations. The centralized controller compares configuration attributes with corresponding operational attributes to detect discrepancies between intended state and current observed state. In some implementations, each device maintains a corresponding operational attribute for each configuration attribute, enabling systematic detection across all configurable parameters.

Abstract: In an Ultra Accelerator Link (UALink)-based network, a centralized controller communicates with a first management agent associated with a switch platform via a first management protocol and with a second management agent associated with a system node via a second management protocol different from the first management protocol. The first management agent communicates with switches via a hardware abstraction interface. The centralized controller transmits configuration to, and receives operational state from, the management agents via the respective management protocols. In some implementations, a management agent presents a resource tree describing accelerators and their ports using identifiers, wherein the resource tree remains stable across disconnections, restarts, or configuration changes, enabling the centralized controller to resume management operations upon reconnection using the same identifiers.

Abstract: Enabling memory pooling and disaggregation across hosts, GPUs, and accelerators in scale-up AI fabrics, expanding usable memory capacity beyond the limits of a single server. Implementations incorporating a resource provisioning unit (RPU) coupled to a coherent interconnect of a processor, the coherent interconnect coupled to memory accessible utilizing a physical address space. An MMU maps virtual addresses utilized by an operating system to physical addresses within the physical address space. The RPU comprises a port configured to exchange messages with an entity based on CXL.mem. The RPU is configured to perform physical address translations from physical addresses carried by requests received via the port to physical addresses within the physical address space, whereby the entity is enabled to access the memory via the port. The embodiments may support memory-intensive AI, machine learning, large language model, database, and/or cloud workloads.

Abstract: Embodiments for communicating using a switch that includes first and second dies. The first die includes first and second sets of Compute Express Link (CXL) upstream switch ports (USPs) and downstream switch ports (DSPs), and routing hardware. The second die includes CXL Root Ports (RPs) coupled to the second set of USPs, CXL Endpoints (EPs) coupled to the second set of DSPs, and DRAM memory channels coupled to DRAM memory modules. A computer monitors data traffic through the USPs and DSPs, stores frequently accessed data in the DRAM memory modules, and utilizes the DRAM memory modules as a cache. Optionally, the DRAM memory modules includes at least 4 GB of memory capacity organized in multiple channels enabling concurrent cache operations.

Abstract: Modern processors maintain their own physical address space, mapping processing cores, memory channels, and memory-mapped devices onto a coherent internal fabric. External hosts communicating via CXL, however, operate within their own distinct physical address space, creating an addressing gap that conventionally prevents a host from accessing resources within a processor's internal fabric. Some implementations describe a processor that bridges this gap. The processor includes processing cores coupled via a coherent interconnect, memory channels for accessing external memory, and MMUs for translating virtual addresses to physical addresses within the processor's internal address space.

Abstract: In an Ultra Accelerator Link (UALink)-based network, a centralized controller receives identity information and operational state of switches and ports from management agents, assigns a unique accelerator identifier to each accelerator, generates forwarding entries based on the received information wherein each forwarding entry maps a destination accelerator identifier to an egress port, and distributes the forwarding entries to the switches via the management agents. In some implementations, the centralized controller manages virtual partitions (such as virtual pods) by generating forwarding entries that restrict traffic to a designated subset of accelerators, distributing the forwarding entries, and after installation, enabling switch ports associated with the designated subset to carry transactions. The enabling may be applied to the switch ports associated with the designated subset.

Abstract: Implementations for hierarchical error recovery with scoped containment in an accelerator interconnect network. In some implementations, a system comprises accelerators coupled to switches via protocol-level interface links, each switch comprising one or more logical switches. Upon a link failure on a first logical switch, a centralized controller removes forwarding entries on the first logical switch from a virtual partition while maintaining forwarding entries on a second logical switch, enabling continued operation at reduced bandwidth. In some implementations, the centralized controller applies different recovery strategies based on the failed component type: for link failures, removing forwarding entries on the impacted logical switch; for accelerator failures, removing forwarding entries to the failed accelerator across all logical switches; and for switch failures, re-configuring the switch and re-installing forwarding entries after restart.