Abstract: A system for one-sided read remote memory access is disclosed. In some embodiments, the system is configured to receive, at a responder SFA, a first packet comprising a read request to read a remote memory of a second host from a first host, wherein a payload of the first packet is mapped to be a transmit header queue (TxHQ) entry (TxHQE), and the TxHQE includes a pointer to a memory map; separate, the received packet into portions including a upper level protocol (ULP) portion, the ULP portion being the TxHQE; create a ULP header queue for the TxHQE; generate a read response based on mapping the ULP header queue into hardware as the TxHQ, wherein the TxHQE includes a pointer to data from a valid memory region of the second host identified by the memory mapping; and transmit a read response packet with the data identified by the pointer using the TxHQ to the first host.

Abstract: Methods, systems, and apparatus including a special purpose hardware chip for training neural networks are described. The special-purpose hardware chip may include a scalar processor configured to control computational operation of the special-purpose hardware chip. The chip may also include a vector processor configured to have a 2-dimensional array of vector processing units which all execute the same instruction in a single instruction, multiple-data manner and communicate with each other through load and store instructions of the vector processor. The chip may additionally include a matrix multiply unit that is coupled to the vector processor configured to multiply at least one two-dimensional matrix with a second one-dimensional vector or two-dimensional matrix in order to obtain a multiplication result.

Abstract: A system for sharing peripheral component interconnect express (PCIe) devices across multiple host servers is disclosed. In some embodiments, a switch includes a plurality of hosts associated with a plurality of hierarchies, one or more endpoints associated with one or more of the plurality of hierarchies, and a switch communicatively connectable to the plurality of hosts and the one or more endpoints. The switch is configured to: receive a transaction layer packet (TLP); determine a policy group identifier based on parsing and processing the TLP; perform packet forward matching based on the policy group identifier and destination fields of the TLP; based on whether the TLP is communicated between the hosts and endpoints in different hierarchies of the plurality of hierarchies, determine whether to edit the TLP using one or more rewrite rules; and forward the TLP to an appropriate destination link.

Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.

Abstract: Methods, systems, and apparatus including a special purpose hardware chip for training neural networks are described. The special-purpose hardware chip may include a scalar processor configured to control computational operation of the special-purpose hardware chip. The chip may also include a vector processor configured to have a 2-dimensional array of vector processing units which all execute the same instruction in a single instruction, multiple-data manner and communicate with each other through load and store instructions of the vector processor. The chip may additionally include a matrix multiply unit that is coupled to the vector processor configured to multiply at least one two-dimensional matrix with a second one-dimensional vector or two-dimensional matrix in order to obtain a multiplication result.

Abstract: A system and method for producing and transmitting reduced data is disclosed. In some embodiments, the system comprises an ACF and reduction processors. The reduction processors are configured to perform a data reduction process. The ACF is configured to obtain access to input data from multiple flows, the input data identified by SGLEs included in input SGLs, and move a portion of the input data from each flow of the multiple flows to a respective reduction processor of the multiple reduction processors, such that each reduction processor receives a respective portion of the input data from each flow. The ACF is further configured to obtain access to reduced data produced from the input data using the data reduction process performed by the multiple reduction processors and move the reduced data to one or more destinations, where the reduced data identified by an output SGL.

Abstract: A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.

Abstract: A system for sharing peripheral component interconnect express (PCIe) devices across multiple host servers is disclosed. In some embodiments, a switch includes a plurality of hosts associated with a plurality of hierarchies, one or more endpoints associated with one or more of the plurality of hierarchies, and a switch communicatively connectable to the plurality of hosts and the one or more endpoints. The switch is configured to: receive a transaction layer packet (TLP); determine a policy group identifier based on parsing and processing the TLP; perform packet forward matching based on the policy group identifier and destination fields of the TLP; based on whether the TLP is communicated between the hosts and endpoints in different hierarchies of the plurality of hierarchies, determine whether to edit the TLP using one or more rewrite rules; and forward the TLP to an appropriate destination link.

Abstract: A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.

Abstract: A system for sharing peripheral component interconnect express (PCIe) devices across multiple host servers is disclosed. In some embodiments, a switch includes a plurality of hosts associated with a plurality of hierarchies, one or more endpoints associated with one or more of the plurality of hierarchies, and a switch communicatively connectable to the plurality of hosts and the one or more endpoints. The switch is configured to: receive a transaction layer packet (TLP); determine a policy group identifier based on parsing and processing the TLP; perform packet forward matching based on the policy group identifier and destination fields of the TLP; based on whether the TLP is communicated between the hosts and endpoints in different hierarchies of the plurality of hierarchies, determine whether to edit the TLP using one or more rewrite rules; and forward the TLP to an appropriate destination link.

Abstract: Methods, systems, and apparatus, including computer-readable media, are described for a hardware circuit configured to implement a neural network. The circuit includes a first memory, respective first and second processor cores, and a shared memory. The first memory provides data for performing computations to generate an output for a neural network layer. Each of the first and second cores include a vector memory for storing vector values derived from the data provided by the first memory. The shared memory is disposed generally intermediate the first memory and at least one core and includes: i) a direct memory access (DMA) data path configured to route data between the shared memory and the respective vector memories of the first and second cores and ii) a load-store data path configured to route data between the shared memory and respective vector registers of the first and second cores.

Abstract: A system for providing memory access is disclosed. In some embodiments, the system is configured to receive at a source server fabric adapter (SFA), from a server, a memory access request comprising a virtual memory address; using associative mapping, determining whether the virtual address corresponds to a source-local memory associated with the source SFA or to a remote memory. If the virtual address corresponds to the source-local memory, the virtual memory address is translated, at the source SFA, into a physical memory address of the source-local memory. If the virtual address corresponds to the remote memory, a request message is synthesized, and the synthesized request message is transmitted to the destination SFA using a network protocol.

Abstract: A system and method for producing and transmitting reduced data is disclosed. In some embodiments, the system comprises an ACF and reduction processors. The reduction processors are configured to perform a data reduction process. The ACF is configured to obtain access to input data from multiple flows, the input data identified by SGLEs included in input SGLs, and move a portion of the input data from each flow of the multiple flows to a respective reduction processor of the multiple reduction processors, such that each reduction processor receives a respective portion of the input data from each flow. The ACF is further configured to obtain access to reduced data produced from the input data using the data reduction process performed by the multiple reduction processors and move the reduced data to one or more destinations, where the reduced data identified by an output SGL.

Abstract: DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each memory address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.

Abstract: A system for providing memory access is disclosed. In some embodiments, the system is configured to receive at a source server fabric adapter (SFA), from a server, a memory access request comprising a virtual memory address; using associative mapping, determining whether the virtual address corresponds to a source-local memory associated with the source SFA or to a remote memory. If the virtual address corresponds to the source-local memory, the virtual memory address is translated, at the source SFA, into a physical memory address of the source-local memory. If the virtual address corresponds to the remote memory, a request message is synthesized, and the synthesized request message is transmitted to the destination SFA using a network protocol.

Abstract: A system and method for producing and transmitting reduced data is disclosed. In some embodiments, the system comprises an ACF and reduction processors. The reduction processors are configured to perform a data reduction process. The ACF is configured to obtain access to input data from multiple flows, the input data identified by SGLEs included in input SGLs, and move a portion of the input data from each flow of the multiple flows to a respective reduction processor of the multiple reduction processors, such that each reduction processor receives a respective portion of the input data from each flow. The ACF is further configured to obtain access to reduced data produced from the input data using the data reduction process performed by the multiple reduction processors and move the reduced data to one or more destinations, where the reduced data identified by an output SGL.

Abstract: A system for sharing peripheral component interconnect express (PCIe) devices across multiple host servers is disclosed. In some embodiments, a switch includes a plurality of hosts associated with a plurality of hierarchies, one or more endpoints associated with one or more of the plurality of hierarchies, and a switch communicatively connectable to the plurality of hosts and the one or more endpoints. The switch is configured to: receive a transaction layer packet (TLP); determine a policy group identifier based on parsing and processing the TLP; perform packet forward matching based on the policy group identifier and destination fields of the TLP; based on whether the TLP is communicated between the hosts and endpoints in different hierarchies of the plurality of hierarchies, determine whether to edit the TLP using one or more rewrite rules; and forward the TLP to an appropriate destination link.

Abstract: A system for tracking and moving pages within a memory hierarchy is disclosed. In some embodiments, the system comprises a memory hierarchy having low-tier memory and high-tier memory. The system comprises an input/output (I/O) port configured to map into the low-tier memory. The system comprises a central processing unit (CPU) associated with the high-tier memory and configured to make one or more page requests for accessing a page stored in the low-tier memory via the I/O port. The system also comprises a page tracker configured to determine a count of the one or more page requests. The system further comprises a data movement engine configured to move content of the page from the low-tier memory to the high-tier memory when the count of the one or more page requests exceeds a predetermined threshold.

Abstract: A system for using staging buffers in broadcast or multicast operations is disclosed. In some embodiments, the system comprises a server fabric adapter (SFA) communicatively coupled to a plurality of accelerators. The system is configured to provide a memory tier that is accessed by the plurality of accelerators; receive data in a send queue of the memory tier; establish an association between buffers of the send queue and one or more receive queues based on a pattern of sharing defined by one or more of the plurality of accelerators; and transmit the data to the one or more accelerators by sending the data from the send queue to the one or more receive queues based on the association.

Abstract: A system for providing memory access is disclosed. In some embodiments, the system is configured to receive at a source server fabric adapter (SFA), from a server, a memory access request comprising a virtual memory address; using associative mapping, determining whether the virtual address corresponds to a source-local memory associated with the source SFA or to a remote memory. If the virtual address corresponds to the source-local memory, the virtual memory address is translated, at the source SFA, into a physical memory address of the source-local memory. If the virtual address corresponds to the remote memory, a request message is synthesized, and the synthesized request message is transmitted to the destination SFA using a network protocol.