Abstract: According to one embodiment, a computer program product for performing selective multicast delivery includes a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, and where the program instructions are executable by a selector of an intelligent processing unit (IPU) to cause the selector to perform a method comprising identifying, by the selector, an address header appended to an instance of data, comparing, by the selector, address data in the address header to identifier data stored at the selector, and conditionally delivering, by the selector, the instance of data, based on the comparing.

Abstract: Mapping of neural network layers to physical neural cores is provided. In various embodiments, a neural network description describing a plurality of neural network layers is read. Each of the plurality of neural network layers has an associated weight tensor, input tensor, and output tensor. A plurality of precedence relationships among the plurality of neural network layers is determined. The weight tensor, input tensor, and output tensor of each of the plurality of neural network layers are mapped onto an array of neural cores.

Abstract: A method, system and computer program product for providing a guest with access to a virtual storage on a physical storage using a peripheral component interface hub. In one embodiment, the method comprises the guest sending to the peripheral component interface hub a request to access the physical storage, the request including physical addresses of the physical storage, and the peripheral component interface hub sending specified information about the request to a hypervisor. This method further comprises the hypervisor determining whether to grant or to reject the request; and when the hypervisor grants the request, the hypervisor sending a configuration command to the peripheral component interface hub. This command includes a mapping of addresses from the physical storage to addresses from the virtual storage. In an embodiment, the peripheral component interface hub uses this mapping to replace the addresses in the request with translated virtual addresses.

Abstract: Networks of distributed neural cores are provided with hierarchical parallelism. In various embodiments, a plurality of neural cores is provided. Each of the plurality of neural cores comprises a plurality of vector compute units configured to operate in parallel. Each of the plurality of neural cores is configured to compute in parallel output activations by applying its plurality of vector compute units to input activations. Each of the plurality of neural cores is assigned a subset of output activations of a layer of a neural network for computation. Upon receipt of a subset of input activations of the layer of the neural network, each of the plurality of neural cores computes a partial sum for each of its assigned output activations, and computes its assigned output activations from at least the computed partial sums.

Abstract: Instruction distribution in an array of neural network cores is provided. In various embodiments, a neural inference chip is initialized with core microcode. The chip comprises a plurality of neural cores. The core microcode is executable by the neural cores to execute a tensor operation of a neural network. The core microcode is distributed to the plurality of neural cores via an on-chip network. The core microcode is executed synchronously by the plurality of neural cores to compute a neural network layer.

Abstract: Neural network processing hardware using parallel computational architectures with reconfigurable core-level and vector-level parallelism is provided. In various embodiments, a neural network model memory is adapted to store a neural network model comprising a plurality of layers. Each layer has at least one dimension and comprises a plurality of synaptic weights. A plurality of neural cores is provided. Each neural core includes a computation unit and an activation memory. The computation unit is adapted to apply a plurality of synaptic weights to a plurality of input activations to produce a plurality of output activations. The computation unit has a plurality of vector units. The activation memory is adapted to store the input activations and the output activations. The system is adapted to partition the plurality of cores into a plurality of partitions based on dimensions of the layer and the vector units.

Abstract: Hardware neural network processors, are provided. A neural core includes a weight memory, an activation memory, a vector-matrix multiplier, and a vector processor. The vector-matrix multiplier is adapted to receive a weight matrix from the weight memory, receive an activation vector from the activation memory, and compute a vector-matrix multiplication of the weight matrix and the activation vector. The vector processor is adapted to receive one or more input vector from one or more vector source and perform one or more vector functions on the one or more input vector to yield an output vector. In some embodiments a programmable controller is adapted to configure and operate the neural core.

Abstract: Neural inference processors are provided. In various embodiments, a processor includes a plurality of cores. Each core includes a neural computation unit, an activation memory, and a local controller. The neural computation unit is adapted to apply a plurality of synaptic weights to a plurality of input activations to produce a plurality of output activations. The activation memory is adapted to store the input activations and the output activations. The local controller is adapted to load the input activations from the activation memory to the neural computation unit and to store the plurality of output activations from the neural computation unit to the activation memory. The processor includes a neural network model memory adapted to store network parameters, including the plurality of synaptic weights. The processor includes a global scheduler operatively coupled to the plurality of cores, adapted to provide the synaptic weights from the neural network model memory to each core.

Abstract: Neural inference chips and cores adapted to provide time, space, and energy efficient neural inference via parallelism and on-chip memory are provided. In various embodiments, the neural inference chips comprise: a plurality of neural cores interconnected by an on-chip network; a first on-chip memory for storing a neural network model, the first on-chip memory being connected to each of the plurality of cores by the on-chip network; a second on-chip memory for storing input and output data, the second on-chip memory being connected to each of the plurality of cores by the on-chip network.

Abstract: Massively parallel neural inference computing elements are provided. A plurality of multipliers is arranged in a plurality of equal-sized groups. Each of the plurality of multipliers is adapted to, in parallel, apply a weight to an input activation to generate an output. A plurality of adders is operatively coupled to one of the groups of multipliers. Each of the plurality of adders is adapted to, in parallel, add the outputs of the multipliers within its associated group to generate a partial sum. A plurality of function blocks is operatively coupled to one of the plurality of adders. Each of the plurality of function blocks is adapted to, in parallel, apply a function to the partial sum of its associated adder to generate an output value.

Abstract: Defect resistant designs for location-sensitive neural network processor arrays are provided. In various embodiments, plurality of neural network processor cores are arrayed in a grid. The grid has a plurality of rows and a plurality of columns. A network interconnects at least those of the plurality of neural network processor cores that are adjacent within the grid. The network is adapted to bypass a defective core of the plurality of neural network processor cores by providing a connection between two non-adjacent rows or columns of the grid, and transparently routing messages between the two non-adjacent rows or columns, past the defective core.

Abstract: Systems for distributed, event-based computation are provided. In various embodiments, the systems include a plurality of neurosynaptic processors and a network interconnecting the plurality of neurosynaptic processors. Each neurosynaptic processor includes a clock uncoupled from the clock of each other neurosynaptic processor. Each neurosynaptic processor is adapted to receive an input stream, the input stream comprising a plurality of inputs and a clock value associated with each of the plurality of inputs. Each neurosynaptic processor is adapted to compute, for each clock value, an output based on the inputs associated with that clock value. Each neurosynaptic processor is adapted to send to another of the plurality of neurosynaptic processors, via the network, the output and an associated clock value.

Abstract: A system architecture, a method, and a computer program product are disclosed for attaching remote physical devices. In one embodiment, the system architecture comprises a compute server and a device server. The compute server includes a system memory, and one or more remote device drivers; and the device server includes a system memory and one or more physical devices, and each of the physical devices includes an associated device memory. The compute server and the device server are connected through an existing network fabric that provides remote direct memory access (RDMA) services. A system mapping function logically connects one or more of the physical devices on the device server to the compute server, including mapping between the system memories and the device memories and keeping the system memories and the device memories in synchronization using the RDMA.

Abstract: A method, system and computer program product for providing a guest with access to a virtual storage on a physical storage using a peripheral component interface hub. In one embodiment, the method comprises the guest sending to the peripheral component interface hub a request to access the physical storage, the request including physical addresses of the physical storage, and the peripheral component interface hub sending specified information about the request to a hypervisor. This method further comprises the hypervisor determining whether to grant or to reject the request; and when the hypervisor grants the request, the hypervisor sending a configuration command to the peripheral component interface hub. This command includes a mapping of addresses from the physical storage to addresses from the virtual storage. In an embodiment, the peripheral component interface hub uses this mapping to replace the addresses in the request with translated virtual addresses.

Abstract: A system architecture, a method, and a computer program product are disclosed for attaching remote physical devices. In one embodiment, the system architecture comprises a compute server and a device server. The compute server includes a system memory, and one or more remote device drivers; and the device server includes a system memory and one or more physical devices, and each of the physical devices includes an associated device memory. The compute server and the device server are connected through an existing network fabric that provides remote direct memory access (RDMA) services. A system mapping function logically connects one or more of the physical devices on the device server to the compute server, including mapping between the system memories and the device memories and keeping the system memories and the device memories in synchronization using the RDMA.

Abstract: A system architecture, a method, and a computer program product are disclosed for attaching remote physical devices. In one embodiment, the system architecture comprises a compute server and a device server. The compute server includes a system memory, and one or more remote device drivers; and the device server includes a system memory and one or more physical devices, and each of the physical devices includes an associated device memory. The compute server and the device server are connected through an existing network fabric that provides remote direct memory access (RDMA) services. A system mapping function logically connects one or more of the physical devices on the device server to the compute server, including mapping between the system memories and the device memories and keeping the system memories and the device memories in synchronization using the RDMA.

Abstract: A method, system and computer program product for providing a guest with access to a virtual storage on a physical storage using a peripheral component interface hub. In one embodiment, the method comprises the guest sending to the peripheral component interface hub a request to access the physical storage, the request including physical addresses of the physical storage, and the peripheral component interface hub sending specified information about the request to a hypervisor. This method further comprises the hypervisor determining whether to grant or to reject the request; and when the hypervisor grants the request, the hypervisor sending a configuration command to the peripheral component interface hub. This command includes a mapping of addresses from the physical storage to addresses from the virtual storage. In an embodiment, the peripheral component interface hub uses this mapping to replace the addresses in the request with translated virtual addresses.

Abstract: A method, system and computer program product are disclosed for direct storage device sharing in a virtualized environment. In an embodiment, the method comprises assigning each of a plurality of virtual functions an associated memory area of a physical memory, and executing the virtual functions in a single root-input/output virtualization environment to provide each of a plurality of guests with direct access to the physical memory. In one embodiment, each of the guests is associated with a respective one of the virtual functions; and the assigning each of the plurality of virtual functions an associated memory area includes maintaining a per-virtual function mapping table identifying a respective one mapping function for each of the virtual functions, and each of the mapping functions mapping one of the memory areas of the physical area to an associated virtual memory.

Abstract: A method, system and computer program product are disclosed for direct storage device sharing in a virtualized environment. In an embodiment, the method comprises assigning each of a plurality of virtual functions an associated memory area of a physical memory, and executing the virtual functions in a single root-input/output virtualization environment to provide each of a plurality of guests with direct access to the physical memory. In one embodiment, each of the guests is associated with a respective one of the virtual functions; and the assigning each of the plurality of virtual functions an associated memory area includes maintaining a per-virtual function mapping table identifying a respective one mapping function for each of the virtual functions, and each of the mapping functions mapping one of the memory areas of the physical area to an associated virtual memory.

Abstract: A method, system and computer program product for providing a guest with access to a virtual storage on a physical storage using a peripheral component interface hub. In one embodiment, the method comprises the guest sending to the peripheral component interface hub a request to access the physical storage, the request including physical addresses of the physical storage, and the peripheral component interface hub sending specified information about the request to a hypervisor. This method further comprises the hypervisor determining whether to grant or to reject the request; and when the hypervisor grants the request, the hypervisor sending a configuration command to the peripheral component interface hub. This command includes a mapping of addresses from the physical storage to addresses from the virtual storage. In an embodiment, the peripheral component interface hub uses this mapping to replace the addresses in the request with translated virtual addresses.