Abstract: Computer implemented methods, systems, and computer program products include program code executing on a processor(s) which initiate, from a first chip in a computing system, a request to utilize an accelerator in the computing system to process work associated with a given workload. The program code determines that a local accelerator on the first chip is not available to accept a dispatch of the work. The program code obtains hardware monitoring data from local hardware counters associated with various elements of the computing system. The program code determines, based on the hardware monitoring data, a best remote accelerator for dispatching the work to by selecting an accelerator that utilizes bandwidth and one or more caches which are minimally accessed by other workloads processing in the computing system concurrently with the given workload when compared the other accelerators comprising the one or more remote accelerators.

Abstract: Dimension control in tensor multiplication includes obtaining first and second input tensors for matrix multiplication, obtaining a dimension control indicator that indicates a first dimension for the first input tensor to use as a common dimension for the matrix multiplication and indicates a second dimension for the second input tensor to use as the common dimension for the matrix multiplication, and performing the matrix multiplication to obtain one or more results, where performing the matrix multiplication includes selecting at least one vector of the first input tensor based on the first dimension indicated by the dimension control indicator and selecting at least one vector of the second input tensor based on the second dimension indicated by the dimension control indicator.

Abstract: Tensor multiplication with quantization includes obtaining first and second input tensors, obtaining elements of a selected data type based on elements of the first input tensor and elements of the second input tensor, performing matrix multiplication on the elements of the selected data type, the matrix multiplication including performing quantization of intermediate results, and the quantization scaling the intermediate results to provide scaled results of the matrix multiplication, and generating output elements, for an output tensor, using the scaled results. Optional additional quantization is performed on elements of an input tensor to provide at least some of the elements of the selected data type for the matrix multiplication.

Abstract: Tensor processing includes obtaining an input tensor, the input tensor including a dimension of index size n, determining an element count, c, based on an indicator, the indicator specified by the instruction, and the element count specifying a number of vector elements on which to perform an artificial intelligence function, obtaining an input vector, of the input tensor, of size n, and performing the artificial intelligence function, the performing the artificial intelligence function including performing the artificial intelligence function on a first c number of elements of the input vector to provide a corresponding c number of elements of an output vector of index size n of an output tensor.

Abstract: Tensor processing with broadcasting includes obtaining a first input tensor, the first input tensor including a dimension of index size one, and performing an artificial intelligence processing operation using the first input tensor and a second input tensor to obtain results, the second input tensor including a dimension of index size greater than one, where performing the artificial intelligence processing operation includes broadcasting the dimension of index size one of the first input tensor to match the dimension of index size greater than one of the second input tensor.