Abstract: The disclosed technology relates to automatically optimizing the precision of data types in a computational graph, such as those used in machine learning and artificial intelligence applications. A representation of the computational graph is obtained. Nodes of the computational graph are assigned to one of three sets: a deny set, an allow set, or an infer set, based on a predefined policy. For nodes in the allow set, the method changes at least one of the input data precision, output data precision, or internal computation precision to a lower precision. For nodes in the infer set, the method propagates a data precision requirement from downstream nodes to upstream nodes. The method generates and stores computer instructions for executing the computational graph with the optimized precisions on one or more processors. This approach enhances performance and energy efficiency while maintaining model accuracy for the computational graph.

Abstract: DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each memory address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.

Abstract: A compiler tool and a method of operating a compiler tool for selecting between executing a matrix multiplication operation in a weight stationary mode or in an output stationary mode on a systolic array with reconfigurable processing elements are presented. The compiler tool receives configuration parameters, energy parameters, and performance parameters as well as dimensions of the matrices to be multiplied and estimates energy consumptions and performance numbers for executing the matrix multiplication operation in the weight stationary and the output stationary modes. The compiler tool selects between operating the matrix multiplication operation in the weight stationary and the output stationary mode based on the estimated energy consumption and the estimated performance numbers.

Abstract: A reconfigurable processing element for a systolic array that is configurable for multiplying a first matrix with a second matrix to determine a result matrix in a weight stationary mode or in an output stationary mode is presented. Furthermore, a systolic array for performing matrix multiplication of a first matrix and a second matrix to determine a result matrix is presented that includes a plurality of reconfigurable processing elements that are configurable for operating in a weight stationary mode or in an output stationary mode. Moreover, a method of operating a reconfigurable processing element for a systolic array that is configured for performing matrix multiplication of a first matrix and a second matrix to determine a result matrix in a weight stationary mode or in an output stationary mode is presented.

Abstract: A convolution calculation engine includes a kernel element counter for a convolution operation between a kernel and an input tensor. The kernel element counter wraps back to an initial kernel count value after reaching a maximum kernel count value. The convolution calculation engine also includes an offset look-up table (LUT) that provides a relative input offset into the input tensor based on an output of the kernel element counter and input location calculation logic that provides an input location within an input tensor for the convolution operation based on the relative input offset provided by the offset LUT.

Abstract: A convolution calculation engine to perform a convolution operation includes a convolution address compute unit. The convolution address compute unit includes an outer output base location register to provide an outer output base location for the convolution operation and an outer input base location register to provide an outer input base location for the convolution operation. It also includes a kernel element counter that starts to count from an initial kernel count value to a maximum kernel count value in response to a change in the outer output base location and a kernel offset generator to generate a kernel offset based on an output of the kernel element counter. In addition, the convolution address compute unit includes inner location logic to calculate an output location based on the outer output base location and an input location based on the outer input base location and output of the kernel element counter.

Abstract: DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each memory address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.

Abstract: DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each memory address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.

Abstract: DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each memory address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.

Abstract: DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each memory address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.

Abstract: DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each memory address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.