Abstract: A network of matrix processing units (MPUs) is provided on a device, where each MPU is connected to at least one other MPU in the network, and each MPU is to perform matrix multiplication operations. Computer memory stores tensor data and a master control central processing unit (MCC) is provided on the device to receive an instruction from a host device, where the instruction includes one or more tensor operands based on the tensor data. The MCC invokes a set of operations on one or more of the MPUs based on the instruction, where the set of operations includes operations on the tensor operands. A result is generated from the set of operations, the result embodied as a tensor value.

Abstract: In one embodiment, a matrix processor comprises a memory to store a matrix operand and a strided read sequence, wherein: the matrix operand is stored out of order in the memory; and the strided read sequence comprises a sequence of read operations to read the matrix operand in a correct order from the memory. The matrix processor further comprises circuitry to: receive a first instruction to be executed by the matrix processor, wherein the first instruction is to instruct the matrix processor to perform a first operation on the matrix operand; read the matrix operand from the memory based on the strided read sequence; and execute the first instruction by performing the first operation on the matrix operand.

Abstract: A network of matrix processing units (MPUs) is provided on a device, where each MPU is connected to at least one other MPU in the network, and each MPU is to perform matrix multiplication operations. Computer memory stores tensor data and a master control central processing unit (MCC) is provided on the device to receive an instruction from a host device, where the instruction includes one or more tensor operands based on the tensor data. The MCC invokes a set of operations on one or more of the MPUs based on the instruction, where the set of operations includes operations on the tensor operands. A result is generated from the set of operations, the result embodied as a tensor value.

Abstract: In one embodiment, a matrix operation may be performed using a plurality of input matrices, wherein the matrix operation is associated with one or more convolution operations. The plurality of input matrices may be partitioned into a plurality of input partitions, wherein the plurality of input matrices is partitioned based on a number of available processing elements. The plurality of input partitions may be distributed among a plurality of processing elements, wherein each input partition is distributed to a particular processing element of the plurality of processing elements. A plurality of partial matrix operations may be performed using the plurality of processing elements, and partial matrix data may be transmitted between the plurality of processing elements while performing the plurality of partial matrix operations. A result of the matrix operation may be determined based on the plurality of partial matrix operations.

Abstract: In one embodiment, a matrix operation may be performed to reorder a plurality of dimensions of an input matrix stored in two-dimensional memory. Data associated with the input matrix may be accessed using one or more strided memory operations, wherein the one or more strided memory operations are configured to access the two-dimensional memory at a plurality of locations that are separated by a particular interval. The data accessed using the one or more strided memory operations may be stored in a result matrix, wherein the data accessed using each strided memory operation is stored in the result matrix in non-transpose form or transpose form.

Abstract: In one embodiment, a matrix operation may be performed on one or more matrix operands. For example, matrix data may be received from a multi-dimensional memory, wherein the matrix data is associated with the one or more matrix operands. The one or more matrix operands may be extracted from the matrix data. A matrix routine associated with the matrix operation may be identified. The matrix routine may be executed on a matrix processor using the one or more matrix operands. A result of the matrix operation may be obtained based on the matrix routine executed by the matrix processor.

Abstract: In one embodiment, a matrix processor comprises a memory to store a matrix operand and a strided read sequence, wherein: the matrix operand is stored out of order in the memory; and the strided read sequence comprises a sequence of read operations to read the matrix operand in a correct order from the memory. The matrix processor further comprises circuitry to: receive a first instruction to be executed by the matrix processor, wherein the first instruction is to instruct the matrix processor to perform a first operation on the matrix operand; read the matrix operand from the memory based on the strided read sequence; and execute the first instruction by performing the first operation on the matrix operand.

Abstract: In one embodiment, a matrix operation may be performed, wherein the matrix operation comprises a matrix multiplication operation on a plurality of matrix operands. Matrix data may be received from a multi-dimensional memory, wherein the matrix data is associated with the plurality of matrix operands. The plurality of matrix operands may be extracted from the matrix data, wherein the plurality of matrix operands comprises a first matrix operand and a second matrix operand. A first transform may be performed on the first matrix operand to obtain a transformed matrix operand, wherein performing matrix multiplication using the transformed matrix operand is faster than performing matrix multiplication using the first matrix operand. Matrix multiplication may be performed on the transformed matrix operand to obtain a partial result. A second transform may be performed on the partial result to obtain a result of the matrix multiplication operation.

Abstract: In one embodiment, an apparatus comprises a multi-dimensional memory and a plurality of processing elements to perform a matrix operation, wherein the matrix operation comprises a max pooling operation on one or more matrix operands. The plurality of processing elements comprises one or more matrix processors, and the plurality of processing elements is configured to: receive matrix data from the multi-dimensional memory, wherein the matrix data is associated with the one or more matrix operands; extract the one or more matrix operands from the matrix data; perform the max pooling operation using the one or more matrix operands; and obtain a result of the max pooling operation.

Abstract: In one embodiment, a matrix operation may be performed on one or more matrix operands. For example, matrix data may be received from a multi-dimensional memory, wherein the matrix data is associated with the one or more matrix operands. The one or more matrix operands may be extracted from the matrix data. A matrix routine associated with the matrix operation may be identified. The matrix routine may be executed on a matrix processor using the one or more matrix operands. A result of the matrix operation may be obtained based on the matrix routine executed by the matrix processor.

Abstract: An apparatus may comprise a multi-dimensional memory, a plurality of matrix processors, and a matrix routine memory. The matrix routine memory may store a plurality of programmable matrix routines, wherein each programmable matrix routine comprises a plurality of instructions associated with a particular matrix operation, wherein the plurality of instructions is to be executed by the plurality of matrix processors. Further, the plurality of matrix processors may be configured to: receive a command to perform a matrix operation; receive matrix data from the multi-dimensional memory; extract one or more matrix operands from the matrix data; identify a programmable matrix routine associated with the matrix operation; receive the programmable matrix routine from the matrix routine memory; execute the programmable matrix routine using the one or more matrix operands; and obtain a result of the matrix operation based on execution of the programmable matrix routine.

Abstract: In one embodiment, an apparatus comprises a multi-dimensional memory and a plurality of processing elements to perform a matrix operation, wherein the matrix operation comprises a max pooling operation on one or more matrix operands. The plurality of processing elements comprises one or more matrix processors, and the plurality of processing elements is configured to: receive matrix data from the multi-dimensional memory, wherein the matrix data is associated with the one or more matrix operands; extract the one or more matrix operands from the matrix data; perform the max pooling operation using the one or more matrix operands; and obtain a result of the max pooling operation.

Abstract: In one embodiment, a matrix operation may be performed on one or more matrix operands. For example, matrix data may be received from a multi-dimensional memory, wherein the matrix data is associated with the one or more matrix operands. The one or more matrix operands may be extracted from the matrix data. A matrix routine associated with the matrix operation may be identified. The matrix routine may be executed on a matrix processor using the one or more matrix operands. A result of the matrix operation may be obtained based on the matrix routine executed by the matrix processor.

Abstract: In one embodiment, an apparatus comprises a memory and a memory controller. The memory comprises a plurality of memory modules, wherein each memory module comprises a plurality of storage locations. The memory controller may be configured to write data of a matrix to the memory. For example, the memory controller may be configured to write a particular row or a particular column of the matrix to the memory by: shifting a plurality of matrix elements of the particular row or the particular column; and writing the plurality of matrix elements to the plurality of memory modules.

Abstract: In one embodiment, a matrix operation may be performed using a plurality of input matrices, wherein the matrix operation is associated with one or more convolution operations. The plurality of input matrices may be partitioned into a plurality of input partitions, wherein the plurality of input matrices is partitioned based on a number of available processing elements. The plurality of input partitions may be distributed among a plurality of processing elements, wherein each input partition is distributed to a particular processing element of the plurality of processing elements. A plurality of partial matrix operations may be performed using the plurality of processing elements, and partial matrix data may be transmitted between the plurality of processing elements while performing the plurality of partial matrix operations. A result of the matrix operation may be determined based on the plurality of partial matrix operations.

Abstract: In one embodiment, a matrix operation may be performed to reorder a plurality of dimensions of an input matrix stored in two-dimensional memory. Data associated with the input matrix may be accessed using one or more strided memory operations, wherein the one or more strided memory operations are configured to access the two-dimensional memory at a plurality of locations that are separated by a particular interval. The data accessed using the one or more strided memory operations may be stored in a result matrix, wherein the data accessed using each strided memory operation is stored in the result matrix in non-transpose form or transpose form.

Abstract: In one embodiment, a matrix operation may be performed, wherein the matrix operation comprises a matrix multiplication operation on a plurality of matrix operands. Matrix data may be received from a multi-dimensional memory, wherein the matrix data is associated with the plurality of matrix operands. The plurality of matrix operands may be extracted from the matrix data, wherein the plurality of matrix operands comprises a first matrix operand and a second matrix operand. A first transform may be performed on the first matrix operand to obtain a transformed matrix operand, wherein performing matrix multiplication using the transformed matrix operand is faster than performing matrix multiplication using the first matrix operand. Matrix multiplication may be performed on the transformed matrix operand to obtain a partial result. A second transform may be performed on the partial result to obtain a result of the matrix multiplication operation.

Abstract: In one embodiment, an apparatus comprises a multi-dimensional memory and a plurality of processing elements to perform a matrix operation, wherein the matrix operation comprises a max pooling operation on one or more matrix operands. The plurality of processing elements comprises one or more matrix processors, and the plurality of processing elements is configured to: receive matrix data from the multi-dimensional memory, wherein the matrix data is associated with the one or more matrix operands; extract the one or more matrix operands from the matrix data; perform the max pooling operation using the one or more matrix operands; and obtain a result of the max pooling operation.

Abstract: A cache system has a multilevel cache architecture. An L1 cache receives instructions from an external memory. An L0 cache has a first predetermined number L0 of cache lines for receiving instructions from the L1 cache. An assist cache has a victim cache and a prefetch cache. The victim cache stores a second predetermined number VC of cache lines, and the prefetch cache stores a third predetermined number PC of cache lines. The victim cache receives instructions from the L0 cache. The prefetch cache receives instructions from the L1 cache. A victim filter stores a fourth predetermined number VF of addresses wherein VF is a function of L0 and a number of cache writes. The number of cache writes to the L0 and the victim cache is reduced relative to using the L0 cache without the assist cache.

Abstract: A processor having an execution pipeline and a cache memory including a plurality of cache blocks with instruction words held in selected ones of the cache blocks. An ICBI address buffer is provided for holding addresses of instruction cache blocks to be invalidated by ICBI instructions pending in the processor's execution pipeline. An instruction cache controller coupled to the cache memory generates cache accesses to invalidate specified cache blocks in response to receiving buffered addresses from the ICBI address buffer. Preferably the cache accesses serve to commit ICBI instructions to the instruction cache asynchronously with respect to the processor's execution pipeline.