Abstract: A data processor is disclosed that includes a processing unit operable to process neural network data, and a cache system operable to cache neural network data for the processing unit. When neural network data is required for processing, the processing unit issues a request for the neural network data to the cache system, and if the requested data is not cached in the cache system, a compression codec is caused to decode a part of a compressed neural network data stream that encodes the requested neural network data so as to provide the requested neural network data.

Abstract: A computer implemented method of storing and retrieving feature map data of a neural network the method comprising receiving a first portion of feature map data from local storage, selecting a first set of subportions of the first portion of feature map data, compressing the subportions to produce a first plurality of sections of compressed feature map data and instructing the storage of the sections into external storage. The method also comprises receiving a second plurality of sections of compressed feature map data from the external storage, decompressing the sections to produce a second set of subportions of the second portion of feature map data and storing the second portion of feature map data in local storage. The first and second sets of subportions each correspond to a predetermined format of subdivision and the method comprises selecting the predetermined format of subdivision from a plurality of predetermined formats of subdivision.

Abstract: A computer implemented method, performed in a data processing system comprising a performance monitoring unit. The method comprises receiving a set of computer-readable instructions to be executed by the data processing system to implement at least a portion of a neural network, wherein one or more of the instructions is labeled with one or more performance monitoring labels based upon one or more features of the neural network. The method further comprises configuring the performance monitoring unit to count one or more events occurring in one or more components of the data processing system based on the one or more performance monitoring labels.

Abstract: A neural processing unit comprises an input module for receiving a transaction from at least one program, each program has an associated program privilege level; and a plurality of delegation pages, each delegation page comprising a delegation management unit associated with a page privilege level. The neural processing unit also comprises at least one resource arranged to be accessed by at least one of the delegation pages; and a processing module arranged to process the transaction. Processing the transactions comprises allocating each transaction to a delegation page based on the program privilege level and page privilege level. The program is arranged to instruct the delegation management unit of a first delegation page, having a first-page privilege level to delegate access to the at least one resource to a second delegation page having a second-page privilege level, and wherein the first-page privilege level is higher than the second-page privilege level.

Abstract: Aspects of the present disclosure relate to apparatus comprising execution circuitry comprising at least one execution unit to execute program instructions, and control circuitry. The control circuitry receives a stream of processing instructions, and issues each received instruction to one of said at least one execution unit. Responsive to determining that a first type of context switch is to be performed from an initial context to a new context, issuing continues until a pre-emption point in the stream of processing instructions is reached. Responsive to reaching the pre-emption point, state information is stored, and the new context is switched to. Responsive to determining that a context switch is to be performed to return from the new context to the initial context, the processing status is restored from the state information, and the stream of processing instructions is continued.

Abstract: A method of performing fault detection during computations relating to a neural network comprising a first neural network layer and a second neural network layer in a data processing system, the method comprising: scheduling computations onto data processing resources for the execution of the first neural network layer and the second neural network layer, wherein the scheduling includes: for a given one of the first neural network layer and the second neural network layer, scheduling a respective given one of a first computation and a second computation as a non-duplicated computation, in which the given computation is at least initially scheduled to be performed only once during the execution of the given neural network layer; and for the other of the first and second neural network layers, scheduling the respective other of the first and second computations as a duplicated computation.

Abstract: There is provided a neural processing unit (NPU), including a primary processing node containing primary control registers and processing circuitry configured to write control data to the primary control registers, and multiple secondary processing nodes each having respective secondary control registers and being configured to process data in accordance with control data stored by the respective secondary control registers. The NPU also includes a bus interface for transmitting data between the primary processing node and the plurality of secondary processing nodes. The primary processing node is configured to transmit first control data to a given secondary control register of each of the plurality of secondary processing nodes.

Abstract: A computer implemented method of storing and retrieving feature map data of a neural network the method comprising receiving a first portion of feature map data from local storage, selecting a first set of subportions of the first portion of feature map data, compressing the subportions to produce a first plurality of sections of compressed feature map data and instructing the storage of the sections into external storage. The method also comprises receiving a second plurality of sections of compressed feature map data from the external storage, decompressing the sections to produce a second set of subportions of the second portion of feature map data and storing the second portion of feature map data in local storage. The first and second sets of subportions each correspond to a predetermined format of subdivision and the method comprises selecting the predetermined format of subdivision from a plurality of predetermined formats of subdivision.

Abstract: A computer implemented method, performed in a data processing system comprising a performance monitoring unit. The method comprises receiving a set of computer-readable instructions to be executed by the data processing system to implement at least a portion of a neural network, wherein one or more of the instructions is labeled with one or more performance monitoring labels based upon one or more features of the neural network. The method further comprises configuring the performance monitoring unit to count one or more events occurring in one or more components of the data processing system based on the one or more performance monitoring labels.

Abstract: A neural processing unit comprises an input module for receiving a transaction from at least one program, each program has an associated program privilege level; and a plurality of delegation pages, each delegation page comprising a delegation management unit associated with a page privilege level. The neural processing unit also comprises at least one resource arranged to be accessed by at least one of the delegation pages; and a processing module arranged to process the transaction. Processing the transactions comprises allocating each transaction to a delegation page based on the program privilege level and page privilege level. The program is arranged to instruct the delegation management unit of a first delegation page, having a first-page privilege level to delegate access to the at least one resource to a second delegation page having a second-page privilege level, and wherein the first-page privilege level is higher than the second-page privilege level.

Abstract: There is provided a neural processing unit (NPU), including a primary processing node containing primary control registers and processing circuitry configured to write control data to the primary control registers, and multiple secondary processing nodes each having respective secondary control registers and being configured to process data in accordance with control data stored by the respective secondary control registers. The NPU also includes a bus interface for transmitting data between the primary processing node and the plurality of secondary processing nodes. The primary processing node is configured to transmit first control data to a given secondary control register of each of the plurality of secondary processing nodes.