Abstract: An AI (Artificial Intelligence) processor for Neural Network (NN) Processing shared by multiple users is disclosed. The AI processor comprises a Multiplier Unit (MXU), a Scalar Computing Unit (SCU), a unified buffer coupled to the MXU and SCU to store data and a control circuitry coupled to the CCU and the unified buffer. The MXU comprises a plurality of Processing Elements (PEs) responsible for computing matrix multiplications. The SCU coupled to output of the MXU is responsible for computing the activation function. The control circuitry is configured to perform the space division and time division NN processing for a plurality of users. At one time instance, at least one of the MXU and SCU is shared by two or more users; and at least one user is using a part of the MXU while the other user is using a part of the SCU.

Abstract: Embodiments described herein provide a mission-critical artificial intelligence (AI) processor (MAIP), which includes multiple types of HEs (hardware elements) comprising one or more HEs configured to perform operations associated with multi-layer NN (neural network) processing, at least one spare HE, a data buffer to store correctly computed data in a previous layer of multi-layer NN processing computed, and fault tolerance (FT) control logic. The FT control logic is configured to: determine a fault in a current layer NN processing associated with the HE; cause the correctly computed data in the previous layer of multi-layer NN processing to be copied or moved to said at least one spare HE; and cause said at least one spare HE to perform the current layer NN processing using said at least one spare HE and the correctly computed data in the previous layer of multi-layer NN processing.

Abstract: Embodiments described herein provide a mission-critical artificial intelligence (AI) processor (MAIP), which includes an instruction buffer, processing circuitry, a data buffer, command circuitry, and communication circuitry. During operation, the instruction buffer stores a first hardware instruction and a second hardware instruction. The processing circuitry executes the first hardware instruction, which computes an intermediate stage of an AI model. The data buffer stores data generated from executing the first hardware instruction. The command circuitry determines that the second hardware instruction is a hardware-initiated store instruction for transferring the data from the data buffer. Based on the hardware-initiated store instruction, the communication circuitry transfers the data from the data buffer to a memory device of a computing system, which includes the mission-critical processor, via a communication interface.

Abstract: A scalar element computing device for computing a selected activation function selected from two or more different activation functions is disclosed. The scalar element computing device comprises N processing elements, N command memories and an operator pool. The N processing elements are arranged into a pipeline to cause the outputs of each non-last-stage processing element coupled to the inputs of one next-stage processing element. The N command memories are coupled to the N processing elements individually. The operator pool is coupled to the N processing elements, where the operator pool comprises a set of operators for implementing any activation function in an activation function group. The N processing elements are configured according to command information stored in the N command memories to calculate a target activation function selected from the activation function group by using one or more operators in the set of operations.

Abstract: Embodiments described herein provide a mission-critical artificial intelligence (AI) processor (MAIP), which includes an instruction buffer, processing circuitry, a data buffer, command circuitry, and communication circuitry. During operation, the instruction buffer stores a first hardware instruction and a second hardware instruction. The processing circuitry executes the first hardware instruction, which computes an intermediate stage of an AI model. The data buffer stores data generated from executing the first hardware instruction. The command circuitry determines that the second hardware instruction is a hardware-initiated store instruction for transferring the data from the data buffer. Based on the hardware-initiated store instruction, the communication circuitry transfers the data from the data buffer to a memory device of a computing system, which includes the mission-critical processor, via a communication interface.