Abstract: The present application discloses a warp execution method used for SPs of an SM of a GPU and an associated GPU.

Abstract: The present application discloses a GPU and a method of the same. The GPU includes: a plurality of streaming multiprocessor (SMs), each including: a plurality of streaming processors (SPs), each including a register, wherein each SP has a predetermined upper bound of warp number, and the register has a predetermined upper bound of register capacity; and a global dispatcher, including: a register occupancy status table, for recording the warp number and an occupancy status of the register of each SP of each SM; a TB (TB) dispatch module, for dispatching the TB to a first SM of the SMs according to a warp type classification table and the register occupancy status table; and a warp dispatch module, for dispatching a plurality of warps to the plurality of SPs of the first SM according to the warp type classification table and the register occupancy status table.

Abstract: Neural network (NN) model training techniques can include computing activations in a forward pass using a sparse weight matrix that is transpose invariant. The neural network (NN) model training techniques can further include computing activation gradients and weight gradients in a backward pass using the sparse weight matrix.