Abstract: Aspects of the disclosure provide an evolutionary neural architecture search (ENAS) method. For example, the ENAS method can include steps (a) performing one or more evolutionary operations on an initial population of neural architectures to generate offspring neural architectures, (b) evaluating performance of each of the offspring neural architectures to obtain at least one evaluation value of the offspring neural architecture with respect to a performance metric, (c) adjusting the evaluation values of the offspring neural architectures based on at least one constraint on the evaluation values, (d) selecting at least one of the offspring neural architectures as a new population of neural architectures, and (e) outputting the new population of neural architectures as a last population of neural architectures when a stopping criterion is achieved, or (f) iterating steps (a) to (d) with the new population of neural architectures being taken as the initial population of neural architectures.

Abstract: Aspects of the present disclosure provide a method for training a predictor that predicts performance of a plurality of machine learning (ML) models on platforms. For example, the method can include converting each of the ML models into a plurality of instructions or the instructions and a plurality of intermediate representations (IRs). The method can also include simulating execution of the instructions corresponding to each of the ML models on a platform and generating instruction performance reports. Each of the instruction performance reports can be associated with performance of the instructions corresponding to one of the ML models that are executed on the platform. The method can also include training the predictor with the instructions or the IRs as learning features and the instruction performance reports as learning labels, compiling the predictor into a library file, and storing the library file in a storage device.

Abstract: An accelerator for neural network computing includes hardware engines and a buffer memory. The hardware engines include a convolution engine and at least a second engine. Each hardware engine includes circuitry to perform neural network operations. The buffer memory stores a first input tile and a second input tile of an input feature map. The second input tile overlaps with the first input tile in the buffer memory. The convolution engine is operative to retrieve the first input tile from the buffer memory, perform convolution operations on the first input tile to generate an intermediate tile of an intermediate feature map, and pass the intermediate tile to the second engine via the buffer memory.

Abstract: A processing unit performs multiply-and-accumulate (MAC) operations on asymmetrically quantized data. The processing unit includes a MAC hardware unit to perform the MAC operations on a first data sequence and a second data sequence to generate an asymmetric MAC output. Both the first data sequence and the second data sequence are asymmetrically quantized. The processing unit further includes an accumulator hardware unit to accumulate the first data sequence concurrently with the MAC operations to generate an accumulated output. The processing unit further includes a multiply-and-add (MAD) hardware unit to multiply the accumulated output with a second offset to generate a multiplication output, and to add the multiplication output, the asymmetric MAC output and a pre-computed value calculated before runtime to generate a final output. The second offset indicates an amount of asymmetry of the second data sequence with respect to zero.

Abstract: A processing unit performs multiply-and-accumulate (MAC) operations on asymmetrically quantized data. The processing unit includes a MAC hardware unit to perform the MAC operations on a first data sequence and a second data sequence to generate an asymmetric MAC output. Both the first data sequence and the second data sequence are asymmetrically quantized. The processing unit further includes an accumulator hardware unit to accumulate the first data sequence concurrently with the MAC operations to generate an accumulated output. The processing unit further includes a multiply-and-add (MAD) hardware unit to multiply the accumulated output with a second offset to generate a multiplication output, and to add the multiplication output, the asymmetric MAC output and a pre-computed value calculated before runtime to generate a final output. The second offset indicates an amount of asymmetry of the second data sequence with respect to zero.

Abstract: An accelerator for neural network computing includes hardware engines and a buffer memory. The hardware engines include a convolution engine and at least a second engine. Each hardware engine includes circuitry to perform neural network operations. The buffer memory stores a first input tile and a second input tile of an input feature map. The second input tile overlaps with the first input tile in the buffer memory. The convolution engine is operative to retrieve the first input tile from the buffer memory, perform convolution operations on the first input tile to generate an intermediate tile of an intermediate feature map, and pass the intermediate tile to the second engine via the buffer memory.