Abstract: A processor-implemented method for a neural architecture search (NAS) starts by generating an over-parameterized super network having multiple layers. The super network has multiple operator types. Each of the layers includes a largest super kernel corresponding to a search space. The method also includes performing gradient descent to evolve a largest super kernel to a small kernel corresponding to the search space in order to generate a range of kernel encodings. The method further includes identifying a subset of kernel encodings from the range of kernel encodings, for each layer of the super network, based on the gradient descent. The method determines a set of candidate architectures based on the subset of kernel encodings, each of the candidate architectures having a different model size. The method selects a target model, from the set of architectures, based on meeting hardware specifications, and then applies the target model.

Abstract: A method and system for pruning a neural network (NN) block of a neural network during training, wherein the NN block comprises: a convolution operation configured to convolve an input feature map with a plurality of filters, each filter including a plurality of weights, to generate a plurality of filter outputs each corresponding to a respective filter; an activation operation configured to generate, for each of the filter outputs, a respective non-linearized output; a scaling operation configured to scale the non-linearized output generated in respect of each filter by multiplying the non-linearized output with a mask function and a respective scaling factor that corresponds to the filter.