Abstract: A convolutional neural network (CNN) system is provided that includes a flexible accelerator configured to convert an input feature map into a set of input sub-feature maps, each having a similar amount of sparsity. The system allows each of the sub-feature maps to be processed independently while taking advantage of the sparsity. In some aspects, the CNN system is configured with an index processor that receives data value indexes and weight indexes and generates data path processor commands for processing by a separate data path processor. In other aspects, unroll circuitry is configured to unroll feature maps to provide index-value compression. The unroll/compression scheme allows an input feature map to be read sequentially (tile-by-tile) so that an accumulate buffer can be implemented with a single read-only path and single write-only path. This can simplify memory control design, eliminating requirements for expensive cache-like structures while also reducing power.