Abstract: A method of constructing an adaptive multiply accumulate layer in a convolutional neural network, including determining an activation data map width, an activation data map height, a channel depth, a batch, a kernel width, a kernel height and a filter set number, setting a first dimension of an adaptive multiplier layer based on the activation data map width, setting a second dimension of the adaptive multiplier layer based on the channel depth, setting a third dimension of the adaptive multiplier layer based on the filter set number and constructing the adaptive multiplier layer based on the first dimension, the second dimension and the third dimension.

Abstract: The present invention relates to convolution neural networks (CNN) and methods for improving computational efficiency of multiply accumulate (MAC) array structure Specifically, the invention relates to cutting of activation data into a number of tiles for increasing overall computation efficiency. The invention discloses techniques to cut an activation data into a plurality of tiles by using a 3-D convolution computation core and support bigger tensor sizes. Lastly, the invention provides adaptive scheduling of MAC array to achieve high utilization in multi-precision neural network acceleration.

Abstract: A system on a chip, including a first domain having a first processor, a first local memory coupled to the first processor, wherein the first local memory having a first memory format and a first sub-network coupled to the first processor, a second domain having a second processor, a second local memory coupled to the second processor and a second sub-network coupled to the second processor, wherein the second local memory having a second memory format which differs from the first memory format, a multi-tier network coupled to the first sub-network and the second sub-network, a global memory coupled to the multi-tier network and a multi-port DDR controller coupled to the global memory to receive, transmit and share the first local memory having the first memory format and the second local memory having the second memory format based on a predetermined criteria.

Abstract: The invention discloses a method and a system for achieving a unified flow control system for multiple camera devices. The system provides inline and offline streams to share resources by converting the streams into one another. The resource sharing is performed by using different time interval to process inline and offline streams. The system also includes a STALL & REDO operation to keep whole image un-broken and shut down the write stream from ISP right away.

Abstract: A method of subgraph tile fusion in a convolutional neural network, including partitioning a network into at least one subgraph node, determining a layer order of at least one layer of the at least one subgraph node, determining a input layer of the at least one subgraph node, determining a weight layer of the at least one subgraph node, determining a output layer of the at least one subgraph node and fusing the at least one subgraph node, the input layer, the weight layer and the output layer in the layer order.

Abstract: A method of hierarchical structuring a multi-level memory in a convolutional neural network, includes partitioning a memory into a plurality of sections, partitioning the plurality of sections into a plurality of stripes, utilizing input data from the plurality of stripes in a MAC array, outputting an intermediate result from the MAC array to at least one of the plurality of stripes of a result buffer, looping back the intermediate result from the at least one of the plurality of stripes of the result buffer to at least one of the plurality of stripes of an input data buffer and outputting a final result from the at least one of the plurality of stripes of the result buffer to at least one of the plurality of stripes of an output buffer.

Abstract: A system on a chip, including a first domain having a first processor, a first local memory coupled to the first processor, wherein the first local memory having a first memory format and a first sub-network coupled to the first processor, a second domain having a second processor, a second local memory coupled to the second processor and a second sub-network coupled to the second processor, wherein the second local memory having a second memory format which differs from the first memory format, a multi-tier network coupled to the first sub-network and the second sub-network, a global memory coupled to the multi-tier network and a multi-port DDR controller coupled to the global memory to receive, transmit and share the first local memory having the first memory format and the second local memory having the second memory format based on a predetermined criteria.

Abstract: A system on a chip, including a multi-port memory controller having a multi-level memory hierarchy, a multi-tier bus coupled to the multi-port memory controller to segregate memory access traffic based on the multi-level memory hierarchy, an interconnected plurality of networks on chip coupled to the multi-tier bus, a plurality of networked domains coupled to the plurality of networks on chip and at least one non-networked domain coupled directly to the multi-port memory controller.

Abstract: A method of hierarchical structuring a multi-level memory in a convolutional neural network, includes partitioning a memory into a plurality of sections, partitioning the plurality of sections into a plurality of stripes, utilizing input data from the plurality of stripes in a MAC array, outputting an intermediate result from the MAC array to at least one of the plurality of stripes of a result buffer, looping back the intermediate result from the at least one of the plurality of stripes of the result buffer to at least one of the plurality of stripes of an input data buffer and outputting a final result from the at least one of the plurality of stripes of the result buffer to at least one of the plurality of stripes of an output buffer.

Abstract: A method of constructing an adaptive multiply accumulate layer in a convolutional neural network, including determining an activation data map width, an activation data map height, a channel depth, a batch, a kernel width, a kernel height and a filter set number, setting a first dimension of an adaptive multiplier layer based on the activation data map width, setting a second dimension of the adaptive multiplier layer based on the channel depth, setting a third dimension of the adaptive multiplier layer based on the filter set number and constructing the adaptive multiplier layer based on the first dimension, the second dimension and the third dimension.

Abstract: A method of subgraph tile fusion in a convolutional neural network, including partitioning a network into at least one subgraph node, determining a layer order of at least one layer of the at least one subgraph node, determining a input layer of the at least one subgraph node, determining a weight layer of the at least one subgraph node, determining a output layer of the at least one subgraph node and fusing the at least one subgraph node, the input layer, the weight layer and the output layer in the layer order.

Abstract: An apparatus and method for correcting for distortion in distorted digital data for a distorted image to produce corrected digital data for a corrected image partitions the distorted digital data into a plurality of distorted data blocks. Each distorted data block is separately encoded into an encoded distorted data block. A plurality of corrected regions of the corrected image is defined, each corrected region being associated with a respective corrected data block. For each corrected data block, a plurality of associated encoded distorted data blocks is identified, the plurality of associated encoded distorted data blocks is decoded into a respective plurality of associated decoded distorted data blocks, and corrected image data for the corrected data block are generated using the associated decoded distorted data blocks.

Abstract: An apparatus and method for correcting for distortion in distorted digital data for a distorted image to produce corrected digital data for a corrected image partitions the distorted digital data into a plurality of distorted data blocks. Each distorted data block is separately encoded into an encoded distorted data block. A plurality of corrected regions of the corrected image is defined, each corrected region being associated with a respective corrected data block. For each corrected data block, a plurality of associated encoded distorted data blocks is identified, the plurality of associated encoded distorted data blocks is decoded into a respective plurality of associated decoded distorted data blocks, and corrected image data for the corrected data block are generated using the associated decoded distorted data blocks.