Abstract: A hardware thread scheduler (HTS) is provided for a multiprocessor system. The HTS is configured to schedule processing of multiple threads of execution by resolving data dependencies between producer modules and consumer modules for each thread. Pattern adaptors may be provided in the scheduler that allows mixing of multiple data patterns across blocks of data. Transaction aggregators may be provided that allow re-using the same image data by multiple threads of execution while the image date remains in a given data buffer. Bandwidth control may be provided using programmable delays on initiation of thread execution. Failure and hang detection may be provided using multiple watchdog timers.

Abstract: A method for filtering noise for imaging includes receiving an image frame having position and range data. A filter size divides the frame into filter windows for processing each of the filter windows. For the first pixel, a space to the center pixel and a range difference between this pixel and the center pixel is determined and used for choosing a selected weight from weights in a 2D weight LUT including weighting for space and range difference, a filtered range value is calculated by applying the selected 2D weight to the pixel, and the range, filtered range value and selected 2D weight are summed. The determining, choosing, calculating and summing are repeated for at least the second pixel. A total sum of contributions from the first and second pixel are divided by the sum of selected 2D weights to generate a final filtered range value for the center pixel.

Abstract: An optical flow system includes a binary mask generation circuit and an optical flow circuit. The binary mask generation circuit is configured to receive a plurality of points of interest from a captured image that contains an array of pixels arranged as rows and columns and includes width lines that correspond to the rows and height lines that correspond to the columns. The binary mask generation circuit is also configured to generate a binary mask based on the plurality of points of interest. The binary mask includes a representation of a subset of the plurality of points of interest. The optical flow circuit is configured to receive the binary mask and generate an optical flow map of the subset of the plurality of points of interest.

Abstract: An apparatus and method for geometrically correcting an arbitrary shaped input frame and generating an undistorted output frame. The method includes capturing arbitrary shaped input images with multiple optical devices and processing the images, identifying redundant blocks and valid blocks in each of the images, allocating an output frame with an output frame size and dividing the output frame into regions shaped as a rectangle, programming the apparatus and disabling processing for invalid blocks in each of the regions, fetching data corresponding to each of the valid blocks and storing in an internal memory, interpolating data for each of the regions with stitching and composing the valid blocks for the output frame and displaying the output frame on a display module.

Abstract: A method for geometrically correcting a distorted input frame and generating an undistorted output frame includes capturing and storing an input frame in an external memory, allocating an output frame with an output frame size and dividing the output frame into output blocks, computing a size of the input blocks in the input image corresponding to each output blocks, checking if the size of the input blocks is less than the size of the internal memory and if not dividing until the required input block size of divided sub blocks is less than the size of the internal memory, programming an apparatus with input parameters, fetching the input blocks into an internal memory, processing each of the divided sub blocks sequentially and processing the next output block in step until all the output blocks are processed; and composing the output frame for each of the blocks in the output frame.

Abstract: An apparatus and method for geometrically correcting a distorted input frame and generating an undistorted output frame. The apparatus includes an external memory block that stores the input frame, a counter block to compute output coordinates of the output frame for a region based on a block size of the region, a back mapping block to generate input coordinates corresponding to each of the output coordinates, a bounding module to compute input blocks corresponding to each of the input coordinates, a buffer module to fetch data corresponding to each of the input blocks, an interpolation module to interpolate data from the buffer module and a display module that receives the interpolated data for each of the regions and stitch an output image. The method includes determining the size of the output block based on a magnification data.

Abstract: A computer vision system is provided that includes a camera capture component configured to capture an image from a camera, a memory, and an image compression decompression engine (ICDE) coupled to the memory and configured to receive each line of the image, and compress each line to generate a compressed bit stream. To compress a line, the ICDE is configured to divide the line into compression units, and compress each compression unit, wherein to compress a compression unit, the ICDE is configured to perform delta prediction on the compression unit to generate a delta predicted compression unit, compress the delta predicted compression unit using exponential Golomb coding to generate a compressed delta predicted compression unit, and add the compressed delta predicted compression unit to the compressed bit stream.

Abstract: A computer vision system is provided that includes an image generation device configured to capture consecutive two dimensional (2D) images of a scene, a first memory configured to store the consecutive 2D images, a second memory configured to store a growing window of consecutive rows of a reference image and a growing window of consecutive rows of a current image, wherein the reference image and the current image are a pair of consecutive 2D images stored in the first memory, a third memory configured to store a sliding window of pixels fetched from the growing window of the reference image, wherein the pixels in the sliding window are stored in tiles, and a dense optical flow engine (DOFE) configured to determine a dense optical flow map for the pair of consecutive 2D images, wherein the DOFE uses the sliding window as a search window for pixel correspondence searches.

Abstract: A CNN based-signal processing includes receiving of an encrypted output from a first layer of a multi-layer CNN data. The received encrypted output is subsequently decrypted to form a decrypted input to a second layer of the multi-layer CNN data. A convolution of the decrypted input with a corresponding decrypted weight may generate a second layer output, which may be encrypted and used as an encrypted input to a third layer of the multi-layer CNN data.

Abstract: A computer vision system is provided that includes an image generation device configured to generate consecutive two dimensional (2D) images of a scene, and a dense optical flow engine (DOFE) configured to determine a dense optical flow map for pairs of the consecutive 2D images, wherein, for a pair of consecutive 2D images, the DOFE is configured to perform a predictor based correspondence search for each paxel in a current image of the pair of consecutive 2D images, wherein, for an anchor pixel in each paxel, the predictor based correspondence search evaluates a plurality of predictors to select a best matching pixel in a reference image of the pair of consecutive 2D images, and determine optical flow vectors for each pixel in a paxel based on the best matching pixel selected for the anchor pixel of the paxel.

Abstract: An outer product multiplier (GPM) system/method that integrates compute gating and input/output circular column rotation functions to balance time spent in compute and data transfer operations while limiting overall dynamic power dissipation is disclosed. Matrix compute gating (MCG) based on a computation decision matrix (CDM) limits the number of computations required on a per cycle basis to reduce overall matrix compute cycle power dissipation. A circular column rotation vector (CRV) automates input/output data formatting to reduce the number of data transfer operations required to achieve a given matrix computation result. Matrix function operators (MFO) utilizing these features are disclosed and include: matrix-matrix multiplication; matrix-matrix and vector-vector point-wise multiplication, addition, and assignment; matrix-vector multiplication; vector-vector inner product; matrix transpose; matrix row permute; and vector-column permute.

Abstract: A method for analyzing images to generate a plurality of output features includes receiving input features of the image and performing Fourier transforms on each input feature. Kernels having coefficients of a plurality of trained features are received and on-the-fly Fourier transforms (OTF-FTs) are performed on the coefficients in the kernels. The output of each Fourier transform and each OTF-FT are multiplied together to generate a plurality of products and each of the products are added to produce one sum for each output feature. Two-dimensional inverse Fourier transforms are performed on each sum.

Abstract: A method for filtering noise for imaging includes receiving an image frame having position and range data. A filter size divides the frame into filter windows for processing each of the filter windows. For the first pixel, a space to the center pixel and a range difference between this pixel and the center pixel is determined and used for choosing a selected weight from weights in a 2D weight LUT including weighting for space and range difference, a filtered range value is calculated by applying the selected 2D weight to the pixel, and the range, filtered range value and selected 2D weight are summed. The determining, choosing, calculating and summing are repeated for at least the second pixel. A total sum of contributions from the first and second pixel are divided by the sum of selected 2D weights to generate a final filtered range value for the center pixel.

Abstract: A method for analyzing images to generate a plurality of output features includes receiving input features of the image and performing Fourier transforms on each input feature. Kernels having coefficients of a plurality of trained features are received and on-the-fly Fourier transforms (OTF-FTs) are performed on the coefficients in the kernels. The output of each Fourier transform and each OTF-FT are multiplied together to generate a plurality of products and each of the products are added to produce one sum for each output feature. Two-dimensional inverse Fourier transforms are performed on each sum.

Abstract: The disclosure provides a sample adaptive offset (SAO) encoder. The SAO encoder includes a statistics collection (SC) block and a rate distortion optimization (RDO) block coupled to the SC block. The SC block receives a set of deblocked pixels and a set of original pixels. The SC block categorizes each deblocked pixel of the set of deblocked pixels in at least one of a plurality of band and edge categories. The SC block estimates an error in each category as difference between a deblocked pixel of the set of deblocked pixels and corresponding original pixel of the set of original pixels. The RDO block determines a set of candidate offsets associated with each category and selects a candidate offset with a minimum RD cost. The minimum RD cost is used by a SAO type block and a decision block to generate final offsets for the SAO encoder.

Abstract: Described examples include an integrated circuit including a vector multiply unit including a plurality of multiply/accumulate nodes, in which the vector multiply unit is operable to provide an output from the multiply/accumulate nodes, a first data feeder operable to provide first data to the vector multiply unit in vector format, and a second data feeder operable to provide second data to the vector multiply unit in vector format.

Abstract: A data processing device is provided that includes a plurality of hardware data processing nodes, wherein each hardware data processing node is configured to execute a task, and a hardware thread scheduler coupled to the plurality of hardware data processing nodes, the hardware thread scheduler configurable to concurrently execute a first thread of tasks and a second thread of tasks on the plurality of hardware data processing nodes.

Abstract: An image signal processor includes a first matrix processing circuit, a post processing circuit, a second matrix processing circuit, and a split visual and analytics circuit. The first matrix processing circuit is configured to receive a plurality of component images generated based on an image captured by an image sensor and generate a plurality of first matrix outputs based on the plurality of component images. The post processing circuit is configured to perform color conversion on the plurality of first matrix outputs to generate a first luminance component of the image and a chrominance component of the image. The second matrix processing circuit is configured to perform color conversion on the plurality of first matrix outputs to generate a second luminance component of the image and a saturation component of the image. The split visual and analytics circuit is configured to generate visual and analytic data of the image.

Abstract: A method of de-blocking filtering a processed video is provided. The processed video includes a plurality of blocks and each block includes a plurality of sub-blocks. A current block of the plurality of blocks includes vertical edges and horizontal edges. The processed video further includes a set of control parameters and reconstructed pixels corresponding to the current block. A boundary strength index is estimated at the vertical edges and at the horizontal edges of the current block. The set of control parameters, the reconstructed pixels corresponding to the current block and partially filtered pixels corresponding to a set of adjacent sub-blocks are loaded. The vertical edges and the horizontal edges of the current block are filtered based on the boundary strength index and the set of control parameters such that a vertical edge of the current block is filtered before filtering at least one horizontal edge of the current block.

Abstract: A data processing device is provided that includes a plurality of hardware data processing nodes, wherein each hardware data processing node performs a task, and a hardware thread scheduler including a plurality of hardware task schedulers configured to control execution of a respective task on a respective hardware data processing node of the plurality of hardware data processing nodes, and a proxy hardware task scheduler coupled to a data processing node external to the data processing device, wherein the proxy hardware task scheduler is configured to control execution of a task by the external data processing device, wherein the hardware thread scheduler is configurable to execute a thread of tasks, the tasks including the task controlled by the proxy hardware task scheduler and a first task controlled by a first hardware task scheduler of the plurality of hardware task schedulers.