Abstract: In one example, a neural network processor comprises an input data register, a weights register, a computing engine configurable to perform multiplication and accumulation (MAC) operations between input data elements of a range of input precisions and weight elements of a range of weight precisions, and a controller. The controller is configured to: receive a first indication of the particular input precision and a second indication of the particular weight precision, and configure the computing engine based on the first and second indications. The controller is also configured to, responsive to an instruction: fetch input data elements and weight elements to the computing engine; and perform, using the computing engine configured based on the first and second indications, MAC operations between the input data elements at the particular input precision and the weight elements at the particular weight precision to generate intermediate output data elements.

Abstract: In one example, a neural network processor comprises a memory interface, an instruction buffer, a weights buffer, an input data register, a weights register, an output data register, a computing engine, and a controller. The controller is configured to: receive a first instruction from the instruction buffer; responsive to the first instruction, fetch input data elements from the memory interface to the input data register, and fetch weight elements from the weights buffer to the weights register. The controller is also configured to: receive a second instruction from the instruction buffer; and responsive to the second instruction: fetch the input data elements and the weight elements from, respectively, the input data register and the weights register to the computing engine; and perform, using the computing engine, computation operations between the input data elements and the weight elements to generate output data elements.

Abstract: In one example, a neural network processor comprises a computing engine and a post-processing engine, the post-processing engine configurable to perform different post-processing operations for a range of output precisions and a range of weight precisions. The neural network processor further comprises a controller configured to: receive a first indication of a particular output precision, a second indication of the particular weight precision, and post-processing parameters; and configure the post-processing engine based on the first and second indications and the first and second post-processing parameters. The controller is further configured to, responsive to a first instruction, perform, using the computing engine, multiplication and accumulation operations between input data elements and weight elements to generate intermediate data elements.

Abstract: Systems and articles of manufacture provide an efficient safety mechanism for signal processing hardware. An example system includes a hardware accelerators, including a first hardware accelerator, and a second hardware accelerator coupled to the first hardware accelerator. Each of the first and second hardware accelerators includes a protected memory and an unprotected memory, and at least one of the hardware accelerators has an outlier filter. The system also includes a memory coupled to the hardware accelerators; and interface protectors, including a first interface protector coupled between the first hardware accelerator and the memory; a second interface protector coupled between the first hardware accelerator, the memory, and the second hardware accelerator; and a third interface protector coupled between the second hardware accelerator and the memory.

Abstract: An apparatus and method for geometrically correcting an arbitrary shaped input frame and generating an undistorted output frame. The method includes capturing arbitrary shaped input images with multiple optical devices and processing the images, identifying redundant blocks and valid blocks in each of the images, allocating an output frame with an output frame size and dividing the output frame into regions shaped as a rectangle, programming the apparatus and disabling processing for invalid blocks in each of the regions, fetching data corresponding to each of the valid blocks and storing in an internal memory, interpolating data for each of the regions with stitching and composing the valid blocks for the output frame and displaying the output frame on a display module.

Abstract: Methods, apparatus, and articles of manufacture providing an efficient safety mechanism for signal processing hardware are disclosed. An example apparatus includes an input interface to receive an input signal; a hardware accelerator to process the input signal, the hardware accelerator including: unprotected memory to store non-critical data corresponding to the input signal; and protected memory to store critical data corresponding to the input signal; and an output interface to transmit the processed input signal.

Abstract: A system to implement debugging for a multi-threaded processor is provided. The system includes a hardware thread scheduler configured to schedule processing of data, and a plurality of schedulers, each configured to schedule a given pipeline for processing instructions. The system further includes a debug control configured to control at least one of the plurality of schedulers to halt, step, or resume the given pipeline of the at least one of the plurality of schedulers for the data to enable debugging thereof. The system further includes a plurality of hardware accelerators configured to implement a series of tasks in accordance with a schedule provided by a respective scheduler in accordance with a command from the debug control. Each of the plurality of hardware accelerators is coupled to at least one of the plurality of schedulers to execute the instructions for the given pipeline and to a shared memory.

Abstract: A system to implement debugging for a multi-threaded processor is provided. The system includes a hardware thread scheduler configured to schedule processing of data, and a plurality of schedulers, each configured to schedule a given pipeline for processing instructions. The system further includes a debug control configured to control at least one of the plurality of schedulers to halt, step, or resume the given pipeline of the at least one of the plurality of schedulers for the data to enable debugging thereof. The system further includes a plurality of hardware accelerators configured to implement a series of tasks in accordance with a schedule provided by a respective scheduler in accordance with a command from the debug control. Each of the plurality of hardware accelerators is coupled to at least one of the plurality of schedulers to execute the instructions for the given pipeline and to a shared memory.

Abstract: Systems include data processors to process a set of image data in parallel, and thread schedulers coupled to the data processors. Each of the thread schedulers provides a respective task start signal for a respective data processor. Such systems also include a bandwidth controller coupled to one or more data processors. The bandwidth controller is configured to, for each of the data processor(s): maintain a respective token count, and determine whether to stall or propagate the respective task start signal from the respective thread scheduler to the data processor based on the respective token count. Other aspects include pattern adaptors respectively provided in the schedulers to allow mixing of multiple data patterns across blocks of data, transaction aggregators that allow re-using the same image data by multiple threads of execution while the image data remains in a given data buffer, and timers to detect failure and hang events.

Abstract: A computer vision system is provided that includes an image generation device configured to capture consecutive two dimensional (2D) images of a scene, a first memory configured to store the consecutive 2D images, a second memory configured to store a growing window of consecutive rows of a reference image and a growing window of consecutive rows of a current image, wherein the reference image and the current image are a pair of consecutive 2D images stored in the first memory, a third memory configured to store a sliding window of pixels fetched from the growing window of the reference image, wherein the pixels in the sliding window are stored in tiles, and a dense optical flow engine (DOFE) configured to determine a dense optical flow map for the pair of consecutive 2D images, wherein the DOFE uses the sliding window as a search window for pixel correspondence searches.

Abstract: A hardware thread scheduler (HTS) is provided for a multiprocessor system. The HTS is configured to schedule processing of multiple threads of execution by resolving data dependencies between producer modules and consumer modules for each thread. Pattern adaptors may be provided in the scheduler that allows mixing of multiple data patterns across blocks of data. Transaction aggregators may be provided that allow re-using the same image data by multiple threads of execution while the image data remains in a given data buffer. Bandwidth control may be provided using programmable delays on initiation of thread execution. Failure and hang detection may be provided using multiple watchdog timers.

Abstract: A hardware thread scheduler (HTS) is provided for a multiprocessor system. The HTS is configured to schedule processing of multiple threads of execution by resolving data dependencies between producer modules and consumer modules for each thread. Pattern adaptors may be provided in the scheduler that allows mixing of multiple data patterns across blocks of data. Transaction aggregators may be provided that allow re-using the same image data by multiple threads of execution while the image date remains in a given data buffer. Bandwidth control may be provided using programmable delays on initiation of thread execution. Failure and hang detection may be provided using multiple watchdog timers.

Abstract: A computer vision system is provided that includes an image generation device configured to generate consecutive two dimensional (2D) images of a scene, and a dense optical flow engine (DOFE) configured to determine a dense optical flow map for pairs of the consecutive 2D images, wherein, for a pair of consecutive 2D images, the DOFE is configured to perform a predictor based correspondence search for each paxel in a current image of the pair of consecutive 2D images, wherein, for an anchor pixel in each paxel, the predictor based correspondence search evaluates a plurality of predictors to select a best matching pixel in a reference image of the pair of consecutive 2D images, and determine optical flow vectors for each pixel in a paxel based on the best matching pixel selected for the anchor pixel of the paxel.

Abstract: An apparatus and method for geometrically correcting a distorted input frame and generating an undistorted output frame. The apparatus includes an external memory block that stores the input frame, a counter block to compute output coordinates of the output frame for a region based on a block size of the region, a back mapping block to generate input coordinates corresponding to each of the output coordinates, a bounding module to compute input blocks corresponding to each of the input coordinates, a buffer module to fetch data corresponding to each of the input blocks, an interpolation module to interpolate data from the buffer module and a display module that receives the interpolated data for each of the regions and stitch an output image. The method includes determining the size of the output block based on a magnification data.

Abstract: A video hardware engine which support dynamic frame padding is disclosed. The video hardware engine includes an external memory. The external memory stores a reference frame. The reference frame includes a plurality of reference pixels. A motion estimation (ME) engine receives a current LCU (largest coding unit), and defines a search area around the current LCU for motion estimation. The ME engine receives a set of reference pixels corresponding to the current LCU. The set of reference pixels of the plurality of reference pixels are received from the external memory. The ME engine pads a set of duplicate pixels along an edge of the reference frame when a part area of the search area is outside the reference frame.

Abstract: A method for identifying regions of interest (ROIs) includes receiving, by a processor from a video camera, a video image and computing, by the processor, an optical flow image, based on the video image. The method also includes computing, by the processor, a magnitude of optical flow image based on the video image and computing a histogram of optical flow magnitudes (HOFM) image for the video image based on the magnitude of optical flow image. Additionally, the method includes generating, by the processor, a mask indicating ROIs of the video image, based on the HOFM.

Abstract: A video hardware engine which support dynamic frame padding is disclosed. The video hardware engine includes an external memory. The external memory stores a reference frame. The reference frame includes a plurality of reference pixels. A motion estimation (ME) engine receives a current LCU (largest coding unit), and defines a search area around the current LCU for motion estimation. The ME engine receives a set of reference pixels corresponding to the current LCU. The set of reference pixels of the plurality of reference pixels are received from the external memory. The ME engine pads a set of duplicate pixels along an edge of the reference frame when a part area of the search area is outside the reference frame.

Abstract: A method for identifying regions of interest (ROIs) includes receiving, by a processor from a video camera, a video image and computing, by the processor, an optical flow image, based on the video image. The method also includes computing, by the processor, a magnitude of optical flow image based on the video image and computing a histogram of optical flow magnitudes (HOFM) image for the video image based on the magnitude of optical flow image. Additionally, the method includes generating, by the processor, a mask indicating ROIs of the video image, based on the HOFM.

Abstract: An apparatus for scaling images is provided that includes at least two input ports, a scaling component coupled to the at least two input ports, the scaling component including a plurality of scalers, the scaling component configurable to map any scaler to any input port of the at least two input ports and configurable to map more than one scaler to any input port, and a memory coupled to the at least two input ports and to outputs of the plurality of scalers, the memory configured to store image data for each input port and scaled image data output by the plurality of scalers.

Abstract: A processing accelerator includes a shared memory, and a stream accelerator, a memory-to-memory accelerator, and a common DMA controller coupled to the shared memory. The stream accelerator is configured to process a real-time data stream, and to store stream accelerator output data generated by processing the real-time data stream in the shared memory. The memory-to-memory accelerator is configured to retrieve input data from the shared memory, to process the input data, and to store, in the shared memory, memory-to-memory accelerator output data generated by processing the input data. The common DMA controller is configured to retrieve stream accelerator output data from the shared memory and transfer the stream accelerator output data to memory external to the processing accelerator; and to retrieve the memory-to-memory accelerator output data from the shared memory and transfer the memory-to-memory accelerator output data to memory external to the processing accelerator.