Abstract: A video hardware engine with multi-threading functionality is disclosed. The video hardware engine includes a video hardware accelerator unit and a controller. The controller is coupled to the video hardware accelerator unit. The controller operates in an encode mode and a decode mode. In the encode mode, the controller receives a plurality of frames and encode attributes associated with each frame of the plurality of frames. The encode attributes associated with a frame of the plurality of frames is processed to generate encode parameters associated with the frame. The video hardware accelerator unit is configured to process the frame based on the encode parameters to generate an output. The output of the video hardware accelerator unit is processed to generate a compressed bit-stream and an encode status. In decode mode, the controller receives a compressed bit-stream and decode attributes and generates a plurality of frames and a decode status.

Abstract: Systems include data processors to process a set of image data in parallel, and thread schedulers coupled to the data processors. Each of the thread schedulers provides a respective task start signal for a respective data processor. Such systems also include a bandwidth controller coupled to one or more data processors. The bandwidth controller is configured to, for each of the data processor(s): maintain a respective token count, and determine whether to stall or propagate the respective task start signal from the respective thread scheduler to the data processor based on the respective token count. Other aspects include pattern adaptors respectively provided in the schedulers to allow mixing of multiple data patterns across blocks of data, transaction aggregators that allow re-using the same image data by multiple threads of execution while the image data remains in a given data buffer, and timers to detect failure and hang events.

Abstract: This disclosure relates to various implementations an embedded computing system. The embedded computing system comprises a hardware accelerator (HWA) thread user and a second HWA thread user that creates and sends out message requests. The HWA thread user and the second HWA thread user is communication with a microcontroller (MCU) subsystem. The embedded computing system also comprises a first inter-processor communication (IPC) interface between the HWA thread user and the MCU subsystem and a second IPC interface between the second HWA thread user and the MCU subsystem, where the first IPC interface is isolated from the second IPC interface. The MCU subsystem is also in communication with a first domain specific HWA and a second domain specific HWA.

Abstract: An electronic device may be configured to detect a fault in imaging and vision hardware accelerators. The electronic device may include a controller configured to select a first golden input frame of multiple golden input frames to perform a first self-test, and retrieve a first reference image signature corresponding to the first golden input frame. The electronic device may include a hardware accelerator module configured to obtain the first golden input frame, and generate a first output frame based on the first golden input frame. The electronic device may include a signature generator configured to generate a first generated image signature based on the first output frame. The electronic device may include a signature comparison module configured to compare the first generated image signature to the first reference image signature in order to determine whether the hardware accelerator module includes a fault at a first time.

Abstract: A system-on-chip (SoC) may be configured to enable a Multi-Chip Daisy Chain Topology using peripheral component interface express (PCIe). The SoC may include a processor, a local memory, a root complex operably connected to the processor and the local memory, and a multi-function endpoint controller. The root complex may obtain forwarding information to configure routing of transactions to one or more PCIe endpoint functions or to the local memory. The root complex may initialize, based on the forwarding information, access between a host and the one or more PCIe endpoint functions. The multi-function endpoint controller may obtain a descriptor and endpoint information to configure outbound portals for transactions to at least one remote host. The multi-function endpoint controller may establish a communication path between the host and a function out of a plurality of functions.

Abstract: Local automatic white balance (AWB) of wide dynamic range (WDR) images is provided. Methods and systems include collecting, by an image signal processor (ISP), statistics for local AWB from at least one wide dynamic range (WDR) image received by the ISP; generating, by a processor, based on the statistics, local gain lookup tables (LUTs), one for each color channel represented in the WDR image(s), each local gain LUT providing a correlation between gain and intensity; and storing the local gain LUTs. Further processing includes, for each of multiple pixels of a WDR image to be output calculating an intensity value, accessing the local gain LUT for the color channel corresponding to that pixel using the calculated intensity value to identify a corresponding local gain value, and applying the local gain value to that pixel.

Abstract: A hardware neural network engine which uses checksums of the matrices used to perform the neural network computations. For fault correction, expected checksums are compared with checksums computed from the matrix developed from the matrix operation. The expected checksums are developed from the prior stage of the matrix operations or from the prior stage of the matrix operations combined with the input matrices to a matrix operation. This use of checksums allows reading of the matrices from memory, the dot product of the matrices and the accumulation of the matrices to be fault corrected without triplication of the matrix operation hardware and extensive use of error correcting codes. The nonlinear stage of the neural network computation is done using triplicated nonlinear computational logic. Fault detection is done in a similar manner, with fewer checksums needed and correction logic removed as compared to the fault correction operation.

Abstract: A method of de-blocking filtering a processed video is provided. The processed video includes a plurality of blocks and each block includes a plurality of sub-blocks. A current block of the plurality of blocks includes vertical edges and horizontal edges. The processed video further includes a set of control parameters and reconstructed pixels corresponding to the current block. A boundary strength index is estimated at the vertical edges and at the horizontal edges of the current block. The set of control parameters, the reconstructed pixels corresponding to the current block and partially filtered pixels corresponding to a set of adjacent sub-blocks are loaded. The vertical edges and the horizontal edges of the current block are filtered based on the boundary strength index and the set of control parameters such that a vertical edge of the current block is filtered before filtering at least one horizontal edge of the current block.

Abstract: A configurable multi-function Peripheral Component Interchange Express (PCIe) endpoint controller, integrated in a system-on-chip (SoC), that exposes multiple functions of multiple processing subsystems (e.g., peripherals) to a host. The SoC may include a centralized transaction tunneling unit and a multi-function interrupt manager. The processing subsystems output data to the host via the centralized transaction tunneling unit, which translates addresses provided by the host to a local address of the SoC. Therefore, the centralized transaction tunneling unit enables those processing subsystems to consume addresses provided by the host without the need for software intervention and software-based translation. The SoC may also provide isolation between each function provided by the processing systems. The multi-function interrupt manager enables the endpoint controller to propagate interrupt messages received from the processing subsystems to the host.

Abstract: A peripheral proxy subsystem is placed between multiple hosts, each having a root controller, and single root I/O virtualization (SR-IOV) peripheral devices that are to be shared. The peripheral proxy subsystem provides a root controller for coupling to the endpoint of the SR-IOV peripheral device or devices and multiple endpoints for coupling to the root controllers of the hosts. The peripheral proxy subsystem maps the virtual functions of an SR-IOV peripheral device to the multiple endpoints as desired to allow the virtual functions to be allocated to the hosts. The physical function of the SR-IOV peripheral device is managed by the peripheral proxy device to provide the desired number of virtual functions. The virtual functions of the SR-IOV peripheral device are then presented to the appropriate host as a physical function or a virtual function.

Abstract: In an example, a device includes a memory and a processor core coupled to the memory via a memory management unit (MMU). The device also includes a system MMU (SMMU) cross-referencing virtual addresses (VAs) with intermediate physical addresses (IPAs) and IPAs with physical addresses (PAs). The device further includes a physical address table (PAT) cross-referencing IPAs with each other and cross-referencing PAs with each other. The device also includes a peripheral virtualization unit (PVU) cross-referencing IPAs with PAs, and a routing circuit coupled to the memory, the SMMU, the PAT, and the PVU. The routing circuit is configured to receive a request comprising an address and an attribute and to route the request through at least one of the SMMU, the PAT, or the PVU based on the address and the attribute.

Abstract: A technique including receiving an image stream for processing; processing the received image stream in a real time mode of operation; outputting an indication that an image processing pipeline has begun processing the received image stream; receiving, in response to the indication, first configuration information associated with test data for testing the image processing pipeline; switching the image processing pipeline to a non-real time mode of operation to process the test data based on the first configuration information during a vertical blanking period of the received image stream; loading the test data from an external memory; switching an input of the image processing pipeline from the image stream to the test data; determining a checksum based on the processed test data; comparing the determined checksum to an expected checksum to determine that the test data was successfully processed; and outputting an indication that the test data was successfully processed.

Abstract: Techniques including a memory controller with a set of memory channel queues, wherein memory channel queues of the set of memory channel queues correspond to memory channels to access a set of memory modules, a first arbitration module, and a second arbitration module. The memory controller is configured to receive a first memory request from the peripheral and place one or more portions of the first memory request in the memory channel queues of the set of memory channel queues. The first arbitration module is configured to determine an arbitration algorithm, select a first memory channel queue based on the arbitration algorithm, present the one or more portions of the first memory request in the selected first memory channel queue to the second arbitration module, and output the presented one or more portions of the first memory request based on a selection by the second arbitration module.

Abstract: In the advanced driver-assistance systems (ADAS) field, RAW sensor image processing for machine vision (MV) applications can be of critical importance. Due to red/green/blue (RGB) image components being focused by the lens at different locations in image plane, the lateral chromatic aberration (LCA) phenomenon may sometimes be observed, which causes false color around edges in the final image output, especially for high contrast edges, which can impede MV applications. Disclosed herein are low-latency, efficient, optimized designs for chromatic aberration correction (CAC) modules.

Abstract: A computer vision system is provided that includes an image generation device configured to capture consecutive two dimensional (2D) images of a scene, a first memory configured to store the consecutive 2D images, a second memory configured to store a growing window of consecutive rows of a reference image and a growing window of consecutive rows of a current image, wherein the reference image and the current image are a pair of consecutive 2D images stored in the first memory, a third memory configured to store a sliding window of pixels fetched from the growing window of the reference image, wherein the pixels in the sliding window are stored in tiles, and a dense optical flow engine (DOFE) configured to determine a dense optical flow map for the pair of consecutive 2D images, wherein the DOFE uses the sliding window as a search window for pixel correspondence searches.

Abstract: A method of enabling memory access freedom from interference (FFI) rules, comprising: determining a first safety privilege access ID (PrivID) for a first component of a system (e.g., based on Automotive Safety Integrity Level (ASIL) attributes of tasks executed by the first component); determining a first access attribute for a first software task executing on the first component; receiving, at a first firewall component of the system, a request from the first software task to access a first memory region of a second component of the system, wherein the request specifies the first PrivID and the first access attribute; and determining, by the first firewall component, whether to permit the first software task to access the first memory region based on the first PrivID, the first access attribute, and the first memory region.

Abstract: A hardware thread scheduler (HTS) is provided for a multiprocessor system. The HTS is configured to schedule processing of multiple threads of execution by resolving data dependencies between producer modules and consumer modules for each thread. Pattern adaptors may be provided in the scheduler that allows mixing of multiple data patterns across blocks of data. Transaction aggregators may be provided that allow re-using the same image data by multiple threads of execution while the image data remains in a given data buffer. Bandwidth control may be provided using programmable delays on initiation of thread execution. Failure and hang detection may be provided using multiple watchdog timers.

Abstract: Techniques for image processing including receiving input image data, wherein the input image data includes data associated with a clear color channel, receiving a color offset value associated with a color channel, wherein color values for the color channel are not provided in the input image data, based on the color offset value, generating intermediate estimated color values for the color channel, wherein generating the intermediate estimated color values includes: clipping color values that have a magnitude greater than the color offset value, and adjusting color values that have a magnitude less than the color offset value based on the color offset value, applying a color correction function to the intermediate estimated color values based on the color offset value to determine color corrected estimated color values, and outputting the color corrected estimated color values.

Abstract: A technique for rendering an under-vehicle view including obtaining a first location of a vehicle, the vehicle having a set of cameras disposed about the vehicle, capturing a set of images; storing images of the set of images in a memory, wherein the images are associated with a time the images were captured, moving the vehicle to a second location, obtaining the second location of the vehicle, determining an amount of time for moving the vehicle from the first location to the second location, generating a set of motion data, the motion data indicating a relationship between the second location of the vehicle and the first location of the vehicle, obtaining one or more stored images from the memory based on the determined amount of time, rendering a view under the vehicle based on the one or more stored images and set of motion data, and outputting the rendered view.

Abstract: A technique for image processing, comprising: receiving input image data, wherein the image data is companded into a first bit depth, wherein the image data includes incomplete color values for pixels of the image data, and wherein the image data is associated with a first color space, interpolating the image data to generate color values for the incomplete color values for pixels of the image data, expanding the image data from the first bit depth to a second bit depth, wherein the color values of the expanded image data have a linear dynamic range, and wherein the second bit depth is higher than the first bit depth, converting the color values for pixels of the expanded image data from the first color space to a second color space, and compressing the color values for pixels of the image data to a third bit depth, the third bit depth lower than the second bit depth, and wherein the compressed color values have a nonlinear dynamic range.