Abstract: Techniques for executing machine learning (ML) models including receiving an indication to run a ML model, receiving synchronization information for organizing the running of the ML model with other ML models, determining, based on the synchronization information, to delay running the ML model, delaying the running of the ML model, determining, based on the synchronization information, a time to run the ML model; and running the ML model at the time.

Abstract: An Ethernet switch and a switch microcontroller or CPU are integrated onto a system-on-a-chip (SoC). The Ethernet switch remains independently operating at full speed even though the remainder of the SoC is being reset or is otherwise nonoperational. The Ethernet switch is on a separated power and clock domain from the remainder of the integrated SoC. A warm reset signal is trapped by control microcontroller (MCU) to allow the switch CPU to isolate the Ethernet switch and save state. When the Ethernet switch is isolated and operating independently, the warm reset request is provided to the other entities on the integrated SoC. When warm reset is completed, the state is restored and the various DMA and flow settings redeveloped in the integrated SoC to allow return to normal operating condition.

Abstract: A computer vision system is provided that includes a camera capture component configured to capture an image from a camera, a memory, and an image compression decompression engine (ICDE) coupled to the memory and configured to receive each line of the image, and compress each line to generate a compressed bit stream. To compress a line, the ICDE is configured to divide the line into compression units, and compress each compression unit, wherein to compress a compression unit, the ICDE is configured to perform delta prediction on the compression unit to generate a delta predicted compression unit, compress the delta predicted compression unit using exponential Golomb coding to generate a compressed delta predicted compression unit, and add the compressed delta predicted compression unit to the compressed bit stream.

Abstract: A method of de-mosaicing pixel data from an image processor includes generating a pixel block that includes a plurality of image pixels. The method also includes determining a first image gradient between a first set of pixels of the pixel block and a second image gradient between a second set of pixels of the pixel block. The method also includes determining a first adaptive threshold value based on intensity of a third set of pixels of the pixel block. The pixels of the third set of pixels are adjacent to one another. The method also includes filtering the pixel block in a vertical, horizontal, or neutral direction based on the first and second image gradients and the first adaptive threshold value utilizing a plurality of FIR filters to generate a plurality of component images.

Abstract: A hub that receives sensor data streams and then distributes the data streams to the various systems that use the sensor data. A demultiplexer (demux) receives the streams, filters out undesired streams and provides desired streams to the proper multiplexer (mux) or muxes of a series of muxes. Each mux combines received streams and provides an output stream to a respective formatter or output block. The formatter or output block is configured based on the destination of the mux output stream, such as an image signal processor, a processor, memory or external transmission. The output block reformats the received stream to a format appropriate for the recipient and then provides the reformatted stream to that recipient.

Abstract: A video hardware engine which support dynamic frame padding is disclosed. The video hardware engine includes an external memory. The external memory stores a reference frame. The reference frame includes a plurality of reference pixels. A motion estimation (ME) engine receives a current LCU (largest coding unit), and defines a search area around the current LCU for motion estimation. The ME engine receives a set of reference pixels corresponding to the current LCU. The set of reference pixels of the plurality of reference pixels are received from the external memory. The ME engine pads a set of duplicate pixels along an edge of the reference frame when a part area of the search area is outside the reference frame.

Abstract: A method for analyzing images to generate a plurality of output features includes receiving input features of the image and performing Fourier transforms on each input feature. Kernels having coefficients of a plurality of trained features are received and on-the-fly Fourier transforms (OTF-FTs) are performed on the coefficients in the kernels. The output of each Fourier transform and each OTF-FT are multiplied together to generate a plurality of products and each of the products are added to produce one sum for each output feature. Two-dimensional inverse Fourier transforms are performed on each sum.

Abstract: An Ethernet switch and a switch microcontroller or CPU are integrated onto a system-on-a-chip (SoC). The Ethernet switch remains independently operating at full speed even though the remainder of the SoC is being reset or is otherwise nonoperational. The Ethernet switch is on a separated power and clock domain from the remainder of the integrated SoC. A warm reset signal is trapped by control microcontroller (MCU) to allow the switch CPU to isolate the Ethernet switch and save state. When the Ethernet switch is isolated and operating independently, the warm reset request is provided to the other entities on the integrated SoC. When warm reset is completed, the state is restored and the various DMA and flow settings redeveloped in the integrated SoC to allow return to normal operating condition.

Abstract: Several systems and methods for processing of video frames based on one or more video formats are disclosed. In an embodiment, a video processing system comprises a memory and a video engine. The memory stores a plurality of video frames, a primary set of instructions and a plurality of secondary sets of processing instructions. Each secondary set of processing instructions is associated with a video format. The video engine is loaded with the primary set of instructions and is configured to fetch one or more video frames and a secondary set of processing instructions from the memory based on the loaded primary set of instructions. The fetched secondary set of processing instructions corresponds to a video format determined for processing of the one or more video frames. The video engine performs processing of the one or more video frames based on the secondary set of processing instructions.

Abstract: In some examples, a method comprises receiving pixel data from an image capture device having a color filter, wherein the pixel data represents a portion of an image. The method further includes performing wavelet decomposition on the pixel data to produce decomposed pixel data and determining a local intensity of the pixel data. The method also includes determining a noise threshold value based on the local intensity and a noise intensity function that is based on the color filter; determining a noise value for the pixel data based on the decomposed pixel data and the noise threshold value; and correcting the pixel data based on the noise value to produce an output image.

Abstract: A computer vision system is provided that includes a camera capture component configured to capture an image from a camera, a memory, and an image compression decompression engine (ICDE) coupled to the memory and configured to receive each line of the image, and compress each line to generate a compressed bit stream. To compress a line, the ICDE is configured to divide the line into compression units, and compress each compression unit, wherein to compress a compression unit, the ICDE is configured to perform delta prediction on the compression unit to generate a delta predicted compression unit, compress the delta predicted compression unit using exponential Golomb coding to generate a compressed delta predicted compression unit, and add the compressed delta predicted compression unit to the compressed bit stream.

Abstract: A system is provided. In some examples, the system includes a first peripheral circuit and a memory management circuit coupled to the first peripheral circuit. The memory management circuit comprises a first table that associates virtual identification values with address space select values. The system also includes a transaction mapper circuit coupled to the memory management circuit. The transaction mapper circuit comprises a second table that associates virtual identification values with bus-device-function values.

Abstract: A configurable multi-function Peripheral Component Interchange Express (PCIe) endpoint controller, integrated in a system-on-chip (SoC), that exposes multiple functions of multiple processing subsystems (e.g., peripherals) to a host. The SoC may include a centralized transaction tunneling unit and a multi-function interrupt manager. The processing subsystems output data to the host via the centralized transaction tunneling unit, which translates addresses provided by the host to a local address of the SoC. Therefore, the centralized transaction tunneling unit enables those processing subsystems to consume addresses provided by the host without the need for software intervention and software-based translation. The SoC may also provide isolation between each function provided by the processing systems. The multi-function interrupt manager enables the endpoint controller to propagate interrupt messages received from the processing subsystems to the host.

Abstract: A method for processing RGB-Infrared (RGB-IR) sensor data is provided that includes receiving a raw RGB-IR image, determining whether to process the raw RGB-IR image in day mode or night mode, generating, when day mode is determined, an infrared (IR) subtracted raw Bayer image from the raw RGB-IR image and processing the IR subtracted raw Bayer image in an image signal processor (ISP), and generating, when night mode is determined, an IR image from the raw RGB-IR image.

Abstract: A method for local automatic white balance (AWB) of wide dynamic range (WDR) images is provided that includes collecting statistics for local AWB by an image signal processor (ISP) from a first WDR image generated by the ISP, receiving, by the ISP, a plurality of local gain lookup tables (LUTs), one for each color channel, wherein the plurality of local gain LUTs is generated using the statistics, and applying, by the ISP, a gain value to each pixel in a second WDR image generated by the ISP, wherein the gain value for the pixel is determined by the ISP using the local gain LUT for the color channel of the pixel.

Abstract: An apparatus for scaling images is provided that includes at least two input ports, a scaling component coupled to the at least two input ports, the scaling component including a plurality of scalers, the scaling component configurable to map any scaler to any input port of the at least two input ports and configurable to map more than one scaler to any input port, and a memory coupled to the at least two input ports and to outputs of the plurality of scalers, the memory configured to store image data for each input port and scaled image data output by the plurality of scalers.

Abstract: Several systems and methods for processing of video frames based on one or more video formats are disclosed. In an embodiment, a video processing system comprises a memory and a video engine. The memory stores a plurality of video frames, a primary set of instructions and a plurality of secondary sets of processing instructions. Each secondary set of processing instructions is associated with a video format. The video engine is loaded with the primary set of instructions and is configured to fetch one or more video frames and a secondary set of processing instructions from the memory based on the loaded primary set of instructions. The fetched secondary set of processing instructions corresponds to a video format determined for processing of the one or more video frames. The video engine performs processing of the one or more video frames based on the secondary set of processing instructions.

Abstract: In some examples, a method comprises receiving pixel data from an image capture device having a color filter, wherein the pixel data represents a portion of an image. The method further includes performing wavelet decomposition on the pixel data to produce decomposed pixel data and determining a local intensity of the pixel data. The method also includes determining a noise threshold value based on the local intensity and a noise intensity function that is based on the color filter; determining a noise value for the pixel data based on the decomposed pixel data and the noise threshold value; and correcting the pixel data based on the noise value to produce an output image.

Abstract: An electronic device, comprising: a first component configured to transmit a first set of data to a second component by providing a first memory request specifying the first set of data for and an input memory address, and a transaction tracking unit coupled to a first transport interface, the transaction tracking unit configured to: receive the first memory request; transmit a second memory request that specifies at least a first portion of the first set of data, via the first transport interface, to the second component; receive a response to the second memory request from the second component; determine that the response corresponds to the second memory request; and provide, to the first component, an output response based on the received response to the second memory request.

Abstract: A configurable multi-function Peripheral Component Interchange Express (PCIe) endpoint controller, integrated in a system-on-chip (SoC), that exposes multiple functions of multiple processing subsystems (e.g., peripherals) to a host. The SoC may include a centralized transaction tunneling unit and a multi-function interrupt manager. The processing subsystems output data to the host via the centralized transaction tunneling unit, which translates addresses provided by the host to a local address of the SoC. Therefore, the centralized transaction tunneling unit enables those processing subsystems to consume addresses provided by the host without the need for software intervention and software-based translation. The SoC may also provide isolation between each function provided by the processing systems. The multi-function interrupt manager enables the endpoint controller to propagate interrupt messages received from the processing subsystems to the host.